Some years ago, I was asked to be the external evaluator on a project involving a small, rural community bordering the Rio Grande river. The community had received funding from the US government to support a goat co-op that would produce products (eg: milk, cheese, meat) that could provide a sustainable livelihood for its members, and also counter the attraction of drug money so prevalent in the area.
The original evaluation plan involved two site visits. On the first visit, I would work out an evaluation plan with the co-op members who would then collect the data over the next 6 months. The second visit would involve data analysis and results. It all sounded reasonable until I arrived for the first visit, only to discover that the two “leaders” (who had written the grant proposal) were at war with each other. Both were non-Spanish speaking ‘anglos’ in a primarily Hispanic setting. I was also taken aside by the model herd manager and told that I, as a woman, could not speak with the men. So, what to do?
Though efforts to mediate the ‘war’ failed, I was able to involve all co-op members in a set of bi-lingual evaluation workshops which yielded some positive results. But in the end, my efforts were undermined locally once I left the site and the co-op dissolved in acrimony. I learned that short, fly in fly out evaluations, so typical of development work, are fraught with pitfalls and subject to local politics that an evaluator cannot control. I concluded that the only way to do this kind of evaluation is to work with a partner who lives in the area, one who is known and trusted locally and can be regularly on site. An outsider can offer expertise and perhaps objectivity, but cannot know the subtleties of local history and relationships and is thus vulnerable to manipulation by those who want to control the outcome.
(The details are published in Whitmore, E. (1998) ‘We need to rebuild this house:’ The role of empowerment in evaluation of a Mexican Farmer’s Cooperative, Chapter 13, in Kassam, Y. and Jackson, E.T. (Eds.). Knowledge Shared: Participatory evaluation in development cooperation. (pp. 217-230) West Hartford, CT: Kumarian Press)
Elizabeth (Bessa) Whitmore
Professor Emerita
Carleton University
