Performance Evaluation in Non-Governmental Organizations (NGOs): An Analysis of Evaluation Models and their Applications in Brazil

Abstract

The objective of this article is to deepen the debate surrounding the practice of evaluation as performed by Brazilian NGOs. In order to accomplish this goal, we examined the results of an investigation conducted between 2007 and 2009 that had two stages: first, a national study and second, a quantitative and qualitative study made in the state of Santa Catarina. We also considered secondary sources of data from two important studies on the national level that were conducted in 2008 and 2009 by the Fonte Institute, with partnership of the Itaú Social Foundation. The results showed that most NGOs use internal rather than external evaluation and conduct evaluations focused on projects and programs. Furthermore, they used evaluation mainly to help decision making about projects. Consequently, the majority of NGOs focused their evaluations on results and were not concerned with creating spaces for self-reflection and learning.Résumé L'objectif de cet article est d'approfondir le débat autour de la pratique de l'évaluation réalisée par des ONG brésiliennes. Pour y parvenir, nous avons examiné les résultats d'une enquête menée entre 2007 et 2009, et qui s'est déroulée en deux étapes : tout d'abord, une étude nationale, ensuite, une étude quantitative et qualitative menée dans l'É tat de Santa Catarina. Nous avons également analysé des sources de données secondaires provenant de deux importantes études réalisées au niveau national, qui ont été conduites en 2008 et 2009 par l'Institut Fonte en partenariat avec la Fondation Itaú Social. Les résultats ont montré que la plupart des ONG utilisent une évaluation interne plutôt qu'externe et procèdent à des évalua-tions axées sur les projets et les programmes. En outre, elles ont principalement utilisé les évaluations pour aider à la prise de décision sur les projets. Par consé-quent, la majorité des ONG ont concentré leurs évaluations sur les résultats et ne se sont pas intéressées à la création d'un espace pour l'autoréflexion et l'apprentissage.Zusammenfassung Ziel dieses Beitrags ist die Vertiefung der Diskussionen über die Bewertungspraktik nicht-staatlicher Organisationen in Brasilien. Dazu betrachteten wir die Ergebnisse einer zwischen 2007 und 2009 in zwei Phasen durchgeführten Untersuchung: sie bestand erstens aus einer landesweiten Studie und zweitens aus einer quantitativen und qualitativen Studie im brasilianischen Bundesstaat Santa Catarina. Des Weiteren untersuchten wir Sekundärdatenquellen zweier wichtiger landesweiter Studien, die 2008 und 2009 vom Fonte Institut in Partnerschaft mit der Sozialstiftung Itaú durchgeführt wurden. Die Ergebnisse zeigten, dass die Mehrheit der nicht-staatlichen Organisationen statt externe, interne Bewertungen durchführen und sich dabei auf Projekte und Programme konzentrieren. Zudem nutzten sie die Bewertungen hauptsächlich zur Entscheidungshilfe bei Projekten. Folglich konzentrierten sich die meisten der nicht-staatlichen Organisationen bei ihren Bewertungen auf die Ergebnisse und befassten sich nicht mit der Schaffung von Spielr...

Exploring the perspectives of International Nongovernmental Organizations (INGOs) on the use of program evaluation and impact assessment in their work

“…In turn, this study found that Organizations A and B are both actively conducting some form of program evaluation and impact assessment, which largely corroborates with the findings from previous studies (Campos et al, 2010;Morley, Vinson & Hatry, 2001;Travers, 2011). It was further found that members of both organizations were well aware of the need to demonstrate the effectiveness of INGO work and generally saw the use of program evaluation and impact assessment as being a positive tool to address this need.…”

“…First, program evaluation can be defined as a systematic process of collecting information and applying approaches, techniques, and knowledge to analyze, research, and assess the activities, characteristics, and outcomes of a program(s) to document its accomplishments and to improve the planning, implementation, and effectiveness of these programs (Alaimo, 2008;Perret, 2009). In INGO work, program evaluation can be conducted before, during, or after the implementation of a program or intervention (Bouchard, 2009;Campos, Andion, Serva, Rossetto, & Assumpção, 2010).…”

Evaluation Logics in the Third Sector

“…All three techniques are well articulated through an assortment of 'how to' guides (detailed below) and there is evidence that the techniques (or those that are very similar) are used within third sector organizations (e.g., see Charities Evaluation Service, 2008;Wallace et al, 2007;Campos et al, 2011;Reed & Morariu, 2010;Carman, 2007;Eckerd & Moulson, 2010;Jacobs et al, 2010;Fine et al, 2000;Hoefer et al, 2000). Whilst the three techniques share some common features, they present quite different approaches to evaluation, particularly concerning the focus on quantitative and qualitative types of data, the level of technical sophistication, the role for consultants and external experts, and the level of participation by different stakeholders.…”

“…In this way, the ideal of belief revision has an outlook that is orientated towards the future, where analysis of what has already happened is premised on making changes to future plans and activities. Here, the evaluation process can construct self-reflective moments that allow individuals to examine the realities they confront, which creates an atmosphere of continual learning (Campos et al, 2011).…”

“…Performance measurement and evaluation is also used for a variety of purposes, such as demonstrating accountability to donors and beneficiaries, analyzing areas of good and bad performance, and promoting the work of third sector organizations to potential funders and a wider public audience (Ebrahim, 2005;Barman, 2007;Roche, 1999;Fine et al, 2000;Hoefer et al, 2000;Campos et al, 2011). This wide range of purposes can result in a field that is often cluttered with new ideas, novel approaches and the latest toolkits (Jacobs et al, 2010).…”