Teaching People to Fish? Building the Evaluation Capability of Public Sector Organizations

Abstract

In response to an increasing demand for public sector accountability, many government agencies have sought to develop their internal evaluation capabilities. Often these efforts have focused on increasing the capacity to supply credible evaluations, yet addressing demand is just as important. This article focuses on a government agency and tracks its five-year journey towards developing such a capability. It documents contextual matters, drivers for change, the actions taken by the agency, and its response to emergent challenges during four phases. Based on feedback from project staff and managers and those involved in the capability development project, it offers seven recommendations. These are: start small and grow evaluation; address both supply and demand; work top-down and bottom-up simultaneously; use a theory of change behaviour; develop a common evaluation framework, including a generic programme theory; build knowledge of what works within the agency's context; and systematically and visibly evaluate each stage.

Assessing evaluation education in African tertiary education institutions: Opportunities and reflections

“…These tenets include the following: (a) it empowers recipients to understand the supply and demand for evaluations (McDonald et al 2003); (b) it equips recipients with the technical expertise for conducting valid evaluations; (c) it provides professional development through building knowledge, beliefs and skills (Taylor-Powell and Boyd 2008); (d) it takes into account organizational factors that breed or inhibit an "evaluative environment"; and (e) it elucidates the myriad of roles of managers and practitioners in the planning/evaluation cycle, to allow for an understanding of how to integrate the use of evaluation into public service action.…”

“…ECD has in most instances, focused on the supply side (improving technical expertise) at the expense of the demand side i.e. utilization of results (McDonald, Rogers and Kefford 2003).…”

An Impact Evaluation Framework: Local Government Community Festivals

“…Working collaboratively along with community and organizer participation can capitalize on the strengths (Green & South, 2006;Katz & Peberdy, 1997) by providing both "outside" objectivity and "inside" contextual and practitioner know-how. This collaboration and participation is necessary also in order to build evaluation capability in public sector and community organizations (McDonald et al, 2003). Indeed, McDonald et al suggest that only evaluation design and implementation can be done by external parties and that the other evaluation processes of describing, framing, using, and overall management of the evaluation framework must be done internally.…”

“…The purpose of evaluation within the public sector can be categorized as being for accountability, for learning, for program management and development, or as an ethical obligation (McDonald, Rogers, & Kefford, 2003). These are certainly valued reasons for undertaking festival evaluation, and the framework attempts to incorporate these aims within it.…”

The Historical Development of Evaluation Use

“…''Planned process use'' (King, 2007) emerged as a potential means of creating a sustainable organizational process by building on people's engagement in evaluation activities. On the developmental timeline, scholarly discussion of the concept of evaluation capacity building emerged a few years following that of process use (Gilliam et al, 2003;McDonald, Rogers, & Kefford, 2003;Stevenson, Florin, Mills, & Andrade, 2002;Stockdill, Baizerman, & Compton, 2002).…”