Evaluation2 – Evaluating the national evaluation system in South Africa: What has been achieved in the first 5 years?

Abstract

Background: South Africa has pioneered national evaluation systems (NESs) along with Canada, Mexico, Colombia, Chile, Uganda and Benin. South Africa’s National Evaluation Policy Framework (NEPF) was approved by Cabinet in November 2011. An evaluation of the NES started in September 2016.Objectives: The purpose of the evaluation was to assess whether the NES had had an impact on the programmes and policies evaluated, the departments involved and other key stakeholders; and to determine how the system needs to be strengthened.Method: The evaluation used a theory-based approach, including international benchmarking, five national and four provincial case studies, 112 key informant interviews, a survey with 86 responses and a cost-benefit analysis of a sample of evaluations.Results: Since 2011, 67 national evaluations have been completed or are underway within the NES, covering over $10 billion of government expenditure. Seven of South Africa’s nine provinces have provincial evaluation plans and 68 of 155 national and provincial departments have departmental evaluation plans. Hence, the system has spread widely but there are issues of quality and the time it takes to do evaluations. It was difficult to assess use but from the case studies it did appear that instrumental and process use were widespread. There appears to be a high return on evaluations of between R7 and R10 per rand invested.Conclusion: The NES evaluation recommendations on strengthening the system ranged from legislation to strengthen the mandate, greater resources for the NES, strengthening capacity development, communication and the tracking of use.

Evaluating the evaluators: What have we learned from “neutral assessments” of the Canadian federal evaluation function?

“…They sought to determine the extent to which evaluations were used in policymaking. Second, Goldman and et al (2019) assessed the South African evaluation system and its impacts on the programs and policies evaluated, the departments involved and other key stakeholders. In both cases, the methodology employed was based on a theory-based approach that included multiple lines of inquiry.…”

Parliaments in National Evaluation Systems: Current Practice, Future Potential

“…The last annual report undertaken on the NES was for the period 2016/17 (DPME, 2017), and it summarised the situation at that time, as does the evaluation of the NES conducted in 2016/2017. The report indicated that while many elements were working, the improvement plan system needed to be followed up and tracked more rigorously (Goldman et al, 2019).…”

“…In addition, while it is difficult to assess use of evaluations, case studies revealed that instrumental and process use were prominent. It has been argued that there is a high return on evaluations of between R7 and R13 per rand invested (Goldman et al, 2019).…”

An overview of the provincial evaluation system of the Western Cape Government of South Africa as a response to the evaluation of the National Evaluation System

“…As discussed in Goldman et al (2019), the evaluation made use of a theory-based approach including international benchmarking; five national and four provincial case studies; 112 key informant interviews; a survey with 86 responses; and a cost-benefit analysis of a sample of evaluations. From 2011 to the end of December 2016, 67 national evaluations were…”