Using Root Cause Analysis for Evaluating Program Improvement

Abstract

A common evaluation purpose is to determine whether a policy or program was implemented as intended: referred to as formative evaluation, process evaluation, or evaluating program improvement. A well-designed formative evaluation is important in: detecting program drift; providing timely feedback to program staff to make cost-saving mid-course corrections; reassuring the sponsor that quality assurance measures are implemented to protect investments; and interpreting impact/outcome evaluation. A formative evaluation should not just gather data on deviations from an anticipated course of action, but provide recommendations for improvement. Current methods for program improvement vary in their ability to solicit targeted recommendations. Root cause analysis (RCA) is a well-established, robust methodology used in a variety of disciplines. RCA has been primarily used by evaluators operating from a theory-driven orientation to evaluate the merit and worth of a program or policy. Surprisingly, a review of the literature suggests that RCA's utility as a program improvement tool has remained largely unrecognised in evaluation. This article illustrates the application of RCA in evaluating program improvement. The conditions under which RCA might be preferred over other formative evaluation methods are discussed.

Exploratory study of the underutilization of CTSA module services

“…However, this study addresses topics that are fairly insensitive, one of the reasons why RCA is increasingly recommended as an analysis tool for its utility in program evaluation [21]. Second, with this small sample of investigators and CTSAs we cannot generalize to the populations of investigators using any number of module services or to the population of CTSAs.…”

Lessons learned in evaluating system interdependencies using qualitative methods

“…We asked participants to expand on what we considered vital points for efficiency and interdependencies. If the interviewee flagged a possible system inefficiency, then we conducted a brief verbal root-cause-analysis, drilling down to determine the cause(s) (Coskun et al, 2012). After the interviews, we discussed the quality of interdependencies and plausible recommendations.…”

New Insights for Tracking and Reporting Milestones

“…We presented a snapshot of this information in a pie chart to visualize and compare each core's progress toward their milestones. If a milestone was not completed as expected, we then followed up with the leadership personnel in each core and engaged in root cause analysis (Coşkun, Akande, & Renger, 2012) to determine why progress was halted and work to resolve underlying issues.…”