Ideas for Extending the Approach to Evaluating Health in All Policies in South Australia Comment on "Developing a Framework for a Program Theory-Based Approach to Evaluating Policy Processes and Outcomes: Health in All Policies in South Australia"

Abstract

Since 2008, the government of South Australia has been using a Health in All Policies (HiAP) approach to achieve their strategic plan (South Australia Strategic Plan of 2004). In this commentary, we summarize some of the strengths and contributions of the innovative evaluation framework that was developed by an embedded team of academic researchers. To inform how the use of HiAP is evaluated more generally, we also describe several ideas for extending their approach, including: deeper integration of interdisciplinary theory (eg, public health sciences, policy and political sciences) to make use of existing knowledge and ideas about how and why HiAP works; including a focus on implementation outcomes and using developmental evaluation (DE) partnerships to strengthen the use of HiAP over time; use of systems theory to help understand the complexity of social systems and changing contexts involved in using HiAP; integrating economic considerations into HiAP evaluations to better understand the health, social and economic benefits and trade-offs of using HiAP.

What Can Policy-Makers Get Out of Systems Thinking? Policy Partners’ Experiences of a Systems-Focused Research Collaboration in Preventive Health

“…Systems thinking focuses on casual patterns rather than independent forces, and on root causes rather than symptoms. 22,23 This, in turn, often indicates different leverage points from conventional methods, emphasises the need to respond to emergent developments rather than stick to an a priori formula, 11,12,19,25,26 and places "high value on understanding context and looking for connections between the parts, actors and processes of the system. " 11 These concepts have much in common with ecological models in community health and development that consider the social determinants of health, 26 and resonate with the realities of policy-making which are frequently complex and tend to progress in recursive cycles rather than linear rational steps.…”

Legitimising Inter-Sectoral Public Health Policies: A Challenge for Professional Identities?

“…We might also question whether the challenge of gaining legitimacy for the HiAP approach might be related to the experience of health being dominant, making representatives from the cultural sector, in particular, which is often represented by low budgets, feel devaluated. Other scholars have recognised the challenge regarding the health sector's large size and budget in collaborations, such as the HiAP initiative, and understand that this might be met with distrust [25,33]. In line with Huxham and Vangen's [18] paradox, individuals might experience HiAP as a requirement to blend into a dominant group which economically threatens their discipline, profession or department, as well as their professional identity.…”

Power, Process and Context in Theory Based Evaluation of Policy Implementation: A Response to Recent Commentaries

“…The use of multiple theories to interpret our results and test and develop program theory was seen as both a strength and limitation. Like Shankardass et al 2 we believe future work would benefit from transdisciplinary approaches and integration of theories from diverse fields including political science and economics. Indeed we have further examined theory driven analysis in an in-depth engagement between public health and social science researchers, and health bureaucrats.…”

“…The commentaries argue that this approach, with some caveats, holds promise as a foundation for future Health in All Policies (HiAP) research. Shankardass et al 2 note the potential of adapting the approach to produce bespoke frameworks for other contexts. Labonté 3 suggests that the approach provides a useful ‘roadmap’ combining methodological rigour, practice and theory.…”

“…A number of suggestions were made for strengthening or modifying our approach. Shankardass et al 2 suggest that engagement could be further developed through adoption of a Developmental Evaluation approach. 7 We agree and note that it has considerable overlap with action research.…”