Critical factors that affect the functioning of a research and evaluation capacity building partnership: A causal loop diagram

Abstract

Introduction Public health policy and practice is strengthened by the application of quality evidence to decision making. However, there is limited understanding of how initiatives that support the generation and use of evidence in public health are operationalised. This study examines factors that support the internal functioning of a partnership, the Western Australian Sexual Health and Blood-borne Virus Applied Research and Evaluation Network (SiREN). SiREN aims to build research and evaluation capacity and increase evidence-informed decision making in a public health context. Methods This study was informed by systems concepts. It developed a causal loop diagram, a type of qualitative system model that illustrated the factors that influence the internal operation of SiREN. The causal loop diagram was developed through an iterative and participatory process with SiREN staff and management (n = 9) via in-depth semi-structured interviews (n = 4), workshops (n = 2), and meetings (n = 6). Results Findings identified critical factors that affected the functioning of SiREN. Central to SiREN’s ability to meet its aims was its capacity to adapt within a dynamic system. Adaptation was facilitated by the flow of knowledge between SiREN and system stakeholders and the expertise of the team. SiREN demonstrated credibility and capability, supporting development of new, and strengthening existing, partnerships. This improved SiREN’s ability to be awarded new funding and enhanced its sustainability and growth. SiREN actively balanced divergent stakeholder interests to increase sustainability. Conclusion The collaborative development of the diagram facilitated a shared understanding of SiREN. Adaptability was central to SiREN achieving its aims. Monitoring the ability of public health programs to adapt to the needs of the systems in which they work is important to evaluate effectiveness. The detailed analysis of the structure of SiREN and how this affects its operation provide practical insights for those interested in establishing a similar project.

Applications of Participatory System Dynamics Methods to Public Health

“…The "multiple and overlapping pathways identified through [PSD] process [yield] consideration of a wider array of intervention approaches" 32 can help develop interventions that similarly acknowledge and account for complexity. 27,29,32,[52][53][54][55][56][57] Rather than continue to pursue single objective interventions or develop policy in silos, PSD instead facilitates decision making that considers shared objectives, as well as policy levers that optimize the shared objectives of diverse stakeholders. 54 Conceptualizing and visualizing public health issues as complex and adaptive systems help stakeholders identify existing gaps in evidence, knowledge, and practice more readily, heightening capacity for intervention efficacy.…”

“…Stakeholder biases are purposively present, which limits generalizability. 30,34,36,37,39,52,[54][55][56][59][60][61]64,66,72,76 CLDs may also have limited generalizability due to purposive or snowball sampling often used to attract PSD participants 25 or power dynamics in the workshop leading to social desirability biases. 30 Qualitative data generally are additionally susceptible to participant recall bias, further limiting generalizability of qualitative PSD methods.…”

“…34,50,59,62,64 In addition, several studies noted that despite the intentional focus on capturing perspectives of stakeholders, group social dynamics within the participant group sometimes created difficulty in ensuring that all stakeholder perspectives were represented in the model-building process. 34,36,50,55,56,60,65 As the process relies heavily on consensus building, capturing dissenting or confounding perspectives was described as difficult. 36 In addition, a few studies noted that having to use a small sample of stakeholders sometimes meant that insights from various professional or community perspectives would not be present, threatening the validity of the model.…”

“…An additional consideration reported about the use of PSD methods was the significant preparation and time commitment required to engage in the methods successfully. 25,35,36,47,51,52,56,68,69,71,76 The use of PSD methods was described in several articles as requiring substantial time and resources. 36,47,52,56,68,69,71,76 Successful partnership was discussed as requiring "considerable effort, collaboration, preparation work, staff power, and time."…”

Utilizing Causal Loop Diagramming to Explore a Research and Evaluation Capacity Building Partnership

“…The size of SiREN limits the scale of change; currently, it employs 1.4 full-time equivalent (FTE) staff as part of core funding and a further 4.0 FTE staff through additional grants. Additional descriptions of SiREN are available in previous publications ( 25 – 27 ).…”