Bridging the Gap Both-Ways: Enhancing Evaluation Quality and Utilisation in a Study of Remote Community Safety and Wellbeing with Indigenous Australians

Abstract

This paper demonstrates how the voices of people affected by a policy can contribute to evaluation quality and utilisation. In this participatory mixed-methods approach, Indigenous Australians in remote parts of the Northern Territory of Australia were involved in evaluation of an intervention that significantly affected their daily lives. The evaluation was inspired by the ideas and values demonstrated in a both-ways learning model. This mixed-methods approach combines a people-centered participatory methodology with a more standardised policy-focused application of survey methods to contribute to policy development and local planning. The aim of the study was to hear from local people in communities and encourage them to share their views about changes in community safety. It provided a voice for Indigenous Australians affected by the Northern Territory Emergency Response (NTER), which sought to protect children in 73 Indigenous communities across the Northern Territory. The aim of the NTER was to create safer communities and to improve health, nutrition, educational outcomes, service delivery and engagement with Indigenous people. Regardless of views about the NTER, a clear gap in the evidence base for evaluation was to understand if people felt safer after the intervention. Increased police, safe houses, night patrols, health services, family support services and management of income support payments were introduced, but what did Indigenous people living in remote communities think? What was their experience and what will happen to the information they may choose to share? Adopting a mixed-methods approach that included a large-scale survey and qualitative research conducted with local Indigenous residents of remote communities strengthened evaluative findings, particularly for policy makers and community residents. A participatory research cycle was used to engage local people and return results in an ethical feedback process that aimed to build capacity on the ground and link into future planning. We sought to promote the role of evaluation in remote Northern Territory communities as a worthwhile way for local people to share their voice, stories and experiences with the policy makers whose decisions can so significantly affect their lives.

Evaluation in the field of early childhood development: A scoping review

“…In other words, these groups were used to leverage community experience and knowledge to contribute to decision-making. There were a few studies ( n = 7) that did not include stakeholder committees but were characterised by collaboration between stakeholders and research staff to design an evaluation that was applicable and relevant to their specific context (Andrews et al, 2019; Cook, 2006; Flaspohler et al, 2003; Grey et al, 2016; Kearney et al, 2014; Newell & Graham, 2012; Scougall, 2006). These studies tried to include stakeholder insights by way of consultation and collaboration in the development of the evaluation.…”

“…Nine of the included studies had a cultural component; of these, six studies used a community-driven approach to enhance the cultural relevance and appropriateness of the evaluation (Aronson et al, 2007a, 2007b; Grey et al, 2016; Haviland & Pillsbury, 2012; Peter, 2003; Scougall, 2006; Stack et al, 2018). These studies engaged members of the target community during the early stages of the evaluation to design an evaluation plan that reflects community values and considers the culture of the community involved.…”

“…Five of these studies were situated within Indigenous communities (Grey et al, 2016; Haviland & Pillsbury, 2012; Kearney et al, 2014; Peter, 2003; Scougall, 2006). These studies made genuine efforts to include the community at nearly every stage of the evaluation process and were characterised by attempts to gain community consent, provide training to local researchers, and allow stakeholders the final word on all evaluation decisions (Grey et al, 2016; Scougall, 2006).…”

Indigenous Perspectives in Program Evaluation: A scoping literature review exploring wise practices for program evaluation with Indigenous communities in northern Manitoba

“…Bridging these relationships created the opportunity for learning and understanding, thus moving away from one-off judgments. Scholars such as Grey et al (2016), ), McKinley (2020, and Brown and Di Lallo (2020) all utilized the method of employing local community members to conduct components of the evaluation process-creating the questions for the surveys and questionnaires, collecting the data, and doing analysis or disseminating the findings. There are several advantages to hiring community members to help with the evaluation process.…”

“…Furthermore, they may have other relationships that prove to benefit the evaluation that an external evaluation team may not have. Finally, having local community members participating in the evaluation process from beginning to end builds capacity, harnesses a level of confidentiality in hopes that the information collected will be shared back with the community at large and kept within the community (Grey et al (2016), Grover (2010) and Brown and Di Lallo (2020)).…”