Evaluating as an Outsider or an Insider: A Two-Way Approach Guided by the Knowers of Culture

Abstract

Evaluating programs in remote Indigenous communities requires contextual consideration and a degree of connection that goes beyond the usual focus on output measurement and system monitoring. Evaluators who are experienced with working in remote communities become more and more cognisant of the issues and complexities that impact the quality of evaluations. As this reality presents itself, evaluators rely more and more on the help of the local community to explain and reveal such realities. A learning journey takes place that allows ‘outsiders’ to be accepted into the community and be part of a productive matrix of support and ‘insiders’ begin to assume their legitimate rights to participate, advise, educate and guide evaluators. Without a participative and collaborative styled process, evaluation risks being non-reality based without a guarantee of quality and has the potential to impact the community in a negative way. This paper presents the views and experiences of ‘insiders’ and ‘outsiders’ who have been involved in evaluations in remote communities. Their collective views demonstrate the need for a two-way approach that establishes middle ground for a respectful and quality evaluation process. This approach ensures evaluation remains relevant and purposeful for those who are impacted by the program outcomes, providing a focus on the everyday reality that depends on cultural responsiveness and ensures legitimacy for Indigenous people.

Indigenous health program evaluation design and methods in Australia: a systematic review of the evidence

“…Quantitative data provide a measure of the impact of health programs; qualitative data enable participative and collaborative evaluation and are valuable to contextualise and provide culturally relevant inferences about a program's utility and impact. Therefore, mixed methods approaches, utilising both quantitative and qualitative data, are likely to be the best approach to conducting complex program evaluations 13,14 . However, only a third of identified studies utilised a mixed qualitative/quantitative design.…”

“…Therefore, mixed methods approaches, utilising both quantitative and qualitative data, are likely to be the best approach to conducting complex program evaluations. 13,14 However, only a third of identified studies utilised a mixed qualitative/quantitative design. Stronger collaborations between qualitative and quantitative evaluators are needed, particularly by researchers using experimental evaluation designs.…”

Aboriginal Family Planning Circle evaluation: Empowering Aboriginal communities in evaluating and future-proofing Aboriginal-led community programmes

“…The partnership between non-Aboriginal WESTIR researchers and Aboriginal-identifying AFPC staff in co-designing and implementing the evaluation frameworks was a practical way in being culturally responsive. The adoption of this 'two-way approach', where Aboriginal practices and knowledge was combined with non-Aboriginal 'ways of doing', has been known to balance decision-maker needs with people's voice, lessen bias and misinterpretation and make research safer for Aboriginal participants (Rossingh & Yunupingu, 2016). Due to the pro bono nature of the AFPC evaluations, there were not enough resources to ensure the evaluations were wholly administered by the local Aboriginal community.…”

Strengths-based approaches for quantitative data analysis: A case study using the australian Longitudinal Study of Indigenous Children

“…While the literature documents strengths-based approaches for various stages of the research cycle ( Bond, 2009 ; Foley & Schubert, 2013 ; Haswell-Elkins et al, 2007 ; Holmes, Stewart, Garrow, Anderson, & Thorpe, 2002 ; Pyett et al, 2008 ; Rossingh & Yunupingu, 2016 ; Wand & Eades, 2008 ), there is an absence of literature on the practical application of strengths-based approaches to quantitative research. Given increasing demands for strengths-based approaches, there is a need for practical guidance on the application of these methods.…”