Reviewing Health Service and Program Evaluations in Indigenous Contexts: A Systematic Review

Abstract

This study systematically reviewed evidence regarding health program and service evaluations in Indigenous contexts. Following the PRISMA guidelines and combining terms for ‘Indigenous populations’ and ‘health programs and services’. Eight principles emerged: Principle 1: Adopting Indigenous led or co-led approaches is vital to balance power relationships by prioritizing self-determination, Principle 2: Evaluation team should include local Indigenous community members, Principle 3: Indigenous community knowledge and practice should be foundational, Principle 4: Evaluations must be responsive and flexible to meet the needs of the local community, Principle 5: Evaluations should respect and adhere to local Indigenous protocols, culture, wisdom and language, Principle 6: Evaluations should emphasize reciprocity, shared learnings and capacity building, Principle 7: It is important to build strong relationships and trust between and within researcher teams, evaluators and communities, and Principle 8: The evaluation team must acknowledge community capacity and resources by investing in time and relationships.

Defining collective capability in Australian evaluations that are conducted by, for and with Indigenous peoples for health programmes, policies and services: a concept analysis protocol

“…Including Indigenous methodologies and methods, better cultural understanding of the context in which services and programmes are situated and sense-making of findings of evaluations are ways to enhance the quality of evaluations. 2 4 7 8 11 Further, there is a need to create safe spaces for Aboriginal and Torres Strait Islander peoples to actively participate, include an Indigenous lens, and embed such practices as standard practice in evaluation in Indigenous contexts. 3 7 13 19 …”

“… 1–3 Evaluations of programmes, policies and services relevant to Indigenous people and communities are essential to establish an evidence base and an understanding of progress in health and social outcomes. 2 4–7 There is, however, little evidence through quality programme evaluations to show what is working and what is not working, and how Aboriginal and Torres Strait Islander peoples benefit, implying that policies and programmes are not working as well. 4 7–10 With limited and poor-quality published evaluation evidence, there is a gap in knowledge and lessons learnt.…”

“… 7 8 11 Dominant settler-colonial approaches and perspectives applied to evaluation methodologies, engagement, design and methods becomes problematic for evaluations of Indigenous programmes, policies and services within community settings as it often excludes local context, perspectives, experiences and knowledges. 2 12 13 Additionally, current practice commonly fails to address underlying power imbalances due to external evaluation teams who are often outsiders to communities. 2 7 Indigenous cultures are highly contextual; context matters.…”

“… 2 12 13 Additionally, current practice commonly fails to address underlying power imbalances due to external evaluation teams who are often outsiders to communities. 2 7 Indigenous cultures are highly contextual; context matters. 14 For Aboriginal and Torres Strait Islander people, this considers the diversity in culture, place, knowledge systems, experiences and lifeworlds.…”

“… 16–18 Internationally, indigenous evaluation has been defined and operationalised to be ‘by indigenous, for indigenous, with indigenous and as indigenous’ (Wehipeihana, p.370). 2 16 17 18 In Australia, we see some elements of Indigenist approaches integrated into evaluation practice and often manifest as Indigenous governance, 7 11 cultural protocols that inform ethical and respectful relationships with communities, 7 12 19 and processes and strategies for meaningful knowledge translation with communities. 3 4 7 20 However, these are not features of standard evaluation practice.…”