Real‐time evaluation in humanitarian emergencies

Brusset, Emery; Cosgrave, John; MacDonald, Wayne

doi:10.1002/ev.326

Cited by 7 publications

(5 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To do this, we use near-real-time evaluation (RTE). RTE has been used primarily in the emergency response sector ( Brusset et al, 2010 ), where waiting until the end of an event or reporting period to present recommendations can have disastrous consequences. Te advantage of making near-RTE recommendations is that the decision makers' focus is on a single recommendation when it is contextually and temporally relevant.…”

Section: Improving Eums Through Recommendation Timingmentioning

confidence: 99%

Evaluation Utility Metrics (EUMs) in Reflective Practice

Renger¹,

Renger

Eck

et al. 2022

Canadian Journal of Program Evaluation

View full text Add to dashboard Cite

The article proposes three evaluation utility metrics to assist evaluators in evaluating the quality of their evaluation. After an overview of reflective practice in evaluation, the different ways in which evaluators can hold themselves accountableare discussed. It is argued that reflective practice requires evaluators to go beyondevaluation quality (i.e., technical quality and methodological rigor) when assessingevaluation practice to include an evaluation of evaluation utility (i.e., specific actions taken in response to evaluation recommendations). Three Evaluation Utility Metrics (EUMs) are proposed to evaluate utility: whether recommendations are considered(EUMc), adopted (EUMa), and (if adopted) the level of influence of recommendations (EUMli). The authors then reflect on their experience in using the EUMs, noting the importance of managing expectations through negotiation to ensure thatEUM data are collected and the need to consider contextual nuances (e.g., adoptionand influence of recommendations are influenced by multiple factors beyond thecontrol of the evaluators). Recommendations for increasing EUM rates by payingattention to the frequency and timing of recommendations are also shared. Resultsof implementing these EUMs in a real-world evaluation provide evidence of theirpotential value: practice tips led to an EUMc of 100% and and EUMa of over 80%.Methods for considering and applying all three EUMs together to facilitate practiceimprovement are also discussed.

show abstract

Section: Improving Eums Through Recommendation Timingmentioning

confidence: 99%

Evaluation Utility Metrics (EUMs) in Reflective Practice

Renger¹,

Renger

Eck

et al. 2022

Canadian Journal of Program Evaluation

View full text Add to dashboard Cite

show abstract

“…Transient populations make large-scale surveys difficult and it is not often possible or ethical to create control groups for experimental designs. Nor does the unprecedented and fluid nature of the situation lend itself to being measured easily against benchmarks and indicators (Bornemisza et al, 2010;Brusset, Cosgrave, & MacDonald, 2010;Buttenheim, 2009;Few et al, 2014;Janis, Stiefel, & Carbullido, 2010;Puri et al, 2015;Ritchie & MacDonald, 2010b;Spence & Lachlan, 2010;Steinke-Chase & Tranzillo;Walden, 2014).…”

Section: Evaluating In Disaster Contextsmentioning

confidence: 99%

“…Ex-ante and early formative evaluations, such as vulnerability or risk assessments and disaster preparedness reviews, can provide useful information, but because they deal with the hypothetical they do not accurately forecast the complexity and unpredictability of the event when it happens (Bornemisza et al, 2010). Ongoing formative and real-time evaluations give more immediate feedback to those on the ground to inform decision making, but they too have limitations, described by Brusset et al (2010) as: staff who already have heavy workloads; little available documentation; the need for speedy data gathering; and limited time for a thorough analysis.…”

Section: Evaluating In Disaster Contextsmentioning

confidence: 99%

“…Bornemisza et al (2010) recommend these should take place within 3-6 months of an acute major crisis or after 12-24 months in an ongoing or chronic situation. Most common approaches are formal summative evaluations, sometimes called After-Action Reviews, in which those involved in the response, often with the assistance of an outside facilitator, analyse what happened, how and why, with a view to improving systems and processes (Brusset et al, 2010). Common data-gathering methods are: key informant interviews (both audio and video); document, visual, and electronic source analysis; focus groups or community discussions; mobile methods, such as transect walks or sector mapping; observations; and, where possible, surveys using questionnaires or relevant assessment tools (Brusset et al, 2010;Few et al, 2014;Janis et al, 2010;Morris, 2014;Puri et al, 2015;Rencoret et al, 2010;Spence & Lachlan, 2010).…”

Section: Evaluating In Disaster Contextsmentioning

confidence: 99%

“…Alexander (2005) notes the need for multi-organisational plans with "minimum levels of accountability, compatibility and functionality" (p.160). Other writers also argue for less fragmentation and duplication and call for increased interagency collaboration (Bornemisza et al, 2010;Brusset et al, 2010;National Institute of Mental Health, 2007;Ritchie and MacDonald, 2010a;Wulf, 2012).…”

Section: Disseminating Best Practicementioning

confidence: 99%

See 2 more Smart Citations

Evaluating in traumatic contexts: Considering the contextual, ethical, emotional, and political aspects

Mutch¹,

Weir²

2016

View full text Add to dashboard Cite

This article is drawn from the personal experience of two evaluators working in a post-disaster context. While the evaluators, also the authors of this article, were both experienced in a range of evaluation settings, they found that little in the evaluation literature prepared for them for working in such a physically exhausting and emotionally draining context. The article is set out in three parts. First, the authors synthesise the literature on evaluating in disaster and other traumatic contexts. Secondly, they outline the evaluation they undertook for The Salvation Army in the wake of the 2010/2011 Canterbury earthquakes. Finally, the authors reflect on their experiences by discussing the evaluation context, design, ethics , implementation and outcomes in order to add to our growing understanding of evaluating in traumatic contexts.

show abstract

Surge activation by the emergency department for COVID-19

Schmidt

Pham

Liu

et al. 2020

The American Journal of Emergency Medicine

View full text Add to dashboard Cite

In March 2020, the American College of Emergency Physicians (ACEP) published a national strategic plan for COVID-19, which provides general guidelines yet leaves logistical details for institutions to determine. Key capabilities from this plan provided a crucial foundation for a 16-day Emergency Department (ED) surge planning process at one pediatric institution. This paper describes critical milestones and lessons learned during this brief period, including derivation of criteria for ED surge activation, a full-scale surge drill, and the resultant ED surge protocol. The framework of real-time evaluation was used throughout the planning process and involved constant and iterative synthesis of real-time feedback from multidisciplinary stakeholders for responsive decision-making. Ultimately, the objective of this paper is to provide timely and readily actionable information to other institutions seeking guidance to apply the ACEP strategic plan for COVID-19.

show abstract

Real‐time evaluation in humanitarian emergencies

Abstract: The authors describe real-time evaluation (RTE)

Cited by 7 publications

References 1 publication

Evaluation Utility Metrics (EUMs) in Reflective Practice

Evaluation Utility Metrics (EUMs) in Reflective Practice

Evaluating in traumatic contexts: Considering the contextual, ethical, emotional, and political aspects

Surge activation by the emergency department for COVID-19

Contact Info

Product

Resources

About