The evaluator's role in valuing: Who and with whom

Alkin, Marvin C.; Vo, Anne T.; Christie, Christina A.

doi:10.1002/ev.20004

Cited by 24 publications

(26 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tend to know—through reflective narratives and interviews, for instance—about the organizations or programs (the where ) in which these stakeholders’ work is situated, their activities (the what ), and the motivations and underlying logic for such activities (the why ) (Fitzpatrick, Christie, & Mark, ; Wharton & Alexander, ). Additionally, as members of a growing field, evaluators are beginning to address more explicitly and on a broader level issues about valuing; that is, the manner in which value, merit, worth, and significance are systematically attributed to a unit of interest in the real world (Alkin et al., ; Julnes, ; Scriven, ). However, there is a paucity of studies that offer empirically derived explanations for the observations already captured in the literature (the why ).…”

Section: Research On Context In Evaluation: a Way Forwardmentioning

confidence: 99%

Advancing Research on Evaluation Through the Study of Context

Christie

2015

New Drctns Evaluation

Self Cite

View full text Add to dashboard Cite

The importance of context in shaping evaluation practice is broadly accepted. Still, there is much to be learned about context, and we remain at the early stages of this process. In this chapter, we describe a framework for organizing and focusing ideas for conducting research on evaluation context. We hope that this tool will be a source of inspiration for future studies of context, thus contributing to a deeper and higher quality evaluation knowledge base.

show abstract

Section: Research On Context In Evaluation: a Way Forwardmentioning

confidence: 99%

Advancing Research on Evaluation Through the Study of Context

Christie

2015

New Drctns Evaluation

Self Cite

View full text Add to dashboard Cite

show abstract

“…(Note that requirements such as “large effect” or “adequate sample size,” which sometimes appear, are not qualitative criteria but rather imprecise quantitative statements.) The lack of available protocols for assessing qualitative evidence in a systematic way may well be a deterrent to its utilization, which is unfortunate because many practical evaluations today include a qualitative component (6 & Bellamy, 2012 2 ; Alkin, Vo, & Christie, 2012; Chelimsky, 2012; Shadish, Cook, & Campbell, 2002; Shipman, 2012). One such protocol to assess qualitative evidence has recently been proposed by Claes (this issue).…”

Section: Discussionmentioning

confidence: 99%

Comparing rating paradigms for evidence-based program registers in behavioral health: Evidentiary criteria and implications for assessing programs

Means

Magura

Burkhardt

et al. 2015

Evaluation and Program Planning

View full text Add to dashboard Cite

Decision makers need timely and credible information about the effectiveness of behavioral health interventions. Online evidence-based program registers (EBPRs) have been developed to address this need. However, the methods by which these registers determine programs and practices as being “evidence-based” has not been investigated in detail. This paper examines the evidentiary criteria EBPRs use to rate programs and the implications for how different registers rate the same programs. Although the registers tend to employ a standard Campbellian hierarchy of evidence to assess evaluation results, there is also considerable disagreement among the registers about what constitutes an adequate research design and sufficient data for designating a program as evidence-based. Additionally, differences exist in how registers report findings of “no effect,” which may deprive users of important information. Of all programs on the 15 registers that rate individual programs, 79% appear on only one register. Among a random sample of 100 programs rated by more than one register, 42% were inconsistently rated by the multiple registers to some degree.

show abstract

“…This continuum is about valuing more broadly, and not just about making an overall evaluative conclusion, but I use it here to frame the discussion about the role of stakeholders in this particular aspect of valuing (see Figure 1). Starting at one end of the continuum, some evaluators view the task of judging merit or worth as the evaluator's primary and sole responsibility and consider that this should not be passed to stakeholders (Alkin et al, 2012;Christie & Alkin, 2013). Scriven is one such evaluator.…”

Section: If An Overall Conclusion Is Made Who Should Make It?mentioning

confidence: 99%

“…Scriven is one such evaluator. In his view, this ensures judgements are unbiased because stakeholders are not involved and can not influence the evaluator's decision (Alkin et al, 2012). Another perspective on the overall conclusion being the sole domain of the evaluator comes from methods-based evaluators, where value judgements come directly from the scientific process.…”

Section: If An Overall Conclusion Is Made Who Should Make It?mentioning

confidence: 99%

“…The questions of who values, how, and under what conditions are more explicitly considered by some theorists than others, and are contentious aspects of evaluation practice (Alkin, Vo, & Christie, 2012;Gates, 2017). This literature review explores one aspect of valuing-the valuing involved in reaching a warranted conclusion about the overall merit, worth, or significance of an evaluand.…”

mentioning

confidence: 99%

See 1 more Smart Citation

The overall evaluative conclusion as an element of evaluation practice

MacDonald¹

2019

View full text Add to dashboard Cite

The questions of who values, with whom, in what ways, and under what conditions concern all evaluators but are explicitly considered by some theorists more than others. Theorists placed on the valuing branch of Christie and Alkin’s (2013) evaluation theory tree emphasise valuing in their conceptualisation of evaluation, but even among these theorists there is diversity in the ways in which valuing is considered and realised in evaluation practice. This article explores this diversity within one aspect of valuing—the valuing involved in reaching a warranted conclusion about the overall merit, worth, or significance of an evaluand. It considers the extent to which the literature discusses overall evaluative conclusions as an element of evaluation practice; the extent to which drawing such conclusions is seen as the responsibility of the evaluator or stakeholders; and the methods that may be used to reach a warranted evaluative conclusion. The author concludes that there has been little empirical research undertaken on the valuing involved in reaching a warranted conclusion about the overall merit, worth, or significance of an evaluand. Much of the literature is evaluators theorising from different epistemological positions. Thus, while the literature does not definitively inform evaluators of whether they should always reach an overall evaluative conclusion, who they should involve, and what methods they should use, this review does support evaluators to reflect on these issues in their practice, and to make deliberate, informed decisions about the making—or not—of overall conclusions or judgements in future evaluations.

show abstract

The evaluator's role in valuing: Who and with whom

Abstract: Abstract

Cited by 24 publications

References 16 publications

Advancing Research on Evaluation Through the Study of Context

Advancing Research on Evaluation Through the Study of Context

Comparing rating paradigms for evidence-based program registers in behavioral health: Evidentiary criteria and implications for assessing programs

The overall evaluative conclusion as an element of evaluation practice

Contact Info

Product

Resources

About