Appropriate Standards of Evidence for Education Policy Decision Making

Conaway, Carrie; Goldhaber, Dan

doi:10.1162/edfp_a_00301

Cited by 10 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This leads to a second, and related, defense of using an assessment like edTPA. Conaway and Goldhaber (2018) argue that, in developing accountability systems and evaluating measurement errors, it is important to consider the counterfactual , the fact that alternative and status quo methods can be even more problematic from a measurement perspective. We reject this argument for two reasons.…”

Section: Discussionmentioning

confidence: 99%

Assessing the Assessment: Evidence of Reliability and Validity in the edTPA

Gitomer

Martinez

Battey³

et al. 2019

American Educational Research Journal

View full text Add to dashboard Cite

The Educative Teacher Performance Assessment (edTPA) is a system of standardized portfolio assessments of teaching performance mandated for use by educator preparation programs in 18 states, and approved in 21 others, as part of initial certification for preservice teachers. Because of the high stakes involved for examinees, it is critical that the scores produced and resulting decisions are meaningful and meet robust standards of validity and technical quality for educational measurements. We examined the technical documentation of edTPA and raise serious concerns about scoring design, the reliability of the assessments, and the consequential impact on decisions about edTPA candidates. In light of these findings, we argue that the proposed and actual uses of the edTPA are currently unwarranted on technical grounds.

show abstract

Section: Discussionmentioning

confidence: 99%

Assessing the Assessment: Evidence of Reliability and Validity in the edTPA

Gitomer

Martinez

Battey³

et al. 2019

American Educational Research Journal

View full text Add to dashboard Cite

show abstract

“…We should inform policy makers of the conditionality of our findings and of the uncertainty that is a fundamental property of scientific knowledge. Such uncertainties could include, for instance, problems of scaling up effective small-scale interventions (Kizilcec et al 2020) and the uncertainty of statistical effects (Conaway and Goldhaber 2020). In a way, social scientists exchanging knowledge with policy makers are pulled between the certainty requested by policy makers and the fundamental uncertainty of scientific knowledge.…”

Section: A Split Between the Certainty Desired By Policy Makers And T...mentioning

confidence: 99%

Bringing in Theory Building and Social-Political Context to Understand and Reduce Inequality: A Commentary

Werfhorst

2022

Socius: Sociological Research for a Dynamic World

View full text Add to dashboard Cite

The ambition to move from understanding inequality to (also) reducing inequality is laudable and important for the field of sociology. Discussing the contributions to this special collection, the author encourages current developments in our field to identify causal effects of interventions. However, the author has two concerns with this development, partly shared with the contributions to this collection. First, to bridge the split between the certainty a policy maker wants and the uncertainty that is fundamental to scientific knowledge, we should be focused more on theory building and hypotheses testing, using various methods to study similar research problems, collaborating with organizations in the fields that we study. Second, we could be more concerned with the social and political context within which reforms may or may not be implemented. Policy makers and researchers can be more specific about the values underlying proposed interventions or in society at large.

show abstract

“…G. Chambers, 1999). Just as impacts must be measured relative to alternatives, costs must also be measured relative to the relevant policy alternatives (Conaway & Goldhaber, 2018; Rice, 1997). Equity concerns again demand considering the distribution of costs borne by recipients and intervenors.…”

Section: Framework For Research Worth Usingmentioning

confidence: 99%

Research Worth Using: (Re)Framing Research Evidence Quality for Educational Policymaking and Practice

Ming

Goldenberg

2021

Review of Research in Education

View full text Add to dashboard Cite

This chapter calls for researchers to reconceptualize research quality from the perspective of its expected use, attending to power dynamics that influence how knowledge is defined, constructed, and validated through the research enterprise. Addressing these concerns when designing and conducting education research can yield more useful research evidence for building more equitable education systems. Anchored in scholarship on research utilization and methodological critiques, the chapter introduces a research quality framework that integrates relevance and rigor through five key dimensions of Research Worth Using: (1) relevance of question: alignment of research topics to practical priorities; (2) theoretical credibility: explanatory strength and coherence of principles investigated; (3) methodological credibility: internal and external credibility of study design and execution; (4) evidentiary credibility: robustness and consistency of cumulative evidence; and (5) relevance of answers: justification for practical application. This framework simultaneously uplifts the voices and needs of policymakers, practitioners, and community members, while elevating standards for excellence in education research. We call attention to the myriad ways in which the quality of evidence generated can be strengthened, before describing implications for curating and using research. We conclude by offering suggestions for applying and further developing the framework.

show abstract

Appropriate Standards of Evidence for Education Policy Decision Making

Cited by 10 publications

References 32 publications

Assessing the Assessment: Evidence of Reliability and Validity in the edTPA

Assessing the Assessment: Evidence of Reliability and Validity in the edTPA

Bringing in Theory Building and Social-Political Context to Understand and Reduce Inequality: A Commentary

Research Worth Using: (Re)Framing Research Evidence Quality for Educational Policymaking and Practice

Contact Info

Product

Resources

About