What Is the Potential for Applying Cost-Utility Analysis to Facilitate Evidence-Based Decision Making in Schools?

Hollands, Fiona M.; Pan, Yilin; Escueta, Maya

doi:10.3102/0013189x19852101

Cited by 14 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there is increasing policy pressure to improve student outcomes at school by basing decisions about strategies and approaches on robust evidence generated by research (Hollands et al , 2019). Evidence derived from research is considered important in contributing to improvements as agreed by most professionals working in public policy and practice (Davies, 1999).…”

Section: Introductionmentioning

confidence: 99%

What is the evidence on the best way to get evidence into use in education?

Gorard

See

Siddiqui

2020

Review of Education

View full text Add to dashboard Cite

For decades there have been calls by concerned stakeholders to improve the quality of education research, and some progress has been made towards creating a more secure evidence base in some areas. More programmes and approaches that have a reasonable evidence base are now also being used in schools (but not in policy, and not necessarily because they have a reasonable evidence base). However, there has been no equivalent improvement in secure knowledge about how best to get that evidence into use, or even what difference it makes when such evidence is used. This paper looks at what little is already known about the different ways to get research evidence into use in education by summarising the results of a large-scale review of the literature. A total of 323 of the most relevant studies were looked at across all areas of public policy, and judged for quality and contribution. Very few (33) were of the appropriate design and quality needed to make robust causal claims about evidence-intouse, and even fewer of these concerned education. This means that despite over 20 years of modest improvement in research on what works in education policy and practice, the evidence on how best to deploy these findings is still very weak. We consider studies in terms of several issues, including whether they look at changes in user knowledge and behaviour, or student outcomes, and how evidence is best modified before use. Providing access to raw research evidence or even slightly simplified evidence is not generally an effective way of getting it used, even if that evidence is presented to users by knowledge-brokers, in short courses or similar. What is more likely to work for both policy and practice is the engineering of high quality evidence into a more usable format and presenting it actively or iteratively via a respected and trusted conduit, or through population measures such as legislation. Having the users actually do the research is another promising approach. Expecting each individual study they fund to have an impact is not the way forward, as this may encourage widespread use of ineffective or even harmful interventions. Publicly-funded users, including policy-makers, should be required to use evidence-led programmes from those libraries providing them and which are appropriate and relevant to their aims. Research funders should support these approaches, and help to build up libraries of successfully tested programmes. Researchers need to be scrupulous, looking at their new evidence in the context of what is already known and not looking to obtain 'impact' from single studies. More and better research is needed on the best routes for evidence-into-use. However, the improvements required of all parties are as much ethical in nature as they are technical or scientific.Keywords use of evidence, engineering of evidence, translation of evidence, research impact, knowledge transfer, robust evidence.

show abstract

Section: Introductionmentioning

confidence: 99%

What is the evidence on the best way to get evidence into use in education?

Gorard

See

Siddiqui

2020

Review of Education

View full text Add to dashboard Cite

show abstract

“…M. Levin, 1995, 2001; H. M. Levin & Belfield, 2015), or utility, as perceived by various decision makers and stakeholders (Hollands et al, 2019). Impact may alternately be measured with respect to a needs assessment (Watkins et al, 2012), with improvements addressing more urgent needs receiving higher utility values (Ross, 2008).…”

Section: Framework For Research Worth Usingmentioning

confidence: 99%

Research Worth Using: (Re)Framing Research Evidence Quality for Educational Policymaking and Practice

Ming

Goldenberg

2021

Review of Research in Education

View full text Add to dashboard Cite

This chapter calls for researchers to reconceptualize research quality from the perspective of its expected use, attending to power dynamics that influence how knowledge is defined, constructed, and validated through the research enterprise. Addressing these concerns when designing and conducting education research can yield more useful research evidence for building more equitable education systems. Anchored in scholarship on research utilization and methodological critiques, the chapter introduces a research quality framework that integrates relevance and rigor through five key dimensions of Research Worth Using: (1) relevance of question: alignment of research topics to practical priorities; (2) theoretical credibility: explanatory strength and coherence of principles investigated; (3) methodological credibility: internal and external credibility of study design and execution; (4) evidentiary credibility: robustness and consistency of cumulative evidence; and (5) relevance of answers: justification for practical application. This framework simultaneously uplifts the voices and needs of policymakers, practitioners, and community members, while elevating standards for excellence in education research. We call attention to the myriad ways in which the quality of evidence generated can be strengthened, before describing implications for curating and using research. We conclude by offering suggestions for applying and further developing the framework.

show abstract

“…Working with schools, districts, and current and aspiring school leaders, we developed a 10-step decision-making framework to tackle these kinds of complex decisions (Hollands, Pan, & Escueta, 2019). It relies on cost-utility analysis, which is widely used to inform decision making in other fields, especially health care (Robinson, 1993), to determine what programs will have the highest return on investment (ROI) — in other words, what programs will be most worth the cost.…”

Section: A Decision-making Frameworkmentioning

confidence: 99%

How to choose tools and programs that are worth the investment

Hollands¹,

Kushner²

2021

Phi Delta Kappan

Self Cite

View full text Add to dashboard Cite

Making evidence-based decisions in schools seems like an impossible task when hundreds of activities are available, the evidence base fails to address most of them, and budget deadlines are immovable. In addition, different stakeholders hold different views on what it means for a program, strategy, or intervention to “work.” Fiona Hollands and Anna Kushner describe a 10-step decision-making framework developed in collaboration with practicing education decision makers to tackle these kinds of complex decisions. It is based on cost-utility analysis, which addresses two main questions: What resources are required to implement a program, and how useful or satisfying is each program to stakeholders?

show abstract

What Is the Potential for Applying Cost-Utility Analysis to Facilitate Evidence-Based Decision Making in Schools?

Cited by 14 publications

References 33 publications

What is the evidence on the best way to get evidence into use in education?

What is the evidence on the best way to get evidence into use in education?

Research Worth Using: (Re)Framing Research Evidence Quality for Educational Policymaking and Practice

How to choose tools and programs that are worth the investment

Contact Info

Product

Resources

About