This chapter calls for researchers to reconceptualize research quality from the perspective of its expected use, attending to power dynamics that influence how knowledge is defined, constructed, and validated through the research enterprise. Addressing these concerns when designing and conducting education research can yield more useful research evidence for building more equitable education systems. Anchored in scholarship on research utilization and methodological critiques, the chapter introduces a research quality framework that integrates relevance and rigor through five key dimensions of Research Worth Using: (1) relevance of question: alignment of research topics to practical priorities; (2) theoretical credibility: explanatory strength and coherence of principles investigated; (3) methodological credibility: internal and external credibility of study design and execution; (4) evidentiary credibility: robustness and consistency of cumulative evidence; and (5) relevance of answers: justification for practical application. This framework simultaneously uplifts the voices and needs of policymakers, practitioners, and community members, while elevating standards for excellence in education research. We call attention to the myriad ways in which the quality of evidence generated can be strengthened, before describing implications for curating and using research. We conclude by offering suggestions for applying and further developing the framework.
Background/Context Amidst the complex and fast-paced demands in schools and classrooms, identifying what most deserves educators’ attention can pose a challenge. Indicators help focus attention by highlighting key features that signal important outcomes or opportunities to take action or learn. Purpose Informed by multiple literatures, we offer a conceptual framework that elaborates on the connection between an indicator's development and use. Specifically, the framework proposes that how to develop indicators depends on the answers to several questions about how they will be used: the intended use (why), timing (when), content (on what), and threshold (how much). Research Design To explore this conceptual framework, we present two cases on the construction and use of indicators for continuous improvement in the San Francisco Unified School District. In the first case, we applied statistical process control (SPC) theory to create control charts that visualize district-level outliers on student achievement outcomes to examine potential causes of variation among schools. In the second case, we combined SPC with more basic statistical methods to develop indicators highlighting areas of potential promise across multiple measures and student subgroups within a school. In our exploration of the two case studies, we present the statistical methods applied to construct the indicator, an example of the visualizations created, and descriptions of how central office leaders interpreted and used the results. We document how the intended use informed the indicators’ design, and the ramifications of those decisions for educators’ subsequent use of the indicators for improvement. Conclusions and Recommendations In learning from these cases, we highlight the importance of constructing indicators from the perspective of their intended use and linked with specific conditions and potential actions. This requires including more useful process data in the development of those indicators, as well as more follow-up analysis of the effectiveness of the actions taken based on those indicators, contrasted against alternative actions. We also underscore the importance of developing indicators that distinguish between the need for systemic versus targeted improvement efforts, based on SPC. We believe that combining approaches across disciplines and methodologies holds great promise for the development and use of indicators for continuous improvement in education.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.