Measurement for Improvement in Education

Abstract

This article focuses on “measurement for improvement,” which is the analytic work critical to making and spreading effective changes in quality improvement approaches to system transformation. Quality improvement methods aim to trigger and accelerate the learning of people within a system to make that system work better. This is accomplished through the careful attention of those who experience and enact the system at various levels and by leveraging the expertise from within and outside of the system. Measurement provides critical fuel for that learning. “Measurement for improvement” serves as an umbrella term encompassing a range of measures discussed by quality improvement scholars. These other terms include “process measures,” “practical measures,” and “pragmatic measures.” Data about system outcomes are often a key motivating factor for spurring improvement efforts, for example, low students’ graduation rates or low achievement test scores. However, outcome measures typically fall short of the data needed to inform the day-to-day work of trying changes in practice and learning from these change efforts. Instead, measures of key processes that lead to system outcomes are needed. Measures for improvement need to be closely connected to key processes, timely, and easy to collect and analyze on a regular basis by people in the system. They must also function within social processes that engender trust and transparency so that improvers can learn from failures as well as successes. The education sector has looked to quality improvement efforts in health care as a model for this work, and this article draws heavily on key texts from quality improvement in health care. However, there are some key differences in the types of data regularly available between the fields of health care and education; these differences prompt attention to certain measurement concerns, which are taken up in the references included in this article. The article begins with History and Lineage, which includes some key references that trace quality improvement ideas from industry to health care to education. The next section, Features of Measures for Improvement covers how the function of measures in a quality improvement endeavor shapes the form they take. The next section, A Set of Measures to Inform Improvement, discusses the types of measures needed in combination to inform quality improvement work in systems. The following section, Rigor of Measures to Inform Improvement, addresses what it means to have rigorous measures for improvement, in the context of the educational field where the phenomena of interest are often difficult to see or count. Another challenge of measurement for improvement is taken up in the next section, Analytic Infrastructure for Measurement for Improvement. The challenges of data collection, analysis, and consumption within the busy work lives of improvers imposes constraints and considerations to the social and technological infrastructure that enables the use of measures for improvement purposes. Finally, this article concludes with cases that illustrate measurement for improvement in educational contexts.

Leading Indicators of Academic Achievement: Investigating the Predictive Validity of an Observation Instrument in a Large District

“…However, we did not observe the same kind of regression in our analysis as we did nationally. The Rigor Appraisal process may have helped slow the decline in achievement that occurred elsewhere across the nation because of its use as part of ''measurement for improvement'' processes that are essential to system transformation (Takahashi et. al., 2020).…”

Leveraging cultural forms in human‐centred learning analytics design

“…A key activity in PMR2 was the development and visualization of practical measures (PMs) focused on students' perspectives of key aspects of mathematics learning. PMs are quick formative assessments that provide actionable feedback to inform instructional action for educators (Takahashi et al, 2022). In our case, the PMs take the form of student-facing surveys that capture students' perspectives of, for example, whole-class and small-group discussions (Jackson et al, 2016(Jackson et al, , 2022.…”

When Should I Use a Measure to Support Instructional Improvement at Scale? The Importance of Considering Both Intended and Actual Use in Validity Arguments

“…Not surprisingly, multiple measures are needed for this ambitious purpose of improving instruction at scale (see e.g., Takahashi, Norman, Jackson, Ing, & Chinen, 2020). No single measure can provide all the necessary information for what is required to increase the quality of instruction (Bennett, 2015; Bennett & Gitomer, 2009; Newton, 2010).…”