Measurement Matters

Abstract

There has been perennial interest in personal qualities other than cognitive ability that determine success, including self-control, grit, growth mindset, and many others. Attempts to measure such qualities for the purposes of educational policy and practice, however, are more recent. In this article, we identify serious challenges to doing so. We first address confusion over terminology, including the descriptor “non-cognitive.” We conclude that debate over the optimal name for this broad category of personal qualities obscures substantial agreement about the specific attributes worth measuring. Next, we discuss advantages and limitations of different measures. In particular, we compare self-report questionnaires, teacher-report questionnaires, and performance tasks, using self-control as an illustrative case study to make the general point that each approach is imperfect in its own way. Finally, we discuss how each measure’s imperfections can affect its suitability for program evaluation, accountability, individual diagnosis, and practice improvement. For example, we do not believe any available measure is suitable for between-school accountability judgments. In addition to urging caution among policymakers and practitioners, we highlight medium-term innovations that may make measures of these personal qualities more suitable for educational purposes.

A Novel Low-Cost Indicator of Student Perseverance and Its Association with College Student Academic Performance

“…Particularly, obtaining survey-based measures is an unwieldy process that remain vulnerable to socially desirable responding and recall bias (Duckworth et al, 2007). Surveys of self-reporting students, or of teachers about their students, can suffer from reference bias while teacher surveys also may reflect a teacher's misinterpretation of student behavior (Duckworth & Yeager, 2015). Performance task testing is an alternative to surveys, but in addition to the costs of conducting well designed performance task tests, other shortcomings may exist as well.…”

“…One set of variables commonly found to be significant are various measures of cognitive ability, such as the Scholastic Aptitude Test (SAT) and American College Testing (ACT) tests often required for admission to U.S. colleges and universities, while another frequently significant variable is a measure of pre-college academic performance such as a high school class rank (Noble & Sawyer, 2004;Sawyer, 2013). In the past decade, however, the scholarly community has shown mounting interest in identifying student qualities beyond cognitive ability that impact performance outcomes (Duckworth & Yeager, 2015). In particular, the concept of grit, or "perseverance and passion for long-term goals" (Duckworth, Peterson, Matthews, & Kelly, 2007, p. 1087, has gained attention as a predictor of academic achievement and success.…”

“…Performance task testing is an alternative to surveys, but in addition to the costs of conducting well designed performance task tests, other shortcomings may exist as well. A one-time task test can assess a student's behavior in that test environment, but it may not be representative of the student's behavioral choices in regular daily life (Duckworth & Yeager, 2015). As such, Duckworth and Yeager (2015) recommend developing new, creative measurements and advocate for a "plurality of measurement approaches" (p. 245) that includes surveys, performance task testing, and other measures to be developed.…”

“…A one-time task test can assess a student's behavior in that test environment, but it may not be representative of the student's behavioral choices in regular daily life (Duckworth & Yeager, 2015). As such, Duckworth and Yeager (2015) recommend developing new, creative measurements and advocate for a "plurality of measurement approaches" (p. 245) that includes surveys, performance task testing, and other measures to be developed. In response to their call to action, this article introduces an innovative non-survey-based measurement, derived from common higher education variables, that reflects the perseverance construct.…”

The Technologisation of Education and the Pathway to Depersonalisation and Dehumanisation

“…(Laura & Chapman, 2009) It is not surprising that many of the 'value-based' qualities such as resilience, perseverance, consistency, confidence, relational intelligence, leadership skills, and service to the community are the very virtues that experienced and proficient teachers seek to inculcate in their students (Becker and Luther 2002). It has also been established that endeavouring to teach such values with electronically based devices serves to compromise the pedagogic efficacy of student capacity for value appropriation (Duckworth and Yeager 2015;Seider 2012), It has also been argued that computechnologically structured teaching is not as conducive to developing social and moral sensibilities and may even prove to impede the holistic progress of the child, particularly in the areas of creativity and socio-emotional health. For example, Gardner and Davis (2013); Turkle (2011); Uhls et al (2014) found that preteens who refrained from using their devices for five days (in favor of face-to-face interactions) were more adept at reading and interpreting human emotions.…”