Comparing apples and oranges: Curricular intensification reforms can change the meaning of students’ grades!

Abstract

Teacher-assigned grades provide important information that is used by universities and colleges to make admission decisions and by employers to make hiring decisions. Besides grades, the results of standardized achievement tests are frequently used for student selection and allocation. However, correlations between the two achievement measures are far from perfect, and researchers have argued that this discrepancy can be at least partially attributed to norm-referenced grading, which is based on the composition of performance in a class. In this study, we investigated the results of a curricular intensification reform, which introduced changes in the distribution of student performance by making enrollment in certain courses mandatory. We investigated whether the reform resulted in changes in the relationship between standardized achievement-test results and teacher-assigned grades. We analyzed cohort control design data from two large representative samples of students from two German states (Baden-Württemberg: N = 5,574; Thuringia: N = 2,202) before and after the reform. Results indicated that students who received a certain grade before the reform (e.g., a grade of A, B, C, or D) differed in their standardized test performance from students who received the same grade after the reform. Furthermore, in math, course-level-specific reform effects on the association between grades and standardized test performance were found to vary between groups of students receiving good and poor grades. Implications for educational policy and school reforms and suggestions for grading are discussed.

When academic achievement (also) reflects personality: Using the personality-achievement saturation hypothesis (PASH) to explain differential associations between achievement measures and personality traits.

“…The degree of standardization also requires researchers to assess process markers when deriving measures. As outlined in prior research, different teachers derive grades in quite different ways (Bowers, 2009; Brookhart, 2015; Hübner et al, 2020; Koretz, 2017). Assessing cognitive ability saturation would (at least) require them to administer sufficient cognitive ability tests and estimate the associations between these general cognitive abilities and the different achievement outcomes beforehand.…”

“…WHEN ACADEMIC ACHIEVEMENT (ALSO) REFLECTS PERSONALITY 4 When Academic Achievement (Also) Reflects Personality: Using the Personality-Achievement Saturation Hypothesis (PASH) to Explain Differential Associations Between Achievement Measures and Personality Traits Both school grades and results from standardized achievement tests are key predictors of various life outcomes such as educational and socioeconomic success or educational transitions and choices (e.g., Göllner, Damian, Nagengast, Roberts, & Trautwein, 2018;Hanushek & Woessmann, 2010;Kuncel, Hezlett, & Ones, 2001Ng, Eby, Sorensen, & Feldman, 2005). Furthermore, they matter for access to colleges or universities, drop-out, and later employment (Bowers, 2011;Hübner, Wagner, Hochweber, Neumann, & Nagengast, 2020;Richardson, Abraham, & Bond, 2012;Robbins et al, 2004;Shepard et al, 2005).…”

To Those Who Have, More Will Be Given? Effects of an Instructional Time Reform on Gender Disparities in STEM Subjects, Stress, and Health

“…We did this separately for standardized test achievement, self-concept, interest, school-related stress, and health. For achievement, we used multidimensional (multiple-group) item response theory (IRT) models (see Jöreskog and Goldberger, 1975 ; Hübner et al, 2020 ). For the remaining constructs traditional structural equation models (SEMs) were applied.…”

One Program Fits All? Patterns and Outcomes of Professional Development During a Large-Scale Reform in a High-Stakes Science Curriculum

“…In order to answer our RQ 3, we more closely investigated self-reported challenges with the AP redesign and enacted classroom practices using adjusted multiple group item response theory models (e.g., Embretson & Reise, 2000; Hübner et al, 2020). These models provide a flexible approach which allows for estimation of the predicted latent standardized challenges and practices for each of the four PD groups separately, while controlling for (grand-mean-centered) teacher characteristics (see supplemental material for exemplary code for this model).…”