You Are What You Assess: The Case for Emphasizing Chemistry on Chemistry Assessments

Abstract

What we emphasize and reward on assessments signals to students what matters to us. Accordingly, a great deal of scholarship in chemistry education has focused on defining the sorts of performances worth assessing. Here, we unpack observations we made while analyzing what “success” meant across three large-enrollment general chemistry environments. We observed that students enrolled in two of the three environments could succeed without ever connecting atomic/molecular behavior to how and why phenomena happen. These environments, we argue, were not really “chemistry classes” but rather opportunities for students to gain proficiency with a jumble of skills and factual recall. However, one of the three environments dedicated 14–57% of points on exams to items with the potential to engage students in using core ideas (e.g., energy, bonding interactions) to predict, explain, or model observable events. This course, we argue, is more aligned with the intellectual work of the chemical sciences than the other two. If our courses assess solely (or largely) decontextualized skills and factual recall we risk (1) gating access to STEM careers on the basis of facility with skills most students will never use outside the classroom and (2) never allowing students to experience the tremendous predictive and explanatory power of atomic/molecular models. We implore the community to reflect on whether “what counts” in the courses we teach aligns with the performances we actually value.

Beyond instructional practices: Characterizing learning environments that support students in explaining chemical phenomena

“…There is ample evidence that assessments tell students what sort of intellectual work is valued in a course (Biggs, 1996;Crooks, 1988;Entwistle, 1991;Momsen et al, 2013;Scouller, 1998;Scouller & Prosser, 1994;Snyder, 1973). As in prior research (Stowe et al, 2020(Stowe et al, , 2021, we continue to argue that characterizing the performances allotted points on assessments is necessary to understand "what counts" in a given learning environment. Suppose an instructor emphasizes using knowledge to predict and explain phenomena during instruction, but the recall of facts can answer all exam questions.…”

“…To support claims that particular interventions or learning environment designs improve the status quo, we argue that researchers should ensure the outcome measure(s) used in their studies engage students in performances of justifiable import. If outcome measures predominately assess recall and decontextualized math skills (as has been observed in prior works; Stowe et al, 2021), "improved outcomes" may amount to improved recall of trivia or success with "plugging and chugging." Recall and execution of rote skills are far from the most critical objectives of chemistry courses and, we argue, should not be the focus of reformed science courses.…”

“…For example, pretransformation chemistry learning environments at Michigan State University dedicated 0% of points to 3D tasks, while transformed chemistry courses dedicated approximately 35% of points to such items (Matz et al, 2018). We have previously argued that, if instructors value connecting big ideas to why phenomena happen (Stowe et al, 2020(Stowe et al, , 2021, there should be a substantial (>35%) emphasis on 3D tasks on assessments administered throughout their course.…”

“…For example, tests and quizzes written by instructors often emphasize “content mastery or the ability to solve quantitative problems” (Freeman et al, 2014; Momsen et al, 2010). Indeed, we observed commonly enacted general chemistry learning environments to focus assessments on mathematical calculations rather than atomic/molecular explanations for phenomena (Stowe et al, 2021). Students are fully capable of recalling facts without understanding the conceptual basis underlying those facts or using these concepts in a slightly different context (Cooper et al, 2013, 2015; Williams et al, 2015).…”

“…However, existing literature provides little insight into whether passing a given science course relates to student engagement in intellectual work authentic to the practice of science. For example, students may succeed in a chemistry course that solely (or primarily) assesses calculations without ever meaningfully relating molecular behavior to phenomena (Matz et al, 2018; Stowe et al, 2021). Furthermore, virtually all “first‐generation” studies of “active learning” compare outcomes from “active” classes to outcomes from lecture‐based classes.…”