Rubric Design For Separating The Roles Of Open-Ended Assessments

Phys. Rev. Phys. Educ. Res.

Doughty

Turnbull

et al. 2017

Self Cite

Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level Classical Mechanics and Math Methods course (CM 1) at CU Boulder, we have developed a tool to assess student learning of CM 1 concepts in the upper-division. The Colorado Classical Mechanics/Math Methods Instrument (CCMI) builds on faculty consensus learning goals and systematic observations of student difficulties. The result is a 9-question open-ended post-test that probes student learning in the first half of a two-semester classical mechanics / math methods sequence. In this paper, we describe the design and development of this instrument, its validation, and measurements made in classes at CU Boulder and elsewhere.

Section: A Rubric Rationalementioning

confidence: 99%

Assessing learning outcomes in middle-division classical mechanics: The Colorado Classical Mechanics and Math Methods Instrument

Phys. Rev. Phys. Educ. Res.

Doughty

Turnbull

et al. 2017

Self Cite

“…[56]. Assessments were developed for Electricity and Magnetism I (CUE [50]), Quantum I (QMAT [53]), Math Methods and Classical Mechanics assessment (CCMI [55,57]) and Electricity and Magnetism II (CURrENT [54]), with multiple-choice formats being developed for the CUE [51] and QMAT [52].…”

Section: E Assessment: What Are Students Learning?mentioning

confidence: 99%

Educational transformation in upper-division physics: The Science Education Initiative model, outcomes, and lessons learned

Chasteen

Wilcox

Phys. Rev. ST Phys. Educ. Res.

et al. 2015

Self Cite

[This paper is part of the Focused Collection on Upper Division Physics Courses.] In response to the need for a scalable, institutionally supported model of educational change, the Science Education Initiative (SEI) was created as an experiment in transforming course materials and faculty practices at two institutions -University of Colorado Boulder (CU) and University of British Columbia. We find that this departmentally focused model of change, which includes an explicit focus on course transformation as supported by a discipline-based postdoctoral education specialist, was generally effective in impacting courses and faculty across the institution. In CU's Department of Physics, the SEI effort focused primarily on upper-division courses, creating high-quality course materials, approaches, and assessments, and demonstrating an impact on student learning. We argue that the SEI implementation in the CU Physics Department, as compared to that in other departments, achieved more extensive impacts on specific course materials, and high-quality assessments, due to guidance by the physics education research group-but with more limited impact on the departmental faculty as a whole. We review the process and progress of the SEI Physics at CU and reflect on lessons learned in the CU Physics Department in particular. These results are useful in considering both institutional and faculty-led models of change and course transformation. I. NEED FOR A MODEL OF CHANGEThere is a rising tide of attention to the improvement of undergraduate science, technology, engineering, and mathematics (STEM) teaching. This attention is driven by the increasing emphasis on science and technology as the current and future driver of national economic growth and the results of discipline-based education research (DBER), which is indicating clear opportunities for improvement. We see evidence of this increased attention in numerous calls for action from disciplinary societies, national networks, and federal agencies [1][2][3][4][5][6].However, as long as the impetus, success, and preservation of change relies on single individuals, rather than organizational structures that support that change, teaching innovations will be inherently fragile, and scaling up those efforts will be challenging. Previous research has demonstrated the high rate of loss of both the number and fidelity of use of instructional innovations [7,8]. A focus on individual decision making and creation of instructional materials is also inherently inefficient due to duplication of effort, as faculty continually reinvent the wheel. A new model for change is needed that guides coherent collective efforts by making innovative, effective teaching an institutional goal that is embedded in the established organizational structures. Such a model would allow this rising tide of attention to STEM education to lift our collective boats towards a higher goal, rather than leaving STEM faculty paddling around on their own in an attempt to keep their heads above water.As a field, scholars ...

“…These rubrics were developed and validated using the same process as the rubrics for the CUE and QMAT; however, the more "all-or-nothing" focus makes the CCMI and CURrENT rubrics considerably simpler [25,28]. Typically, these grading schemes do not award points based on intermediate steps or for partially correct or incomplete responses.…”

Section: Scoringmentioning

confidence: 99%

“…Typically, these grading schemes do not award points based on intermediate steps or for partially correct or incomplete responses. Minimal training is required to achieve a high degree of interrater reliability using these rubrics [25,28]. One potential trade-off of the simpler CCMI rubric is that it is not necessary to include as many examples of common incorrect responses that can help an instructor recognize or anticipate common student difficulties.…”

Section: Scoringmentioning

confidence: 99%

“…One potential trade-off of the simpler CCMI rubric is that it is not necessary to include as many examples of common incorrect responses that can help an instructor recognize or anticipate common student difficulties. To counteract this, we have begun creating a second "difficulties" rubric for the CCMI [28]. This rubric is not designed to provide numerical scores, but instead to describe common student difficulties with each of the questions and present examples of what these difficulties look like.…”

Section: Scoringmentioning

confidence: 99%

See 1 more Smart Citation

Development and uses of upper-division conceptual assessments

Wilcox

Phys. Rev. ST Phys. Educ. Res.

Baily

et al. 2015

Self Cite

[This paper is part of the Focused Collection on Upper Division Physics Courses.] The use of validated conceptual assessments alongside conventional course exams to measure student learning in introductory courses has become standard practice in many physics departments. These assessments provide a more standard measure of certain learning goals, allowing for comparisons of student learning across instructors, semesters, institutions, and pedagogies. Researchers at the University of Colorado Boulder have developed several similar assessments designed to target the more advanced physics of upper-division classical mechanics, electrostatics, quantum mechanics, and electrodynamics courses. Here, we synthesize the existing research on our upper-division assessments and discuss some of the barriers and challenges associated with their development, validation, and implementation as well as some of the strategies we have used to overcome these barriers.