Developing a Rubric to Assess Student Learning Outcomes Using a Class Assignment

Abstract

We developed a rubric to assess several of our department's undergraduate student learning outcomes (SLOs). Target SLOs include applications of principles of research methodology, using appropriate statistics, adherence to the Publication Manual of the American Psychological Association, and written communication skills. We randomly sampled 20 percent (N = 55) of the final written manuscripts from several sections of a research methods course and trained 2 graduate-level raters to use the rubric to score the students' papers. We found statistically significant interrater reliability and convergent validity coefficients. These findings are discussed to encourage the development and evaluation of such rubrics to be used across colleges and universities.In the last decade, undergraduate psychology programs have responded to the need for assessment as a legitimate force in higher education (see Dunn, Mehrotra, & Halonen, 2004). As such, there is an increasing interest in multimethod assessment of student learning outcomes and the development of reliable and valid rubrics to directly measure student learning (e.g., Stellmack, Konheim-Kalkstein, Manor, Massey, & Schmitz, 2009). However, it is quite challenging to develop rubrics that reliably assess student learning based on oral and visual presentations, group work and discussions, final projects, written reports, and other culminated student work.Halonen et al. (2003) created a rubric specifically designed for psychology departments to assess undergraduate students' scientific inquiry skills. Scientific inquiry is defined as forming hypotheses, designing experiments, interpreting outcomes, and communicating results. These four domains converge with McGovern and Hawks's (1986) conclusions on what defines psychology student learning objectives (SLOs)-specifically that scientific reasoning and proficiencies in research methods and communication are their principal components.The rubric that Halonen and colleagues (2003) designed is comprehensive and broad, incorporating elements of communication, collaboration, and selfassessment, as well as scientific inquiry skills. Each domain has five levels of proficiency, ranging from before training to professional graduate and beyond. They designed the rubric to measure a student's progress throughout his or her academic experience, from as early as high school up to college graduation.One limitation Halonen and colleagues (2003) acknowledged is the time and energy required to effectively use a comprehensive multidimensional rubric. Consequently, some educators might lose enthusiasm in this rubric's implementation. This limitation can be addressed by focusing on a simpler rubric and using it as one part of a multimethod model of assessment. Furthermore, Halonen and colleagues' rubric was not empirically tested for reliability and validity. The authors hoped that their article would encourage other researchers to investigate the potential of developing their own rubrics to quantify and assess psychology education achievement. It ...

Grading rubrics: hoopla or help?

“…A relatively large literature base exists on particular aspects of rubric use, with much attention paid to the philosophical rationale of use; design, development, and implementation; and construct validity (Hafner & Hafner, 2003;Stemmack, Konheim-Kalkstein, Manor, Massey & Schmitz, 2009;Thaler, Kazemi, & Huscher, 2009). With respect to the effects of rubric use on academic result/outcome, however, the research base is smaller in size (Price, 2005).…”

Rúbricas para la evaluación de la práctica profesional, el internado y el examen de titulación para la carrera de cinesiología según competencias generales y específicas, perfil de egreso y proyecto educativo institucional, Universidad de Antofagasta

“…Los instrumentos de evaluación de práctica profesional, internado y examen de titulación no poseían formato, estructura, contenidos y descriptores de calificación que los avalaran como instrumentos adecuados, no se ajustaban a los actuales resultados de aprendizaje y en particular a las competencias genéricas de la universidad y específicas consideradas en el rediseño curricular de la carrera según el área de desempeño. Además, no respondían al actual PEI de la UA [1][2][3][4].…”

Design and Validation of a Rubric to Assess the Use of American Psychological Association Style in scientific articles

“…The measurement of reliability in an instrument, per Hernández, Fernández and Baptista (2003), is the extent to which its repeated application to the same subject produces the same results. Similarly, the validity, or repeatability, as argued by Moskal and Leydens (2000), Stellmack et al (2009) andThaler, Kazemi andHuscher (2009), is the extent to which the instrument measures the variables' underlying interest. Finally, in the literature associated with the rubric's design the term precision appears (Brown, Glasswell, & Harland, 2004), which refers to the psychometric position of quality between validity and reliability indicators, which is normally expressed by an exact percentage of correlation between the two factors (Cho, Schunn, & Wilson, 2006;Jonsson & Svingby, 2007).…”