Reliability and Perceived Pedagogical Utility of a Weighted Music Performance Assessment Rubric

Abstract

The purpose of this study was to investigate the reliability and perceived pedagogical utility of a multidimensional weighted performance assessment rubric used in Kansas state high school large-group festivals. Data were adjudicator rubrics (N = 2,016) and adjudicator and director questionnaires (N = 515). Rubric internal consistency was moderately high (.88). Dimension reliability ranged from moderately low (W = .47) to moderate (W = .77). Total score reliability was moderately high (W = .80) and rating reliability was moderate (W = .72). Findings suggested that reliability on the whole was within the range of previously researched music performance assessment tools. Questionnaire results suggested that the rubric provided a better instrument for justifying ratings and more detailed descriptions of what constituted acceptable performances than previously researched nonrubric forms; hence, adjudicators and directors perceived the rubric as possessing improved pedagogical utility.

Large-Group Contest Ratings and Music Teacher Evaluation: Issues and Recommendations

“…Norris and Borst (2007) found that interrater reliability was stronger for individual categories (e.g., tone, intonation, technique) and final scores when judges evaluated choral ensembles using a rubric with detailed descriptions for each possible score (1-5) in all categories as opposed to a traditional form with no category descriptions. Nonetheless, reliability for categories at largegroup festivals in Kansas varied from .47 (other) to .77 (tone), even with descriptors for each level of achievement (Latimer, Bergee, and Cohen 2010).…”

“…Several authors have found that judges were often inconsistent in scoring individual captions , even though final ratings demonstrated a high level of interrater reliability (e.g., Burnsed, Hinkle, and King 1985;Latimer, Bergee, and Cohen 2010). These findings suggest that judges might first determine a final rating and then score captions to add up to the predetermined result.…”

Assessment of instrumental music performance: definitions, criteria, measurement

“…The use of rubrics allows judges to evaluate the performance more reliably (DeLuca and Bolden, 2014;Jonsson and Svingby, 2007). They are helpful in evaluating the performance of persons playing different instruments, at different stages of education (Ciorba and Smith, 2009;Latimer, Bergee and Cohen, 2010).…”

Learner autonomy, microcredentials and self-reflection: a review of a Moodle-based medical English review course

“…In educational settings it is agreed assessments should be 'reliable' (measurements of the same skills or knowledge produce the same results), 'consistent' (all skills and knowledge are evaluated by identical procedures), and 'transparent' (assessors and learners all share the same levels of expectation) (Latimer et al, 2010). Increasingly carefully designed assessment rubrics are being used to ensure reliability, consistency and transparency (Ciorba and Smith, 2009 In this English medical course a rudimentary assessment rubric was developed that defined the requirements for specific learning activities in a medical terminology quiz.…”