Understanding variations between student and teacher application of rubrics

Li, Jinrong; Lindsey, Peggy

doi:10.1016/j.asw.2015.07.003

Cited by 41 publications

(29 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although scoring tendencies of raters cannot be controlled, the reliability of the scoring can be increased by the evaluation method chosen. Giving criteria to raters for the scoring process affects their assessment (Eckes, 2008;Li & Lindsey, 2015;Schaefer, 2008;Tan & Turner, 2015). The fact that no criteria were given during the general evaluation and the different behaviours of the raters under the general evaluation condition support this opinion.…”

Section: Discussion and Future Directionsmentioning

confidence: 99%

“…Therefore, rubrics are the best way to assess complex competencies without compromising reliability and validity (Morrison & Ross, 1998;Wiggins, 1998). Considering that giving criteria to raters will affect evaluation process positively (Eckes, 2008;Eckes, 2012;Li & Lindsey, 2015;Tan & Turner, 2015), it is expected that scoring will be objective when using a rubric. However, in this study, the most objective scoring was obtained when the rating scale was used.…”

Section: Rubric Tells Both Teachers and Students What Is Importantmentioning

confidence: 99%

See 1 more Smart Citation

Investigation of Rater Tendencies and Reliability in Different Assessment Methods with Many Facet Rasch Model

Koçak¹

2020

iejee

View full text Add to dashboard Cite

One of the most commonly used methods for measuring higher-order thinking skills such as problem-solving or written expression is open-ended items. Three main approaches are used to evaluate responses to open-ended items: general evaluation, rating scales, and rubrics. In order to measure and improve problem-solving skills of students, firstly, an error-free measurement process should be performed. Errors caused by raters such as bias, high or low tendency to score is a common problem in the evaluation of open-ended items as they adversely affect the accuracy of decisions to be made. This study utilized open-ended items to evaluate the raters' tendencies in terms of general evaluation, rating scale, and rubric conditions. The raters' behaviours in each assessment method and their opinions about the assessment methods were determined. The participants of the study consisted of 12 different mathematics teachers and the Many Facet Rasch Model was adopted for the analyses. The scoring reliability of each method was estimated. The findings of the rating scale revealed that the raters had a more homogeneous scoring tendency. In addition, while the majority of raters stated that they prefer to use a rubric, they also stated it is the most difficult method to use.

show abstract

Section: Discussion and Future Directionsmentioning

confidence: 99%

Section: Rubric Tells Both Teachers and Students What Is Importantmentioning

confidence: 99%

Investigation of Rater Tendencies and Reliability in Different Assessment Methods with Many Facet Rasch Model

Koçak¹

2020

iejee

View full text Add to dashboard Cite

show abstract

“…In addition to their large-scale national and international use, the TOEFL and IELTS rating scales are used or adapted by schools for their own language assessments (Becker 2010: 126), and the language of these descriptors impacts programs' recommended scores for admission (Golder et al 2011). While not trained as raters -students, teachers, and schools are the intended audience of these publicly available rating scales; and in the absence of training, different audiences often have different interpretations of the language of rating scales (Li & Lindsey 2015), which sometimes proves vague or underspecified even for trained raters. Schaefer (2008) was actually interested in how raters with less training, who would approach the essays and rating process more as "lay readers" (2008: 471), might respond to L2 writing and the rating process.…”

Section: Discussionmentioning

confidence: 99%

Differences Across Levels in the Language of Agency and Ability in Rating Scales for Large-Scale Second Language Writing Assessments

Anderson

2017

Studia Anglica Posnaniensia

View full text Add to dashboard Cite

While large-scale language and writing assessments benefit from a wealth of literature on the reliability and validity of specific tests and rating procedures, there is comparatively less literature that explores the specific language of second language writing rubrics. This paper provides an analysis of the language of performance descriptors for the public versions of the TOEFL and IELTS writing assessment rubrics, with a focus on linguistic agency encoded by agentive verbs and language of ability encoded by modal verbs can and cannot. While the IELTS rubrics feature more agentive verbs than the TOEFL rubrics, both pairs of rubrics feature uneven syntax across the band or score descriptors with either more agentive verbs for the highest scores, more nominalization for the lowest scores, or language of ability exclusively in the lowest scores. These patterns mirror similar patterns in the language of college-level classroom-based writing rubrics, but they differ from patterns seen in performance descriptors for some large-scale admissions tests. It is argued that the lack of syntactic congruity across performance descriptors in the IELTS and TOEFL rubrics may reflect a bias in how actual student performances at different levels are characterized.

show abstract

“…Según este estudio, determinados factores puede moderar el efecto de las rúbricas en el aprendizaje, como las habilidades In-Red (2016) metacognitivas o el género del estudiante, por lo que recomienda analizar la influencia de las características personales en la evaluación formativa. De hecho, otro estudio posterior profundiza en las diferencias observadas en las evaluaciones e interpretaciones hechas por estudiantes y profesores, ante los mismos ensayos y redacciones elaboradas por estudiantes matriculados en un curso de escritura (Li y Lindsey, 2015).…”

Section: Análisis De Fiabilidad Y Validez Aplicado a Rúbricasunclassified

Fiabilidad y validez de rúbricas para la evaluación de las competencias «Comunicación» y «Diseño y proyecto»

Candela¹,

Conchado²,

Egea³

et al. 2016

Libro De Actas IN-RED 2016 - II Congreso Nacional De Innovación Educativa Y De Docencia en Red

View full text Add to dashboard Cite

show abstract

Understanding variations between student and teacher application of rubrics

Cited by 41 publications

References 19 publications

Investigation of Rater Tendencies and Reliability in Different Assessment Methods with Many Facet Rasch Model

Investigation of Rater Tendencies and Reliability in Different Assessment Methods with Many Facet Rasch Model

Differences Across Levels in the Language of Agency and Ability in Rating Scales for Large-Scale Second Language Writing Assessments

Fiabilidad y validez de rúbricas para la evaluación de las competencias «Comunicación» y «Diseño y proyecto»

Contact Info

Product

Resources

About