No abstract

ÖZET Bu çalı manın amacı, dereceli puanlama anahtarı (rubric) kullanarak ö rencilerin üst düzey zihinsel becerilerini ifade etme yollarından biri olan sunu yapma davranı larını de erlendirmektir. Ara tırma, bir özel okulun 24 ö rencisi üzerinde yürütülmü tür. Sunu yapma davranı ı için

When students present writing tasks that require higher order thinking skills to work, one of the most important problems is scoring these writing tasks objectively. The fact that raters give scores below or above their performance based on several environmental factors affects the consistency of the measurements. Inconsistencies in scoring negatively affect the validity and reliability of student performance and cause the scores obtained to be questioned. In regard to the validity and reliability of these measurements, it is significant to identify the rater behavior and correct the sources of error. This study aims to analyze the differential rater functioning (DRF), which is one of the problematic rater behaviors, in evaluating compositions written by middle school 7th-grade students within the scope of the Turkish course. 86 students attending a public school were participated the study. Students' compositions were rated using an analytical rubric by 8 teachers from different institutions. In this correlational research, the many facet Rasch model was used, and five variables including students, raters’ and, students’ gender, students’ qualification, and evaluation criteria were examined. it was examined whether the raters show DRF on an individual and group basis based on the dual interaction analysis, including the gender of the student x rater and the student's competence x rater. The findings have revealed that DRF at the group level does not interfere with the measurements, while the individual level DRF is involved in the measurements. It was determined that the level of DRF mixing in the measurements of successful students was the lowest. Especially rigid and lenient raters were found to show DRF. In the present study, it was observed that the raters showing DRF was also the most lenient raters, while these raters did not show DRF in terms of the gender of the student.

Among the sub-categories of assessment types, alternative assessment including peer-assessment helps students develop metacognition. The reliability and validity issues are the most significant problem in peer-assessment. Furthermore, the studies focusing on rater bias in peer review are limited. This study investigated the rater severity drift, which is one of the rater effects, in peer-assessment. The performance of 8 oral presentations based on group work in Science and Technology course was scored by 7th grade students (N=28) by using the rubric that researchers developed. The presentations lasted for four days with two presentations in each day. While examining the time-dependent drift in rater severity in peer-assessment, the many-Facet Rasch Measurement model was used. Two indexes (interaction term and standardized differences) were calculated with many-Facet Rasch Measurement with an aim to determine whether the raters made rater severity drift on a group or an individual basis. The analysis examined the variance of scores in following days in comparison with the scores of the 1st day. Accordingly, the two methods used to determine rater severity shift gave similar results, and some raters at the individual level tended to be more strict or lenient over time although there was no significant rater drift at the group level, showing that drifts had no specific models.