Evaluation of open items using the many-facet Rasch model

A new approach to measuring Overall Liking with the Many-Facet Rasch Model

“…Therefore, different fit statistics are used in different Rasch software, depending on the estimation methods implemented. The method of comparing the absolute standardized residuals (Eckes, 2011), that has been described in Section 4.2.2., has been used by some authors for assessing the global model fit for MFR models using FACETS (Eckes, 2005;Toffoli, Andrade, & Bornia, 2016). The three original Rasch models and the models with their revised scales all showed acceptable model fit, when comparing their absolute standardized residuals to these criteria (Table 2).…”

An Analysis of Rater Severity and Leniency in Open-ended Mathematic Questions Rated Through Standard Rubrics and Rubrics Based on the SOLO Taxonomy

“…Öğrencilerin objektif olarak puanlanamayan herhangi bir testten aldığı puan, testi puanlayan kişiye göre farklılık göstermektedir (Tekin, 2009). Alanyazında bu durumu örneklendiren çalışmalar Güler, 2008;Kan, 2005;Koretz, McCaffrey, Klein, Bell ve Stecher, 1992;Toffoli, Andrade ve Bornia, 2016) bulunmaktadır. Örneğin, Koretz ve diğerlerinin (1992) yaptığı çalışmada öğrencilerin matematik performansları dörtlü derecelemeye sahip bir rubrik kullanılarak iki puanlayıcı tarafından puanlanmış ve puanlayıcılar arası uyumun düşük olduğu tespit edilmiştir.…”

A Comparison of Generalizability Theory and Many Facet Rasch Measurement in an Analysis of Mathematics Creative Problem Solving Test

“…In the field of education and psychology, the concern about judgments of raters has been growing. Rater effects such as severity/leniency, central tendency are commonly viewed as a source of method variance, that is, as a source of systematic variance in observed ratings that is related with the raters and not with the examinees (Cronbach, 1995;Eckes, 2005;Engelhard, & Wind, 2013;Toffoli et al, 2016). Since examinees naturally vary in terms of their abilities, we do not expect all of them to receive the same rating.…”