An investigation of Mathematical Literacy assessment supported by an application of Rasch measurement

Abstract

Mathematical Literacy (ML) is a relatively new school subject that learners study in the final 3 years of high school and is examined as a matric subject. An investigation of a 2009 provincial examination written by matric pupils was conducted on both the curriculum elements of the test and learner performance. In this study we supplement the prior qualitative investigation with an application of Rasch measurement theory to review and revise the scoring procedures so as to better reflect scoring intentions. In an application of the Rasch model, checks are made on the test as a whole, the items and the learner responses, to ensure coherence of the instrument for the particular reference group, in this case Mathematical Literacy learners in one high school. In this article, we focus on the scoring of polytomous items, that is, items that are scored 0, 1, 2 … m. We found in some instances indiscriminate mark allocations, which contravened assessment and measurement principles. Through the investigation of each item, the associated scoring logic and the output of the Rasch analysis, rescoring was explored. We report here on the analysis of the test prior to rescoring, the analysis and rescoring of individual items and the post rescore analysis. The purpose of the article is to address the question: How may detailed attention to the scoring of the items in a Mathematical Literacy test, through theoretical investigation and the application of the Rasch model, contribute to a more informative and coherent outcome?

Towards validation of a rational number instrument: An application of Rasch measurement theory

“…The application of the Rasch model is premised on the particular strength implicit in the model: that both item and person parameters are aligned on the same scale (Wei, Liu, & Jia, 2014). By http://www.pythagoras.org.za Open Access considering both the validity and reliability of the test items, and of the person responses, the Rasch model identifies aspects for further improvement as well as signs of biases (if any) (Smith, 2004;Bond & Fox, 2007, 2015Long, Debba, & Bansilal, 2014;Bansilal, 2015).…”

A Rasch analysis of a Grade 12 test written by mathematics teachers

“…The mean ability of the four groups becomes the horizontal coordinate of points in the diagrams and the vertical coordinates depict the probability of answering correctly. 18 When the theoretical curve (the expected frequencies) and the observed proportions (the empirically established average of the actual item scores in the four chosen groups) are in alignment, we assume fit to the model, as depicted by the ICC in the right-hand side of Figure 6. On the other hand, a substantial deviation of the observed proportions from the theoretical curve signals some sort of misfit between the data and the model (for example, the first ICC on the left-hand side of Figure 6).…”

“…On the other hand, a substantial deviation of the observed proportions from the theoretical curve signals some sort of misfit between the data and the model (for example, the first ICC on the left-hand side of Figure 6). There are four categorisations that describe how the observed proportions might relate to the theoretical expectation; these categorisations are described in detail with examples in an article by Long et al 18 Each of the 24 items was checked for the item fit statistics, ICC, as well as the functioning of the categories. Based on these diagnostic procedures, some items were rescored if such a rescoring was supported by the qualitative analysis, as described in the discussion following Figure 7.…”

The relationship between mathematical literacy and visual math literacy self-efficacy perceptions of middle school students

“…Uysal and Yenilmez (2011) determined eighthgrade students' mathematical literacy levels. Long, Bansilal and Debba (2014) analyzed mathematical literacy with Rasch model. Besides, it was determined in the related literature that the studies about mathematical literacy had been carried out within the scope of PISA [Programme for International Student Assessment] (Akyüz & Pala, 2010;Çetin & Gök, 2017;Koğar, 2015;Stacey, 2011).…”