A Problematic Leap in the Use of Test Data: From Performance to Inference

Matters, Gabrielle

doi:10.1007/978-1-4020-9964-9_11

Cited by 8 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We note that Mathematical Literacy and its assessment have been introduced relatively recently into the South African high school curriculum. We agree with Matters (2009) that assessment in the 21st century has a powerful influence, but this influence is only warranted if the assessment is of a sufficient quality to support the inferences, in this case the inferences about the mathematical literacy proficiency of learners that are drawn from the test results. The assessment process involves the theoretical exploration of the construct of mathematical literacy, the operationalisation of the construct in items designed to gauge proficiency, the compilation of a test instrument and the administration and marking.…”

Section: Assessment and Measurementsupporting

confidence: 67%

An investigation of Mathematical Literacy assessment supported by an application of Rasch measurement

2014

View full text Add to dashboard Cite

Mathematical Literacy (ML) is a relatively new school subject that learners study in the final 3 years of high school and is examined as a matric subject. An investigation of a 2009 provincial examination written by matric pupils was conducted on both the curriculum elements of the test and learner performance. In this study we supplement the prior qualitative investigation with an application of Rasch measurement theory to review and revise the scoring procedures so as to better reflect scoring intentions. In an application of the Rasch model, checks are made on the test as a whole, the items and the learner responses, to ensure coherence of the instrument for the particular reference group, in this case Mathematical Literacy learners in one high school. In this article, we focus on the scoring of polytomous items, that is, items that are scored 0, 1, 2 … m. We found in some instances indiscriminate mark allocations, which contravened assessment and measurement principles. Through the investigation of each item, the associated scoring logic and the output of the Rasch analysis, rescoring was explored. We report here on the analysis of the test prior to rescoring, the analysis and rescoring of individual items and the post rescore analysis. The purpose of the article is to address the question: How may detailed attention to the scoring of the items in a Mathematical Literacy test, through theoretical investigation and the application of the Rasch model, contribute to a more informative and coherent outcome?

show abstract

Section: Assessment and Measurementsupporting

confidence: 67%

An investigation of Mathematical Literacy assessment supported by an application of Rasch measurement

2014

View full text Add to dashboard Cite

show abstract

“…Si bien lo deseable es que el uso de este tipo de evaluaciones derive en una mejora general de los sistemas educativos, hay usos que desembocan en efectos no deseados. Matters (2009) indica que una acción común es que las ciudades y países se definan en función de los resultados de evaluaciones como PISA.…”

Section: Efectos Del Uso De Los Resultadosunclassified

“…Por su parte, Dussel y otros (2005) encontraron, a partir de una investigación realizada en las provincias de Córdoba, Chaco y la Ciudad Autónoma de Buenos Aires en Argentina, que los docentes consideran que el impacto de los sistemas de evaluación ha sido poco visible o incluso nulo, debido a situaciones tales como "[…] la falta de acceso, el desconocimiento, la escasa necesidad de estadísticas para la labor docente, la falta de presupuesto para modificar situaciones ya identificadas como problemáticas, o porque cada docente está en sus temas específicos" (p. 377). Matters (2009) afirma que, en este siglo, la información derivada de las evaluaciones es extremadamente poderosa, siempre y cuando cumpla con dos condiciones. La primera de ellas consiste en que la prueba, por sí misma, sea de calidad y lo suficientemente fuerte para fundamentar los usos que se hacen a partir de la información que arroja; mientras que la segunda condición señala que los usuarios de esta información deben poseer suficiente pericia para reconstruir adecuadamente la realidad que subyace a los resultados.…”

Section: Efectos Del Uso De Los Resultadosunclassified

“…Para llevar a cabo esto se requiere, como condiciones indispensables, que el propio instrumento de evaluación posea suficiente capacidad para generar un diagnóstico del aprendizaje de los alumnos y que los profesores y, de manera general, los actores educativos que reciben, analizan y usan los resultados, tengan habilidades para la interpretación y el uso de este tipo de datos (Griffin, 2009). Matters (2009) indica que las pruebas estandarizadas pueden ofrecer información a propósito no sólo de lo que los alumnos aprendieron, sino también de aquello que no aprendieron. Ante esto, es necesario que los docentes puedan identificar las condiciones en las cuales los alumnos llegaron o no a determinado nivel de logro.…”

Section: Análisis E Interpretación De La Informaciónunclassified

See 1 more Smart Citation

Una Propuesta de Ejes para el Análisis del Uso de Resultados de Evaluaciones a Gran Escala del Aprendizaje por Docentes y Directores

Cuéllar¹,

Robledo²

2018

RIEE

View full text Add to dashboard Cite

“…Anxiety in learning is considered to be a student factor of the presage stage of the 3P model (Matters, 2009). Anxiety in learning has been associated with surface approaches to learning in various studies which have been selected here because they draw on approaches to learning and other factors that are associated with the 3P model (Balasooriya, et al, 2009;Papinczak, 2009).…”

Section: Anxiety In Learningmentioning

confidence: 99%

Malaysian Tertiary Students' Perceptions of Constructive Alignment in Learning in EAP Classrooms: A Qualitative Study

Vijayan¹

View full text Add to dashboard Cite

<p>English for Academic Purposes (EAP) is offered as a course in many Malaysian institutions of higher learning. In contrast with English taught at secondary schools, at the tertiary level EAP courses primarily cater to learners’ language needs that arise from learning in a range of disciplines, in the workplace, and eventually in the wider society. This study explores 12 Malaysian tertiary students’ perceptions of how learning takes place in EAP classrooms during reading activities. Biggs’ Constructive Alignment framework and the 3P model (Presage, Process and Product) provide a theoretical framework. The research investigates how students’ factors in learning and the tasks given in the classrooms align with the students’ views of outcomes in learning and by considering the constructive alignment of the student factors, the task and the outcomes (the presage, process and product stages), it is possible to see the potential for deep approaches to learning and to consider whether that potential is realised in particular tasks. This study uses a single embedded explanatory case, consisting of 12 Malaysian tertiary students from three EAP classrooms in a public university in Malaysia. The data collection methods used for this study were semi-structured interviews, stimulated recall interviews and students’ written samples of EAP tasks undertaken in the three classrooms. The tasks were adapted from an EAP workbook used at the university. The data were analysed using Braun and Clarke’s thematic analysis. An analytic tool was created using Biggs’ and Collis’ SOLO Taxonomy to ascertain the potential of the tasks to allow a deep approach to learning. The findings of this study showed the complexity of the learning process among these learners in the academic English classroom. The findings showed that the learners did not see the alignment between the presage, process and product stages. In the presage and process stages, the findings showed that there were multiple factors such as background, motives for learning, anxiety in learning, topic and content interests, and prior knowledge that influenced the learners’ engagement with the tasks. Motives for learning were particularly important since the other factors either influenced or were influenced by them. Further, these factors also influenced students’ perceived outcomes of learning in the EAP classroom. In the process and product stages, the students reported that they found the content taught in class did not align with their motives for learning. They also said that they had difficulties seeing the transfer of learning from the content taught in the EAP classroom to their other subjects at the university. Although the students showed some interest and engagement with the reading tasks in the classroom, the lack of alignment could contribute to a surface motive for learning in the EAP classroom. This study also found that a process of internal compromise took place within the learners to adapt themselves to the learning situations in the EAP classroom which was clearly evident in their responses to the reading tasks in the classroom as well as in their opinions about learning in academic English as a whole. Biggs’ concept that learning takes place within a system is particularly important in EAP courses where the learning should be designed to transfer to students’ achievements in other subjects, in the workplace and beyond. An understanding of constructive alignment in EAP courses will enable EAP course designers, material writers and EAP instructors to use this powerful tool to support the achievement of the aims of EAP courses. It also has implications for EAP educators when they consider the design of tasks in their courses and the issues that affect the potential for deep or surface approaches to learning.</p>

show abstract

A Problematic Leap in the Use of Test Data: From Performance to Inference

Cited by 8 publications

References 9 publications

An investigation of Mathematical Literacy assessment supported by an application of Rasch measurement

An investigation of Mathematical Literacy assessment supported by an application of Rasch measurement

Una Propuesta de Ejes para el Análisis del Uso de Resultados de Evaluaciones a Gran Escala del Aprendizaje por Docentes y Directores

Malaysian Tertiary Students' Perceptions of Constructive Alignment in Learning in EAP Classrooms: A Qualitative Study

Contact Info

Product

Resources

About