The importance of improving students' understanding of core concepts in mathematics is well established. However, assessing the impact of different teaching interventions designed to improve students' conceptual understanding requires the validation of adequate measures. Here we propose a novel method of measuring conceptual understanding based on comparative judgement (CJ). Contrary to traditional instruments, the CJ approach allows test questions for any topic to be developed rapidly. In addition, CJ does not require a detailed rubric to represent conceptual understanding of a topic, as it is instead based on the collective knowledge of experts. In the current studies, we compared CJ to already established instruments to measure three topics in mathematics: understanding the use of p-values in statistics, understanding derivatives in calculus, and understanding the use of letters in algebra. The results showed that CJ was valid as compared to established instruments, and achieved high reliability. We conclude that CJ is a quick and efficient alternative method of measuring conceptual understanding in mathematics and could therefore be particularly useful in intervention studies.

No abstract

This paper reports on a collaborative exercise designed to generate a coherent agenda for research on mathematical cognition. Following an established method, the exercise brought together 16 mathematical cognition researchers from across the fields of mathematics education, psychology and neuroscience. These participants engaged in a process in which they generated an initial list of research questions with the potential to significantly advance understanding of mathematical cognition, winnowed this list to a smaller set of priority questions, and refined the eventual questions to meet criteria related to clarity, specificity and practicability. The resulting list comprises 26 questions divided into six broad topic areas: elucidating the nature of mathematical thinking, mapping predictors and processes of competence development, charting developmental trajectories and their interactions, fostering conceptual understanding and procedural skill, designing effective interventions, and developing valid and reliable measures. In presenting these questions in this paper, we intend to support greater coherence in both investigation and reporting, to build a stronger base of information for consideration by policymakers, and to encourage researchers to take a consilient approach to addressing important challenges in mathematical cognition.