Online assessment of students’ reasoning when solving example-eliciting tasks: using conjunction and disjunction to increase the power of examples

Abstract

We argue that examples can do more than serve the purpose of illustrating the truth of an existential statement or disconfirming the truth of a universal statement. Our argument is relevant to the use of technology in classroom assessment. A central challenge of computer-assisted assessment is to develop ways of collecting rich and complex data that can nevertheless be analyzed automatically. We report here on a study concerning a dedicated design pattern of a special kind of task for assessing student's reasoning when establishing the validity of geometry statements that go beyond a single case, concerning the similarity of triangles. In each task students are given three relations that exist either in every triangle or in special types. Their task is to verify, by creating an example, claims that argue for logically compounded claims built out of the given relations. 50 students, aged 15-16, were asked to verify or disprove claims. Their submissions were automatically characterized along categories based on the correctness of the claims they chose and the examples they used to support the claims. We focus on characterizing the properties of the conjunction/disjunction design for automatically assessing conceptions related to examples generated by the learner with interactive diagrams. Our analysis shows that our automated scoring environment, which supports interactive example-eliciting-tasks, and the design principles of conjunction and disjunction of geometric relations, enable one to assess students' exploration of the logic of universal claims, characterize successful and partial answers, and differentiate between students according to their work.

Transformation of the mathematics classroom with the internet

“…The papers of Bennison, Goos and Geiger (2020), and Yerushalmy and Olsher (2020) show how mathematics educators should take into account the conceptual frame underpinning the design of new digital tools (to provide opportunities for teacher professional learning and to asses students' mathematical learning). Explicit principles of design to develop tools (and determine ways of using them), is a growing field in mathematics teacher education (Van Es et al 2020) and teaching mathematics (Leung and Baccaglini-Frank 2017).…”

Pedagogical Considerations for Designing Automated Grouping Systems: the Case of the Parabola

“…This article explored pedagogical considerations for designing content-specific group formation recommendation modules (GFRMs) in terms of the combination among task, personal considerations, and the group formation approach (interpersonal considerations). We combined two learning-instruction principles—example-eliciting tasks (Yerushalmy et al, 2017 ; Yerushalmy & Olsher, 2020 ) and dialogic learning (Wegerif, 2011 ; Wegerif & Major, 2019 )—and applied them in the context of a learning analytics system that can be used to support automated formative assessment in mathematics.…”

“…Breaking down a task to its fundamental mathematical aspects supported a fine-grained analysis of students’ learning sequences. Other tasks developed in our laboratory are geared to probe for students’ PESs regarding various mathematical concepts for many different curricular-focused activities, such as asymptotes, linear functions, and triangles (Olsher et al, 2016 ; Yerushalmy & Olsher, 2020 ). For example, in a current task, we ask students to create three different examples of triangles that will be analyzed via several interweaving aspects, regarding angles (right, acute, or obtuse), line segments (equilateral, isosceles, or scalene), and segment-positioning (vertical, horizontal, or one vertical and one horizontal).…”

“…We exemplified this diversity in the work of four students who were each engaged in a different interpersonal set-up (Table 5 ). This diversity of these PESs indicates the potential of example-eliciting tasks and accompanied analytics in fostering content-specific idiosyncratic learning of specific content (Olsher et al, 2016 ; Yerushalmy & Olsher, 2020 ). Some students’ solutions to the example-eliciting tasks in this experiment unveiled an ambiguity in the analytics—cases in which the form of the function was a hybrid between vertex and polynomial forms (such as in Celia and Paula’s joint examples for task 2; see also Table 5 , row C and D).…”

“…In our laboratory at the University of Haifa, we developed an automated formative assessment system we termed Seeing the Entire Picture (STEP) (Olsher et al, 2016 ). The type of task we use to probe for students’ PESs is called an example-eliciting task —open tasks in which students are asked to generate several examples of a mathematical concept under a closed set of constraints (Yerushalmy et al, 2017 ; Yerushalmy & Olsher, 2020 ).…”