An Alternative Multiple‐Choice Question Format to Guide Feedback Using Student Self‐Assessment of Knowledge

Collignon, Stephane; Chacko, Josey; Martin, Megan Wydick

doi:10.1111/dsji.12213

Cited by 7 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such patterns are indicative of guessing behaviors and are often encountered in learning systems with multiple choice questions and quizzes (Tsai et al, 2012 ). In such cases, attention and/or strategy based feedback might assist the students (Collignon et al, 2020 ).…”

Section: Discussionmentioning

confidence: 99%

Keep Calm and Do Not Carry-Forward: Toward Sensor-Data Driven AI Agent to Enhance Human Learning

Sharma

Lee-Cultura

Giannakos

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

The integration of Multimodal Data (MMD) and embodied learning systems (such as Motion Based Educational Games, MBEG), can help learning researchers to better understand the synergy between students' interactions and their learning experiences. Unfolding the dynamics behind this important synergy can lead to the design of intelligent agents which leverage students' movements and support their learning. However, real-time use of student-generated MMD derived from their interactions with embodied learning systems (MBEG in our case) is challenging and remains under-explored due to its complexity (e.g., handle sensor-data and enable an AI agent to use them). To bridge this gap, we conducted an in-situ study where 40 children, aged 9–12, played MBEG on maths and language development. We automatically, unobtrusively, and continuously monitored students' experiences using eye-tracking glasses, physiological wristbands, and Kinect, during game-play. This allowed us to understand the different cognitive and physiological dimensions of students' progress (right/wrong responses) during the three different stages of the MBEG problem-solving processes, namely the “see-solve-move-respond” (S2MR) cycle. We introduce the novel Carry Forward Effect (CFE); a phenomenon occurring in such games, whereby students propagate, or “carry forward,” the cognitive and physiological effects derived from their MMD, to subsequent phases in the see-solve-move-respond cycle. By identifying moments when the Carry Forward Effect is congruent (or not) to students' learning performance, we uncover opportunities for feedback delivery to encourage or subdue the impact of the CFE. Our results demonstrate the importance of wristband and eye-tracking data as key indicators for prioritizing adaptive feedback to support students in MBEG and emphasize the significance of using MMD to support students' performance in real-time educational settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Keep Calm and Do Not Carry-Forward: Toward Sensor-Data Driven AI Agent to Enhance Human Learning

Sharma

Lee-Cultura

Giannakos

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…Looking at both 'certain when correct' and 'correct when certain' has been proposed as an area for future research, although not specifically for health care. 38 Using assessment responses to explore both efficiency and safety has been studied in the health care context, 19 but only by using logarithmic formula scoring involving the participant's estimation of their likelihood of being correct. 39 Issues with this are, first, that clinicians do not think in terms of percent likelihood correct 33 and, second, that if the results are used to inform high-stakes decisions then people may game the scoring system and respond in ways they think will maximise their score rather than respond as they might in authentic settings.…”

Section: Introductionmentioning

confidence: 99%

“…Other research asked the question ‘Is certainty higher when correct?’, 17,18,21–23,33,36,37 which aligns with efficient decision‐making. Looking at both ‘certain when correct’ and ‘correct when certain’ has been proposed as an area for future research, although not specifically for health care 38 . Using assessment responses to explore both efficiency and safety has been studied in the health care context, 19 but only by using logarithmic formula scoring involving the participant's estimation of their likelihood of being correct 39 .…”

Section: Introductionmentioning

confidence: 99%

Using MCQ response certainty to determine how aspects of self‐monitoring develop through a medical course

Tweed,

Willink,

Wilkinson

2023

Medical Education

View full text Add to dashboard Cite

IntroductionSelf‐monitoring of clinical‐decision‐making is essential for health care professional practice. Using certainty in responses to assessment items could allow self‐monitoring of clinical‐decision‐making by medical students to be tracked over time. This research introduces how aspects of insightfulness, safety and efficiency could be based on certainty in, and correctness of, multiple‐choice question (MCQ) responses. We also show how these measures change over time.MethodsWith each answer on twice yearly MCQ progress tests, medical students provided their certainty of correctness. An insightful student would be more likely to be correct for those answers given with increasing certainty. A safe student would be expected to have a high probability of being correct for answers given with a high certainty. An efficient student would be expected to have a sufficiently low probability of being correct when they have no certainty. The system was developed using first principles and data from one cohort of students. A dataset from a second cohort was then used as an independent validation sample.ResultsThe patterns of aspects of self‐monitoring were similar for both cohorts. Almost all the students met the criteria for insightfulness on all tests. Most students had an undetermined outcome for the safety aspect. When a definitive result for safety was obtained, absence of safety was most prevalent in the middle of the course, while the presence of safety increased later. Most of the students met the criteria for efficiency, with the highest prevalence mid‐course, but efficiency was more likely to be absent later.DiscussionThroughout the course, students showed reassuring levels of insightfulness. The results suggest that students may balance safety with efficiency. This may be explained by students learning the positive implications of decisions before the negative implications, making them initially more efficient, but later being more cautious and safer.

show abstract

“…The information provided by multiple choice tests is richer, meaning that these items can provide more information about students to teachers or other test compilers, especially if these items have high homogeneity (Mutiah et al, 2020;Putri, 2020). The form of multiple-choice tests commonly used by teachers is the usual or conventional multiple-choice form, in which students can choose the most appropriate answer from several alternative answers (Chen et al, 2020;Collignon et al, 2020;McKenna, 2019;Shin et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Effect of Emotional Intelligence on Science Learning Outcomes in Students Given Associational and Conventional Multiple-Choice Tests

Supriyadi

2023

jppipa, pendidikan ipa, fisika, biologi, kimia

View full text Add to dashboard Cite

This study aims to reveal the effect of emotional intelligence (X1) on science learning outcomes (Y) in students who are given multiple choice association tests and conventional multiple-choice tests (X2). The research method used is the experimental method. The population in this study was class IX students at MTs Negeri 7 Model Jakarta. The sampling technique uses cluster random sampling. The research sample consisted of 60 students divided into students who have high and low emotional intelligence. The results of the research instrument trials showed that the reliability of the science learning outcomes instrument in the form of an association multiple choice test was 0.701 and the reliability of the science learning outcomes instrument in the form of conventional multiple-choice tests was 0.827, and emotional intelligence was 0.782. The data analysis used is a two-way ANOVA. The results of the study revealed: (1) The science learning outcomes of students who were given multiple-choice tests of association were higher than those who were given conventional multiple-choice tests. (2) There is an interaction between the form of the test and emotional intelligence on students' science learning outcomes. (3) For students who have high emotional intelligence, the science learning outcomes of students who are given the multiple-choice test form of association are higher than those who are given the conventional multiple choice test form. (4) For students who have low emotional intelligence, the science learning outcomes of students who are given a conventional multiple-choice test are higher than those who are given an association multiple-choice test.

show abstract

An Alternative Multiple‐Choice Question Format to Guide Feedback Using Student Self‐Assessment of Knowledge

Cited by 7 publications

References 31 publications

Keep Calm and Do Not Carry-Forward: Toward Sensor-Data Driven AI Agent to Enhance Human Learning

Keep Calm and Do Not Carry-Forward: Toward Sensor-Data Driven AI Agent to Enhance Human Learning

Using MCQ response certainty to determine how aspects of self‐monitoring develop through a medical course

Effect of Emotional Intelligence on Science Learning Outcomes in Students Given Associational and Conventional Multiple-Choice Tests

Contact Info

Product

Resources

About