Generation and Retrieval Practice Effects in the Classroom Using PeerWise

Kelley, Matthew R.; Chapman-Orr, Elizabeth K.; Calkins, Susanna C; Lemke, Robert J.

doi:10.1177/0098628319834174

Cited by 18 publications

(16 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Regardless of the question selection process, using published student-generated questions on high-stakes examinations introduces a concern that students who either authored or answered the questions prior to the exam may have an unfair advantage. There is evidence that students perform significantly better on subsequent exam questions if they have previously authored or answered questions that merely target the same topics (Kelley et al 2019). Similar research, which addresses a potential bias relating to topic-selection, has shown that these results are robust even when students are randomly assigned topics on which to author questions (Denny et al 2017).…”

Section: Two Approaches For Selecting Student-authored Questionsmentioning

confidence: 88%

Selecting student-authored questions for summative assessments

Huang

Hancock

Clemson

et al. 2021

Research in Learning Technology

View full text Add to dashboard Cite

Production of high-quality multiple-choice questions (MCQs) for both formative and summative assessments is a time-consuming task requiring great skill, creativity and insight. The transition to online examinations, with the concomitant exposure of previously tried-and-tested MCQs, exacerbates the challenges of question production and highlights the need for innovative solutions. Several groups have shown that it is practical to leverage the student cohort to produce a very large number of syllabus-aligned MCQs for study banks. Although student-generated questions are well suited for formative feedback and practice activities, they are generally not thought to be suitable for high-stakes assessments. In this study, we aimed to demonstrate that training can be provided to students in a scalable fashion to generate questions of similar quality to those produced by experts and that identification of suitable questions can be achieved with minimal academic review and editing. Second-year biochemistry and molecular biology students were assigned a series of activities designed to coach them in the art of writing and critiquing MCQs. This training resulted in the production of over 1000 MCQs that were then gauged for potential by either expert academic judgement or via a data-driven approach in which the questions were trialled objectively in a low-stakes test. Questions selected by either method were then deployed in a high-stakes in-semester assessment alongside questions from two academically authored sources: textbook-derived MCQs and past paper questions. A total of 120 MCQs from these four sources were deployed in assessments attempted by over 600 students. Each question was subjected to rigorous performance analysis, including the calculation of standard metrics from classical test theory and more sophisticated item response theory (IRT) measures. The results showed that MCQs authored by students, and selected at low cost, performed as well as questions authored by academics, illustrating the potential of this strategy for the efficient creation of large numbers of high-quality MCQs for summative assessment.

show abstract

Section: Two Approaches For Selecting Student-authored Questionsmentioning

confidence: 88%

Selecting student-authored questions for summative assessments

Huang

Hancock

Clemson

et al. 2021

Research in Learning Technology

View full text Add to dashboard Cite

show abstract

“…Regardless of the question selection process, using published student-generated questions on high-stakes examinations introduces a concern that students who either authored or answered the questions prior to the exam may have an unfair advantage. There is evidence that students perform significantly better on subsequent exam questions if they have previously authored or answered questions that merely target the same topics (Kelley et al, 2019). Similar research, which addresses a potential bias relating to topic-selection, has shown that these results are robust even when students are randomly assigned topics on which to author questions (Denny et al, 2017).…”

Section: Discussionmentioning

confidence: 99%

Selecting Student-Authored Questions for Summative Assessments

Huang¹,

Hancock²,

Clemson³

et al. 2020

Preprint

View full text Add to dashboard Cite

Production of high-quality multiple-choice questions (MCQs) for both formative and summative assessment is a time-consuming task requiring great skill, creativity, and insight. The transition to online examinations, with the concomitant exposure of previously tried-and-tested MCQs, exacerbates the challenges of question production, and highlights the need for innovative solutions.Several groups have shown that it is practical to leverage the student cohort to produce a very large number of syllabus-aligned MCQs for study banks. Although student-generated questions are well suited for formative feedback and practice activities, they are generally not thought to be suitable for high-stakes assessments. In this study, we aimed to demonstrate that training can be provided to students in a scalable fashion to generate questions of similar quality to those produced by experts, and that identification of suitable questions can be achieved with minimal academic review and editing.Biochemistry and Molecular Biology students were assigned a series of activities designed to coach them in the art of writing and critiquing MCQs. This training resulted in the production of over one thousand MCQs that were then gauged for potential by either expert academic judgement, or via a data-driven approach in which the questions were trialled objectively in a low-stakes test. Questions selected by either method were then deployed in a high-stakes in-semester assessment alongside questions from two academically authored sources: textbook-derived MCQs, and past paper questions.A total of 120 MCQs from these four sources were deployed in assessments attempted by over 600 students. Each question was subjected to rigorous performance analysis, including the calculation of standard metrics from classical test theory and more sophisticated Item Response Theory (IRT) measures. The results showed that MCQs authored by students and selected at low cost performed as well as questions authored by academics, illustrating the potential of this strategy for the efficient creation of large numbers of high quality MCQs for summative assessment.

show abstract

“…Generating, solving, and evaluating questions are active tasks that promote various aspects of cognitive and metacognitive learning (Yu, Fu-Yun, 2012;Kelley, Chapman-Orr, Calkins, & Lemke, 2019;Kay, Hardy, & Galloway, 2020). Despite the importance of student question generation (SQG) and its associated activities, teachers do not tend to include it in their teaching, instead focusing students' attention on simply answering questions.…”

Section: Introductionmentioning

confidence: 99%

Question Generation as a Strategy for Advancing Intermediate- and Low-Achieving Students in Higher Education

Aflalo¹

2021

IRHE

View full text Add to dashboard Cite

Student question generation (SQG) is a teaching and learning strategy that promotes higher-order cognitive skills. The purpose of this study is to determine which students gain the most from SQG activities: Is it mainly those with strong academic achievements? The study took place over the course of six years, during which 171 preservice teachers in Israel generated, answered, and peer-evaluated questions at higher and lower orders of thinking. When their exam grades before and after the SQG intervention were checked, the intermediate- and low-achieving students showed the most significant improvement. These findings could contribute to a reassessment of commonly held attitudes about the ostensible inability of underachieving students to engage in higher-order thinking tasks.

show abstract

Generation and Retrieval Practice Effects in the Classroom Using PeerWise

Cited by 18 publications

References 20 publications

Selecting student-authored questions for summative assessments

Selecting student-authored questions for summative assessments

Selecting Student-Authored Questions for Summative Assessments

Question Generation as a Strategy for Advancing Intermediate- and Low-Achieving Students in Higher Education

Contact Info

Product

Resources

About