Effects of computer-based feedback on lower- and higher-order learning outcomes: A network meta-analysis.

Abstract

Feedback is one of the most important factors for successful learning. Contemporary computer-based learning and testing environments allow the implementation of automated feedback in a simple and efficient manner. Previous meta-analyses suggest that different types of feedback are not equally effective. This heterogeneity might depend on learner and test characteristics as well as the assessed outcome measure. Here, we present a novel network meta-analysis approach that allowed us to compare (i.e., rank) different types of feedback regarding their effects on (lower- and higher-order) performance measures. Following an extensive literature search, we were able to use 163 effect sizes from 77 experimental studies to compare classical feedback variations such as Knowledge of Results (KR), Knowledge of Correct Response (KCR), Elaborated Feedback (EF), and Answer-Until-Correct (AUC) feedback, with each other and with a No Feedback (NoFB) control group. Our findings indicate that EF is most likely to be the most effective for lower-order (i.e., recall/recognition) and higher-order (i.e., transfer) learning outcomes compared with the other feedback variants. For KCR and AUC, we typically found small to large effect sizes on learning outcomes. KR was found to be less effective than the other feedback types on improving lower-order and higher-order learning outcomes. Several subgroup analyses are reported to identify moderating factors for the effectiveness of different feedback interventions for different learner characteristics (i.e., sample source, and prior knowledge level) and test characteristics (i.e., learning domain, and test format).

Timing's not everything: Immediate and delayed feedback are equally beneficial for performance in formative multiple‐choice testing

“…We recently reported that more elaborate forms of feedback (i.e. response‐oriented and conceptually‐focused feedback) provided a greater benefit to second‐year medical students than binary right/wrong feedback, 32 a finding congruent with the broader literature 37 . Building upon these findings, the current study aims to determine whether the timing of feedback has an effect on its efficacy as a tool for learning in medical education.…”

“…response-oriented and conceptually-focused feedback) provided a greater benefit to second-year medical students than binary right/wrong feedback, 32 a finding congruent with the broader literature. 37 Building upon these findings, the current study aims to determine whether the timing of feedback has an effect on its efficacy as a tool for learning in medical education. In general, spacing learning opportunities over time to form distributed practice (as opposed to 'massed practice') appears to benefit clinical learning.…”

Automated feedback and writing: a multi-level meta-analysis of effects on students' performance

“…Thus, in future research, additional moderators need to be investigated. In other learning contexts, the type of feedback has been shown to moderate effectiveness (Van der Kleij et al, 2015 ; Wisniewski et al, 2020 ; Mertens et al, 2022 ). In the context of AWE feedback, we need more primary studies that compare different types of feedback or at least provide sufficient information on the details of their feedback intervention.…”

Process Data in Computer-Based Assessment

“…Furthermore, in education, we will find that collecting process data in the context of formative assessments for learning, such as incorporated in computer-based systems (intelligent tutoring systems; see, e.g., Aleven et al, 2010), will become increasingly common in everyday educational settings. Formative assessment results and fine-grained process data can provide valuable additional information for teachers and learners in terms of learning analytics and can, for example, establish the base for the provision of targeted, individualized automated feedback (e.g., Kuklick & Lindner, 2021, 2023; Kuklick et al, 2023; Mertens et al, 2022) and other types of status reports of students’ individual learning progression (e.g., directed at students, parents, teachers, or stakeholders). Thus, in the future, we may expect to see a closer relationship and an overlap between process data research in the assessment context and research in classroom learning settings.…”