The impact of automated feedback on L2 learners’ written causal explanations

Abstract

Even though current technologies allow for automated feedback, evaluating content and generating discourse-specific feedback is still a challenge for automated systems, which explains the gap in research investigating the effect of such feedback. This study explores the impact of automated formative feedback on the improvement of English as a second language (ESL) learners’ written causal explanations within two cause-and-effect essays and across pre- and post-tests. Pre- and post-test drafts, feedback reports for first and revised drafts from the automated writing evaluation system, and screen-capturing videos collected from 31 students enrolled in two sections of an advanced-low-level academic writing class were analyzed through descriptive statistics and the Wilcoxon signed-rank test. Findings revealed statistically significant changes in learners’ causal explanations within one cause-and-effect essay while no significant improvement was observed across pre- and post-tests. The findings of this study offer not only insights into how to further improve automated discourse-specific feedback but also pedagogical implications for better learning outcomes.

Voluntary use of automated writing evaluation by content course students

“…from their first submitted draft of the first assignment to their first submitted draft of the last assignment) were not investigated in this study as we only examined the first two assignments from both groups and not the same students within one group used Criterion for the two assignments. Given that previous research has yielded limited or no benefits of AWE feedback in the long term (Lavolette et al, 2015;Saricaoglu, 2019), we would not expect long-term benefits in voluntary use of AWE either. However, with an interactionist perspective, we consider short-term changes as evidence of learning (Mackey & Polio, 2009;Norris & Ortega, 2003;Polio, 2012).…”

“…When AWE is used during the class or when a teacher asks students to use it (e.g. Lavolette et al, 2015;Li et al, 2017;Saricaoglu, 2019;Wang et al, 2013), almost all students use the AWE tool to meet the requirement. Explicit integration of AWE technology in the learning process in such implementations may better promote its use by students, as Koh (2016) argues.…”

“…Most existing AWE studies were conducted in English or writing classes where the use of an AWE tool was a requirement for a certain assignment and was led by the instructor (e.g. Chen & Cheng, 2008;Lavolette, Polio & Kahng, 2015;Saricaoglu, 2019). Today, more AWE tools are offered to students as self-led supplementary aids, but the potential of AWE for such use is still unknown, which is the concern of the current study.…”

Review of Studies on Technology-Enhanced Language Learning and Teaching

“…Because such feedback does not account for individual differences because its error types are not determined by pedagogical considerations but technological capacities [44]. Moreover, the evaluation and generation of discourse-related feedback is one big challenge for systems [45]. Although perfect accuracy is expected, it is difficult to achieve.…”

The Effects of Online Feedback on ESL/EFL Writing: A Meta-Analysis

“…By providing instant feedback, OAF is believed to be effective in improving students' writing (Kellogg et al, 2010). Students who receive more automated feedback are likely to interact with the tool more frequently, make more writing revisions, and build bridges between their prior and desired knowledge successfully (Link et al, 2020;Morch et al, 2017;Saricaoglu, 2019). There is also evidence showing an increase in writing scores as students make diligent revisions based on automatic feedback on their essays (Zhu et al, 2020).…”