Implementing an automated feedback program for a foreign language writing course: A learner‐centric study

Abstract

Considering the prominence attached to written corrective feedback (WCF) within the domain of second/foreign language (L2) acquisition, automated writing evaluation (AWE) tools have steadily gained ground over the last two decades. The current study was an attempt to investigate the extent incorporating an AWE program, known as Criterion®, within a process writing framework would affect learners' writing quality in an English as a foreign language context. Moreover, we drew a comparison between the overall effects of computer‐ versus teacher‐generated WCF. Participants consisted of 53 tertiary level students from two intact essay‐writing classes. The within‐group findings revealed that the experimental group's writing scores rose significantly from the pretest to posttest after receiving automated WCF over several weeks. Significant progress was also noticed on all rounds of scores obtained from five essays assigned to this group except between the first and the second. As for the comparison between computer‐ and teacher‐administered WCF, the former condition was found to be more efficient. Overall, the findings provide a deeper insight into the workings of an automated evaluation tool and how it was employed in the L2 classroom to impart consistency to the assessment procedure and ease the burden traditionally placed upon teachers.

The Effectiveness of Automated Writing Evaluation on Writing Quality: A Meta-Analysis

“…A considerable number of empirical studies have investigated AWE’s effectiveness in improving writing but produced mixed findings. Most showed positive results (e.g., Cheng, 2017; Chew et al, 2019; Hassanzadeh & Fotoohnejad, 2021), while some reported null or insignificant effects (e.g., Wilson & Czik, 2016; Wilson & Roscoe, 2020). For instance, Cheng (2017) investigated the impact of online automated feedback (OAF) on the quality of EFL students’ reflective journals at the university level and found that students receiving OAF outperformed their counterparts under traditional instruction in terms of the overall score and demonstrated a significant improvement in writing quality.…”

“…The results revealed that the SW-PAL group demonstrated a greater improvement in summary writing quality compared with their conventional counterparts. Hassanzadeh and Fotoohnejad (2021) examined the efficacy of incorporating Criterion in an EFL classroom by comparing the effects of AWE and teacher-generated feedback. Their results showed that AWE feedback was more efficient in improving the overall quality of writing than feedback from teachers over time.…”

Still a fallible tool? Revisiting effects of automated writing evaluation from activity theory perspective

“…According to Lipsey and Wilson (2001, p. 108), the potential outliers that were ‘more than three standard deviations from the mean of all effect sizes’ should be excluded. As such, one selected research ( g = 3.607, Hassanzadeh & Fotoohnejad, 2021) out of the 26 eligible studies that yielded extremely large effect size was excluded, resulting in 25 remaining primary studies that yielded 33 effect sizes for the final analysis.…”

Partnering with Automatic Writing Evaluation

“…However, several experiments to utilize AWE reported mixed results. For example, in an Iranian context, AWE integration resulted in significant student writing progress (Hassanzadeh & Fotoohnejad, 2021). In another study, students' grammatical knowledge increased while their writing apprehension decreased (Waer, 2021).…”