The impact of computer-based feedback on students’ written work

Abstract

While research in second language writing suggests that instructor feedback can have a positive influence on students' written work, the provision of such feedback on a regular basis can be problematic, especially with larger student numbers. A number of computer programs that claim to provide both automatic computer-based holistic scores and computer-based feedback (CBF) on written work are available and therefore have the potential to deal with this issue. Criterion is one such tool that claims to be able to provide automated feedback at word, sentence, paragraph and text level, but there is still a need for more research into the practical value of providing feedback on L2 writing. Quantitative and qualitative data about feedback practice was collected from 31 instructors and 549 Egyptian trainee EFL teachers using pre-treatment questionnaires, interviews and focus groups. 24 of the trainees then received computer-based feedback using Criterion on two drafts of essays submitted on each of 4 topics. Data recorded by the software suggested a positive effect on the quality of students' second drafts and subsequent submissions, and post-treatment questionnaires, interviews and focus groups showed a positive effect on the students' attitudes towards feedback. KEYWORDS:computer-based feedback, Criterion, attitudes and motivation RESUMEN A pesar de que la investigación sobre escritura en segundas lenguas sugiere que los comentarios de los profesores pueden tener una influencia positiva sobre el trabajo escrito de los estudiantes, el proporcionar con regularidad tales comentarios puede ser problemático, especialmente en clases muy numerosas. Sin embargo, existen en el mercado una serie de programas informáticos que garantizan poder proporcionar tanto evaluaciones integrales de carácter automático como comentarios informatizados sobre trabajos escritos y que, por lo tanto, tienen cierto potencial para tratar este problema. Criterion es una de estas herramientas y, como tal, proporciona información automatizada a nivel de palabra, oración, párrafo y texto. En el presente trabajo analizamos el valor práctico que ofrece en la producción de comentarios para la escritura en L2 y, a este respecto, recogimos datos cuantitativos y cualitativos de 31 instructores y 616 profesores en formación de inglés como lengua extranjera de origen egipcio por medio de cuestionarios previos, entrevistas y discusiones en grupo. 24 de los profesores recibieron comentarios informatizados producidos por medio de Criterion sobre dos borradores de redacciones realizadas acerca de 4 temas diferentes. La información registrada en el software indica un efecto positivo sobre la calidad de los segundos borradores realizados por los estudiantes, así como de escritos posteriores. Asimismo, tanto los cuestionarios administrados después de la aplicación, como las entrevistas y las discusiones en grupo revelan un efecto positivo sobre la actitud de los estudiantes hacia los comentarios de los profesor

Automated Writing Analysis for Writing Pedagogy

“…Consequently, evidence of effectiveness began to be sought from users' experiences and outcomes. User-centric results show varied student/teacher perceptions, increased student motivation and learner autonomy, positive feedback effects, as well as enhanced understanding of errors and writing improvement (Attali, 2004;Chen & Cheng, 2008;Chung & Baker, 2003;El Ebyary & Windeatt, 2010;Elliot & Mikulas, 2004;Foltz, Laham, & Landauer, 1999;Li, Link, Ma, Yang, & Hegelheimer, 2014;Link, Dursun, Karakaya, & Hegelheimer, 2014;Rich, Schneider, & D'Brot, 2013;Schroeder, Grohe, & Pogue, 2008;Ware, 2011Ware, , 2014. Although the increasing number of studies sheds positive light on the potential of AWE, evidence accumulated to support its validity is relatively inconsistent and, hence, insufficiently informative for use in writing classrooms.…”

<b>Analysis of the scientific production on feedback on teaching English as a foreign language using ERIC database

“…The use of feedback mediated by electronic media has called the attention of numerous researchers in the past few years. A bulk of research studies have shown that computerdelivered feedback can be synchronous, asynchronous, self-paced, peer or instructor centered and it is suited to in or out of the classroom boundaries, both in distance learning, face-to-face education or blended learning (Seliem & Ahmed, 2009;Ebyary & Windeatt, 2010;Istifci, 2011;Hosseini, 2012;Vurdien, 2012;Lakarnchua & Wasanasomsithi, 2013;Arslan, 2014).…”

“…Nonetheless, e-feedback offered by their peers was perceived as artificial and not as useful. Ebyary and Windeatt (2010) investigated the impact of automatic computer-based feedback in higher education using a computer program named Criterion. The authors collected data from 31 instructors and 549 Egyptian trainee EFL teachers using pre-treatment questionnaires, interviews and focus groups.…”

The Effect of Using Automated Essay Evaluation on ESL Undergraduate Students’ Writing Skill

“…Based on the reviews of Warschauer & Ware (2006) as well as Ebyary & Windeatt (2010), Table 1 provides a detailed description of the most well-known AEE systems in terms of producing companies, software engines, targeted areas of writing assessment, statistical approaches to evaluation and types of scoring and feedback The AEE system chosen for the present study is e-rater®, the AEE platform manufactured by the Educational testing Service (ETS) that provides formative feedback along with an automated holistic score. This system's automated scoring application has been used for the TOEFL IBT as an independent rater for the purpose of scoring the writing independent writing task (see Enright & Quinlan, 2008 for the use of the e-rater® system in the TOEFL IBT independent writing task).…”

“…Students who received constant feedback demonstrated reduced errors in grammar, mechanics, usage and style in their final drafts. Ebyary & Windeatt (2010) applied questionnaires, interviews, and focus groups to investigate the effects of using the Criterion® systems with 549 Egyptian trainees and EFL instructors. The investigation focused on attitudes towards using the Criterion® system and whether there was a noticeable influence on the writing strategies used by students.…”