Investigating the promise of automated writing evaluation for supporting formative writing assessment at scale

Wilson, Joshua; Myers, Matthew C.; Potter, Andrew

doi:10.1080/0969594x.2022.2025762

Cited by 13 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In spite of the mentioned positive opinions, some learners expressed that using the AWE was not adequate for them and they required their instructor's assistance, as well. In line with our findings, Wilson et al (2022) offered the application of computer software to serve the objective of the independent enhancement of writing abilities, especially for EFL authors. According to Williams (2005), if the application of a computer program is attentively modeled, it can suggest to learners both help and independency in the writing process.…”

Section: Discussionsupporting

confidence: 85%

“…In a very recent study, Wilson et al (2022) examined a novel approach to formative writing assessment encompassing an AWE system called MI Write. More precisely, they discovered elementary teachers' perceptions and implementation of MI Write and changes in students' writing performance in three genres.…”

Section: Empirical Backgroundmentioning

confidence: 99%

“…Instructors have utilized different electronic writing media such as word processors, e-mail exchanges and bulletin boards in their training. Recent progress in Automated Writing Evaluation schedules (AWE) has attracted many instructors to execute this novel technology for scoring and measuring learners' writing (Li et al, 2015;Wilson et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Potential Impact of Automated Writing Evaluation (AWE) on EFL Learners’ Writing Accuracy: Learners’ Attitudes in Focus

Asfa

Rezvani²

2023

Innovare J Edu

View full text Add to dashboard Cite

The present study aimed to examine the potential impacts of Automated Writing Evaluation (AWE) on Iranian English as Foreign Language (EFL) learners’ writing accuracy. Another study objective was to identify the learners’ attitudes toward using AWE in English classes. As such, 50 female EFL students at the intermediate level of language proficiency were chosen via convenience sampling to participate in the present study. They were then randomly divided into two groups of Control Group (CG) and the Experimental Group (EG), each consisting of 25 learners. Before the treatment, a researcher-made writing composition pre-test was administered to both groups. Next, the learners in the EG received the treatment in which an AWE system was employed to check the learner’s written productions. Next, learners in the CG were exposed to the conventional pen-and-paper methods of English writing classes. Then, a writing composition post-test was administered. The performances of learners on pre and post-tests were scored based on accuracy in terms of syntactic errors. After administering the post-test, EG learners were asked to fill in a questionnaire on AWE. Moreover, they were asked two questions in a semi-structured interview on their attitudes toward applying to the AWE program. The researcher recorded and later transcribed the learners’ answers for further analysis. The independent sample t-test revealed that Iranian EFL learners’ writing accuracy significantly improved after receiving AWE. Furthermore, the Chi-square test results, in conjunction with the interview results, showed that learners had positive attitudes toward AWE in writing classes. The study will have implications for the second language (L2) writing practices of EFL teachers and students.

show abstract

Section: Discussionsupporting

confidence: 85%

Section: Empirical Backgroundmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Potential Impact of Automated Writing Evaluation (AWE) on EFL Learners’ Writing Accuracy: Learners’ Attitudes in Focus

Asfa

Rezvani²

2023

Innovare J Edu

View full text Add to dashboard Cite

show abstract

“…They were used, for example, to identify at-risk students, authenticate remote students, or prevent cheating (Kaddoura et al, 2022). In England, however, the use (Huang & Wilson, 2021;Wilson et al, 2022;Wilson & Roscoe, 2020) and may improve writing skills (Nunes et al, 2022). The more AI becomes acceptable in low-stakes settings, the higher the chances it will be adopted in high-stakes assessment.…”

Section: Aloisimentioning

confidence: 99%

“…For instance, automated marking of open‐ended responses to provide immediate formative feedback, used in US elementary schools to reduce teacher workload, is increasingly being offered to English schools by commercial companies. Automated writing evaluation systems are positively viewed by teachers (Huang & Wilson, 2021; Wilson et al, 2022; Wilson & Roscoe, 2020) and may improve writing skills (Nunes et al, 2022). The more AI becomes acceptable in low‐stakes settings, the higher the chances it will be adopted in high‐stakes assessment.…”

Section: Barriers For Using Artificial Intelligence and Machine Learn...mentioning

confidence: 99%

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

Aloisi¹

2023

Euro J of Education

View full text Add to dashboard Cite

This article considers the challenges of using artificial intelligence (AI) and machine learning (ML) to assist high‐stakes standardised assessment. It focuses on the detrimental effect that even state‐of‐the‐art AI and ML systems could have on the validity of national exams of secondary education, and how lower validity would negatively affect trust in the system. To reach this conclusion, three unresolved issues in AI (unreliability, low explainability and bias) are addressed, to show how each of them would compromise the interpretations and uses of exam results (i.e., exam validity). Furthermore, the article relates validity to trust, and specifically to the ABI+ model of trust. Evidence gathered as part of exam validation supports each of the four trust‐enabling components of the ABI+ model (ability, benevolence, integrity and predictability). It is argued, therefore, that the three AI barriers to exam validity limit the extent to which an AI‐assisted exam system could be trusted. The article suggests that addressing the issues of AI unreliability, low explainability and bias should be sufficient to put AI‐assisted exams on par with traditional ones, but might not go as far as fully reassure the public. To achieve this, it is argued that changes to the quality assurance mechanisms of the exam system will be required. This may involve, for example, integrating principled AI frameworks in assessment policy and regulation.

show abstract

Improving Writing Feedback for Struggling Writers: Generative AI to the Rescue?

Evmenova,

Regan,

Mergen

et al. 2024

TechTrends

View full text Add to dashboard Cite

Generative AI has the potential to support teachers with writing instruction and feedback. The purpose of this study was to explore and compare feedback and data-based instructional suggestions from teachers and those generated by different AI tools. Essays from students with and without disabilities who struggled with writing and needed a technology-based writing intervention were analyzed. The essays were imported into two versions of ChatGPT using four different prompts, whereby eight sets of responses were generated. Inductive thematic analysis was used to explore the data sets. Findings indicated: (a) differences in responses between ChatGPT versions and prompts, (b) AI feedback on student writing did not reflect provided student characteristics (e.g., grade level or needs; disability; ELL status), and (c) ChatGPT’s responses to the essays aligned with teachers’ identified areas of needs and instructional decisions to some degree. Suggestions for increasing educator engagement with AI to enhance teaching writing is discussed.

show abstract

Investigating the promise of automated writing evaluation for supporting formative writing assessment at scale

Cited by 13 publications

References 26 publications

Potential Impact of Automated Writing Evaluation (AWE) on EFL Learners’ Writing Accuracy: Learners’ Attitudes in Focus

Potential Impact of Automated Writing Evaluation (AWE) on EFL Learners’ Writing Accuracy: Learners’ Attitudes in Focus

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

Improving Writing Feedback for Struggling Writers: Generative AI to the Rescue?

Contact Info

Product

Resources

About