Creating an automated tool to assist with event-conflation studies: An explanation and argument for its importance

Spring, Ryan; Ono, Naoyuki

doi:10.1016/j.rmal.2023.100054

Cited by 1 publication

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Five professional raters were asked to rate the writing from the final exams after the semester had ended, and the same basic procedures as above were taken to create a new model. It should be noted that after the first iteration, I became aware of Kyle's tools, and several measures from the TAASC program were considered for later iterations of the AI-rating model, as well as a separate phrasal complexity measure (i.e., the number of satellite-framed expressions) based on an early version of the Event Conflation Finder (Spring & Ono, 2023). After each iteration, the initial data set, as well as the writing samples from all exams up to that point were both considered, and only variables that showed steady correlation across all data sets were considered.…”

Section: Figure 1 Decision Tree For Ai Model Ratingmentioning

confidence: 99%

A Human-AI Integrated Rating System for Improving Second Language Writing:The Case of Japanese Learners of English for General Academic Purposes

Spring

2023

Preprint

Self Cite

View full text Add to dashboard Cite

In order to solve the problem of teachers not assigning and evaluating student writing but not completely trusting AI raters, I created and tested a rating scheme in which an AI model would rate students’ language use based on understandable criteria and humans would quickly check the AI responses while rating content and structure. Teachers tried the scheme and improvements were made based on new data and newly available research. An online practice tool was also created for students so that they could understand how the AI would rate their language use and practice accordingly. The AI rating models were improved over the course of three semesters based on student data and the ratings of external professional raters. As a result, an increasing number of teachers used the rating scheme, the number of students that practiced writing and were evaluated increased university-wide, and reasonable levels of fairness assessment were maintained.

show abstract

Section: Figure 1 Decision Tree For Ai Model Ratingmentioning

confidence: 99%