THE IMPACT OF SHORT‐TERM USE OF CRITERION^SM ON WRITING SKILLS IN NINTH GRADE

Abstract

In this study, a hierarchical linear model (HLM) was used within a cluster random assignment design to test the impact of Criterion sm on students' writing skill when Criterion was used as a supplementary instructional tool in ninth grade English classrooms over a 4 week period. The study was conducted in three states: Indiana, Pennsylvania, and New Jersey. The sample consisted of 1,464 ninth graders aggregated across 10 schools, 34 teachers, and 61 classes.Findings indicate that over a short time period, Criterion use improved the mechanical aspects of student essays as measured by the analytic scores given to their essays. Students in ninth gradeEnglish classes who used Criterion several times a week over a 1 month period received higher analytic scores on their essays written at the end of the study period than those who were in classes that did not. The effect was small, but statistically significant at the .05 level using a onetailed test. These findings suggest a role for automated feedback systems in supporting writing instruction.

A Comprehensive Meta-analysis of Handwriting Instruction

“…Adjusting standard errors for these investigations was necessary, as a portion of the total variance in such quasiexperiments was likely due to grouping or clustering within conditions, with the total variance representing a sum of group and student variances. Using procedures recommended by Hedges (2007), these studies were adjusted for clustering effects with imputed intraclass correlation (ICC) estimates for reading comprehension from national studies (Hedges and Hedberg 2007) that were adjusted to writing quality ICCs, with data from a large study of writing that involved a single grade level (Rock 2007). While we would have preferred using ICCs based on writing data at each grade level, that information is not yet available.…”

Potential of Automated Writing Evaluation Feedback

“…In Leah Rock's (2007) study, 9th-grade students used Criterion for 4 weeks and, during this short period of time, received higher analytic scores on their essays written at the end of the study period and improved the mechanical aspects of their writing.…”

Preface: Special Issue on Multidisciplinary Approaches to AI and Education for Reading and Writing

“…Automated Essay Scoring (AES) systems constitute a relatively well-developed commercial technology for evaluation of students' written essays (Burstein et al 1998;Burstein 2003;Edelblut and Change 2004;Foltz et al 2013;Landauer et al 2003;Plakans and Gebril 2013;Rock 2007;Rudner et al 2006). They typically assign scores to essays on a 3-point or 4-point scale, and agree with human scorers as well as human scorers agree with each other.…”