The E-rater® Automated Essay Scoring System

Burstein, Jill; Tetreault, Joel; Madnani, Nitin

doi:10.4324/9780203122761.ch4

Cited by 79 publications

(84 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 Burstein et al (2013) provide an excellent depiction of the state of-the-art of automated scoring and evaluation of essays (including detailed descriptions of scoring systems built using supervised statistical modeling, unsupervised modeling, rule-based techniques) as well as research on their efficacy. Swales,and J. Yates.…”

Section: Notesmentioning

confidence: 99%

Automated Writing Analysis for Writing Pedagogy

Cotos

2015

Writing & Pedagogy

View full text Add to dashboard Cite

This article aims to engage specialists in writing pedagogy, assessment, genre study, and educational technologies in a constructive dialog and joint exploration of automated writing analysis as a potent instantiation of computer-enhanced assessment for learning. It recounts the values of writing pedagogy and, from this perspective, examines legitimate concerns with automated writing analysis. Emphasis is placed on the need to substantiate the construct-driven debate with systematic empirical evidence that would corroborate or refute interpretations, uses, and consequences of automated scoring and feedback tools intended for specific contexts. Such evidence can be obtained by adopting a validity argument framework. To demonstrate an application of this framework, the article presents a novel genre-based approach to automated analysis configured to support research writing and provides examples of validity evidence for using it with novice scholarly writers. Automated Writing Analysis for writing pedagogy: From healthy tension to tangible prospects Elena Cotos AbstractThis article aims to engage specialists in writing pedagogy, assessment, genre study, and educational technologies in a constructive dialog and joint exploration of automated writing analysis as a potent instantiation of computer-enhanced assessment for learning. It recounts the values of writing pedagogy and, from this perspective, examines legitimate concerns with automated writing analysis. Emphasis is placed on the need to substantiate the construct-driven debate with systematic empirical evidence that would corroborate or refute interpretations, uses, and consequences of automated scoring and feedback tools intended for specific contexts. Such evidence can be obtained by adopting a validity argument framework. To demonstrate an application of this framework, the article presents a novel genre-based approach to automated analysis configured to support research writing and provides examples of validity evidence for using it with novice scholarly writers.

show abstract

Section: Notesmentioning

confidence: 99%

Automated Writing Analysis for Writing Pedagogy

Cotos

2015

Writing & Pedagogy

View full text Add to dashboard Cite

show abstract

“…Answering this question would tell us whether we capture an important construct of the argument analysis task by recognizing these argumentation features. Specifically, we tested whether these features add predictive value to a model based the state-of-the-art e-rater essay scoring system (Burstein, Tetreault, and Madnani, 2013).…”

Section: Essay Score and Annotation Featuresmentioning

confidence: 99%

Applying Argumentation Schemes for Essay Scoring

Yi¹,

Heilman²,

Klebanov³

et al. 2014

Proceedings of the First Workshop on Argumentation Mining

View full text Add to dashboard Cite

Under the framework of the argumentation scheme theory (Walton, 1996), we developed annotation protocols for an argumentative writing task to support identification and classification of the arguments being made in essays. Each annotation protocol defined argumentation schemes (i.e., reasoning patterns) in a given writing prompt and listed questions to help evaluate an argument based on these schemes, to make the argument structure in a text explicit and classifiable. We report findings based on an annotation of 600 essays. Most annotation categories were applied reliably by human annotators, and some categories significantly contributed to essay score. An NLP system to identify sentences containing scheme-relevant critical questions was developed based on the human annotations.

show abstract

“…statistical modeling, and natural language processing (Shermis & Burstein, 2013). For example, systems such as e-rater developed at Educational Testing Service (Burstein, Chodorow, & Leacock, 2004;Burstein, Tetreault, & Madnani, 2013) and the IntelliMetric Essay Scoring System developed by Vantage Learning (Rudner et al, 2006;Schultz, 2013) rely primarily on combinations of natural language processing techniques and artificial intelligence, whereas the Intelligent Essay Assessor (Foltz, Streeter, Lochbaum, & Landauer, 2013;Landauer et al, 2003) primarily relies on Latent Semantic Analysis. Across AES systems, a typical methodology is followed.…”

Section: Sat Instructionsmentioning

confidence: 99%