A Corpus of Annotated Revisions for Studying Argumentative Writing

Abstract

This paper presents ArgRewrite, a corpus of between-draft revisions of argumentative essays. Drafts are manually aligned at the sentence level, and the writer's purpose for each revision is annotated with categories analogous to those used in argument mining and discourse analysis. The corpus should enable advanced research in writing comparison and revision analysis, as demonstrated via our own studies of student revision behavior and of automatic revision purpose prediction. 2 Corpus Design Decisions Consider this scenario: Alice begins her social science argumentative essay with the sentence "Electronic communication allows people to make connections beyond physical limits." An analytical system might (rightly) identify the sentence as the thesis of her essay, and an evaluative system might give the essay a low score due to this sentence's vagueness and a later lack of evidence (though Alice may not know why she received that score).

Diamonds in the Rough: Generating Fluent Sentences from Early-Stage Drafts for Academic Writing Assistance

“…Other corpora for revisions are available in the academic domain (Lee and Webster, 2012;Tan and Lee, 2014;Zhang et al, 2017). Thus, we provide a notable contribution by exploring the methods to create a dataset of revisions with a scalable crowdsourcing approach.…”

Identifying Editor Roles in Argumentative Writing from Student Revision Histories

“…Our work takes advantage of several corpora of multiple drafts of argumentative essays written by both high-school and college students [12,11], where all data has been annotated for revision using the framework of [12]. We divide our data into a Modeling Corpus (185 paired drafts, 3245 revisions) and an Evaluation Corpus (107 paired draft, 2045 revisions), based on whether expert grades are available before (Score1) and after (Score2) essay revision.…”

Annotation and Classification of Sentence-level Revision Improvement

“…The revisions that we annotated for improvement in quality are a subset of the freely available Ar-gRewrite revision corpus (Zhang et al, 2017) 1 . This corpus was created by extracting revisions from three drafts of argumentative essays written by 60 non-expert writers in response to a prompt 2 .…”

“…We first introduce a corpus of paired original and revised sentences that has been newly annotated as to whether each revision made the original sentence better or not. The revisions are a subset of those in the freely available Ar-gRewrite corpus (Zhang et al, 2017), with improvement annotated using standard rubric criteria for evaluating student argumentative writing. By adapting NLP features used in previous revision classification tasks, we then develop a prediction model that outperforms baselines, even though the size of our non-expert revision corpus is small.…”