The Illinois-Columbia System in the CoNLL-2014 Shared Task

Rozovskaya, Alla; Chang, Kai-Wei; Sammons, Mark; Roth, Dan; Habash, Nizar

doi:10.3115/v1/w14-1704

Cited by 76 publications

(80 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition to calculating scores on a human vs human basis, we also calculated the F-scores for the top three CoNLL-2014 teams, AMU (JunczysDowmunt and Grundkiewicz, 2014), CAMB (Felice et al, 2014), and CUUI (Rozovskaya et al, 2014), versus all the combinations of humans (Equation 3).…”

Section: System Vs Humanmentioning

confidence: 99%

How Far are We from Fully Automatic High Quality Grammatical Error Correction?

Bryant¹,

Ng²

2015

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confere

View full text Add to dashboard Cite

In this paper, we first explore the role of inter-annotator agreement statistics in grammatical error correction and conclude that they are less informative in fields where there may be more than one correct answer. We next created a dataset of 50 student essays, each corrected by 10 different annotators for all error types, and investigated how both human and GEC system scores vary when different combinations of these annotations are used as the gold standard. Upon learning that even humans are unable to score higher than 75% F 0.5 , we propose a new metric based on the ratio between human and system performance. We also use this method to investigate the extent to which annotators agree on certain error categories, and find that similar results can be obtained from a smaller subset of just 10 essays.

show abstract

Section: System Vs Humanmentioning

confidence: 99%

How Far are We from Fully Automatic High Quality Grammatical Error Correction?

Bryant¹,

Ng²

2015

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confere

View full text Add to dashboard Cite

show abstract

“…For instance, the highest performance in the CoNLL-2013 shared task that also used the same evaluation metric was 31.20 (Rozovskaya et al, 2013). 4 The highest score in the HOO-2011 shared task (Dale and Kilgarriff, 2011) that addressed all er-rors was 21.1 (Rozovskaya et al, 2011).…”

Section: Resultsmentioning

confidence: 99%

The First QALB Shared Task on Automatic Text Correction for Arabic

Mohit

Rozovskaya

Habash

et al. 2014

Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)

Self Cite

View full text Add to dashboard Cite

We present a summary of the first shared task on automatic text correction for Arabic text. The shared task received 18 systems submissions from nine teams in six countries and represented a diversity of approaches. Our report includes an overview of the QALB corpus which was the source of the datasets used for training and evaluation, an overview of participating systems, results of the competition and an analysis of the results and systems.

show abstract

“…Both rule-based and data-driven approaches to error correction can be found in the literature (Sidorov et al, 2013;Berend et al, 2013;Yi et al, 2013) as well as hybridization of them (Putra and Szabo, 2013 the proposed systems build distinct models to address individual types of errors (see the CoNLL-2013proceedings (Ng et al, 2013aNg et al, 2014), and combine them afterwords using Integer Linear Programming for instance (Rozovskaya et al, 2013). This approach is relatively time-consuming when the number of error types increases.…”

Section: Related Workmentioning

confidence: 99%