Classifying easy-to-read texts without parsing

Falkenjack, Johan; Jönsson, Arne

doi:10.3115/v1/w14-1213

Cited by 8 publications

(15 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This can follow from the fact that these features can be considered as proxies of the syntactic structure which in these experiments was represented through specific features: in this situation, the grafting process preferred syntactic features over morpho-syntactic ones, in spite of the lower accuracy of the dependency parser with respect to the part-of-speech tagger. Interestingly, this result is in contrast with what reported by Falkenjack and Jönsson (2014) for what concerns document readability assessment, who claim that an optimal subset of text features for readability based document classification does not need features induced via parsing. Among the morphosyntactic features, it appears that verbal features play an important role: this can follow both by the language dealt with which is a a morphologically rich language, and by the fact that these features do not have a counterpart at the syntactic level.…”

Section: Sentence Vs Document Classificationcontrasting

confidence: 55%

Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications

Dell’Orletta⋄

Wieling

Cimino

et al. 2014

View full text Add to dashboard Cite

ii IntroductionThe field of NLP and education has matured dramatically since the first workshop in 1997, where the primary focus was on grammatical error detection and correction. As a community we have continued to improve existing capabilities and to identify and generate innovative and creative methods. Automated writing evaluation systems are now commercially viable, and are used to score millions of test-taker essays on high-stakes assessments. The educational and assessment landscape, especially in the United States, continues to foster a strong interest and high demand that furthers the state-of-the-art in automated writing evaluation capabilities, expanding the analysis of written responses to writing genres beyond those typicallyfound on standardized assessments. Much of the current demand for creative new educational applications results from the development of the Common Core State Standards Initiative (CCSSI). The goal of CCSSI is to ensure college-and workplace-readiness. The CCSSI describes what K-12 students should be learning with regard to reading, writing, speaking, listening, and media and technology.Major advances in speech technology have made it possible to include speech in both assessment and Intelligent Tutoring Systems (ITS). These advances have made it possible for spoken constructed responses are now being evaluated. Consistent with this, there is also a renewed interest in spoken dialog for instruction and assessment. Relative to continued innovation, the explosive growth of mobile applications has increased interest in game-based assessment.In the past few years, the use of NLP in educational applications gained visibility outside of the Computational Linguistics (CL) community. First, the Hewlett Foundation reached out to public and private sectors by sponsoring two competitions (both inspired by the CCSSI): one for automated essay scoring, and one for scoring of short response items. The motivation driving these competitions was to engage the larger scientific community in this enterprise. Massive Open Online Courses (MOOCs) are now also beginning to incorporate automated writing scoring systems to manage the thousands of writing assignments that can be generated in a single MOOC course. Another breakthrough for educational applications within the CL community is the large number of shared task competitions in the last few years. There have been four shared tasks on grammatical error correction, with the most recent edition hosted at CoNLL 2014. In 2013, there was a SemEval Shared Task on Student Response Analysis and one on Native Language Identification (hosted at the 2013 edition of this workshop). All of these competitions increased the visibility of the research space for using NLP to build educational applications.As a community, we continue to improve existing capabilities and to identify and generate innovative ways to use NLP in applications for writing, reading, speaking, critical thinking, curriculum development, and assessment. Steady growth in the development of NLP-b...

show abstract

Section: Sentence Vs Document Classificationcontrasting

confidence: 55%

Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications

Dell’Orletta⋄

Wieling

Cimino

et al. 2014

View full text Add to dashboard Cite

show abstract

“…They show that a small number of features can attain a high accuracy. (Falkenjack and Jönsson, 2014) However their work is carried out at document level and is not consistent with the results of Dell'Orletta et al at sentence level.…”

Section: Related Workmentioning

confidence: 67%

Machine learning for readability of legislative sentences

Curtotti

McCreath

Bruce

et al. 2015

Proceedings of the 15th International Conference on Artificial Intelligence and Law

View full text Add to dashboard Cite

Improving the readability of legislation is an important and unresolved problem. Recently, researchers have begun to apply legal informatics to this problem. This paper applies machine learning to predict the readability of sentences from legislation and regulations. A corpus of sentences from the United States Code and US Code of Federal Regulations was created. Each sentence was labelled for language difficulty using results from a large-scale crowdsourced study undertaken during 2014. The corpus was used as training and test data for machine learning. The corpus includes a version tagged using the Stanford parser context free grammar and a version tagged using the Stanford dependency grammar parser. The corpus is described and made available to interested researchers. We investigated whether extending natural language features available as input to machine learning improves the accuracy of prediction. Among features evaluated are those from the context free and dependency grammars. Letter and word ngrams were also studied. We found the addition of such features improves accuracy of prediction on legal language. We also undertake a correlation study of natural language features and language difficulty drawing insights as to the characteristics that may make legal language more difficult. These insights, and those from machine learning, enable us to describe a system for reducing legal language difficulty and to identify a number of suggested heuristics for improving the writing of legislation and regulations.

show abstract

“…Contrasting these results quickly to previous research on the classification performance of linguistic features in the context of Support Vector Machines (Falkenjack and Jönsson 2014; Falkenjack, Mühlenbock, and Jönsson 2013) we can see that our results are quite different. Falkenjack and Jönsson (2014) found that the ratio of relative/interrogative pronouns performed barely better than chance on the task of classifying mixed-genre easy-to-read texts. The ratio of SweVoc words and the ratio of rightward dependencies were clearly better than chance but were not among the strongest predictors.…”

Section: Posterior For βmentioning

confidence: 94%

Modeling Text Complexity using a Multi-Scale Probit

Falkenjack,

Villani,

Jönsson

2018

Preprint

Self Cite

View full text Add to dashboard Cite

We present a novel model for text complexity analysis which can be fitted to ordered categorical data measured on multiple scales, e.g. a corpus with binary responses mixed with a corpus with more than two ordered outcomes. The multiple scales are assumed to be driven by the same underlying latent variable describing the complexity of the text. We propose an easily implemented Gibbs sampler to sample from the posterior distribution by a direct extension of established data augmentation schemes. By being able to combine multiple corpora with different annotation schemes we can get around the common problem of having more text features than annotated documents, i.e. an example of the p > n problem. The predictive performance of the model is evaluated using both simulated and real world readability data with very promising results.

show abstract

Classifying easy-to-read texts without parsing

Cited by 8 publications

References 7 publications

Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications

Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications

Machine learning for readability of legislative sentences

Modeling Text Complexity using a Multi-Scale Probit

Contact Info

Product

Resources

About