Identifying Patterns For Short Answer Scoring Using Graph-based Lexico-Semantic Text Matching

Ramachandran, Lakshmi; Cheng, Jian; Foltz, Peter W.

doi:10.3115/v1/w15-0612

Cited by 59 publications

(43 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our research work is closely related to the work of [12,13]. In Tandalla's approach, multi-features, including RE from text, are extracted and trained on RF and GBM.…”

Section: Related Work and Literature Reviewmentioning

confidence: 99%

“…The recent approaches of features extractions adopted by the research for essay scoring are regular expressions and semantic evaluation e.g., Tandalla and Rodrigues use the regular expressions as features, as described in Section 3.3.2, and Ramachandran's approach [13] is extracting text patterns containing content tokens and text patterns containing sentence-structure information with the attributes of semantics.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Prognosis Essay Scoring and Article Relevancy Using Multi-Text Features and Machine Learning

Mehmood

Lee

et al. 2017

Symmetry

View full text Add to dashboard Cite

Abstract:This study develops a model for essay scoring and article relevancy. Essay scoring is a costly process when we consider the time spent by an evaluator. It may lead to inequalities of the effort by various evaluators to apply the same evaluation criteria. Bibliometric research uses the evaluation criteria to find relevancy of articles instead. Researchers mostly face relevancy issues while searching articles. Therefore, they classify the articles manually. However, manual classification is burdensome due to time needed for evaluation. The proposed model performs automatic essay evaluation using multi-text features and ensemble machine learning. The proposed method is implemented in two data sets: a Kaggle short answer data set for essay scoring that includes four ranges of disciplines (Science, Biology, English, and English language Arts), and a bibliometric data set having IoT (Internet of Things) and non-IoT classes. The efficacy of the model is measured against the Tandalla and AutoP approach using Cohen's kappa. The model achieves kappa values of 0.80 and 0.83 for the first and second data sets, respectively. Kappa values show that the proposed model has better performance than those of earlier approaches.

show abstract

“…Our research work is closely related to the work of [12,13]. In Tandalla's approach, multi-features, including RE from text, are extracted and trained on RF and GBM.…”

Section: Related Work and Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Prognosis Essay Scoring and Article Relevancy Using Multi-Text Features and Machine Learning

Mehmood

Lee

et al. 2017

Symmetry

View full text Add to dashboard Cite

show abstract

“…We choose learners that (a) have either been shown to perform well with feature sets comparable to ours in previously published work - Mohler et al (2011), Sakaguchi et al (2015, and Zesch et al (2015) all used support vector machines; Ramachandran et al (2015) use a random forest regressor -or (b) are generally known to perform well with a large number of sparse features (Hastie et al, 2001;Fan et al, 2008;Chang and Lin, 2011). We use the scikit-learn (Pedregosa et al, 2011) implementations for all learners.…”

Section: Learnersmentioning

confidence: 99%

“…(b) response-based which use a large number of detailed features extracted from the student responses themselves (e.g., word ngrams, etc.) and human scores assigned to the responses to learn a supervised machine-learning model (Mohler et al, 2011;Dzikovska et al, 2013;Ramachandran et al, 2015;Zesch et al, 2015;Zhu et al, 2016). Response-based approaches generally require training a separate model for each question.…”

Section: Related Workmentioning

confidence: 99%

A Large Scale Quantitative Exploration of Modeling Strategies for Content Scoring

Madnani¹,

Loukina²,

Cahill³

2017

Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

We explore various supervised learning strategies for automated scoring of content knowledge for a large corpus of 130 different content-based questions spanning four subject areas (Science, Math, English Language Arts, and Social Studies) and containing over 230,000 responses scored by human raters. Based on our analyses, we provide specific recommendations for content scoring. These are based on patterns observed across multiple questions and assessments and are, therefore, likely to generalize to other scenarios and prove useful to the community as automated content scoring becomes more popular in schools and classrooms.

show abstract

“…In contrast, scoring for content deals with responses to openended questions designed to test what the student knows, has learned, or can do in a specific subject area (such as Computer Science, Math, or Biology) (Sukkarieh and Stoyanchev, 2009;Sukkarieh, 2011;Mohler et al, 2011;Dzikovska et al, 2013;Ramachandran et al, 2015;Sakaguchi et al, 2015;Zhu et al, 2016). 2 In order to measure the content of the spoken responses in our data, we extract the following set of features from the 1-best ASR hypotheses for each response:…”

Section: Text-driven Featuresmentioning

confidence: 99%

Proceedings of the Workshop on Speech-Centric Natural Language Processing

Ruiz¹,

Bangalore²

2017

View full text Add to dashboard Cite

The purpose of this workshop was to unite the automatic speech recognition (ASR) and natural language processing (NLP) communities to discuss new frameworks for exploiting the rich information present in the speech signal to improve the capabilities of natural language processing applications. Our community objective is to revisit the conventional NLP problems with a focus on incorporating the richness of spoken language, as well as to encourage research contributions that promote cross-fertilization between statistical methods for ASR and NLP.Our inaugural workshop was held at EMNLP to encourage participation amongst the NLP community to consider and discuss the challenges of combining speech recognition with conventional NLP research, as well as to appreciate the recent successes in this exciting field. The authors in these proceedings have combined ASR and NLP in works that address part-of-speech tagging, constituency parsing and dependency parsing on speech, information extraction and spoken term detection, dialog state tracking and speech translation, as well as two research assessments that evaluate the fluency and adequacy of English speakers and the role of speech silence in conversational dialogs.The invited talk was given by Gabriel Skantze, entitled "Modelling turn-taking in spoken interaction."Our workshop also contained an open round-table discussion about the current state of speech-centric NLP and some of the research and pragmatic issues that raise a barrier of entry for the larger research community.We would like to thank the members of the Program Committee for their reviews, as well as our panelists who led our round- AbstractSilence is an integral part of the most frequent turn-taking phenomena in spoken conversations. Silence is sized and placed within the conversation flow and it is coordinated by the speakers along with the other speech acts. The objective of this analytical study is twofold: to explore the functions of silence with duration of one second and above, towards information flow in a dyadic conversation utilizing the sequences of dialog acts present in the turns surrounding the silence itself; and to design a feature space useful for clustering the silences using a hierarchical concept formation algorithm. The resulting clusters are manually grouped into functional categories based on their similarities. It is observed that the silence plays an important role in response preparation, also can indicate speakers' hesitation or indecisiveness. It is also observed that sometimes long silences can be used deliberately to get a forced response from another speaker thus making silence a multi-functional and an important catalyst towards information flow.

show abstract

Identifying Patterns For Short Answer Scoring Using Graph-based Lexico-Semantic Text Matching

Cited by 59 publications

References 17 publications

Prognosis Essay Scoring and Article Relevancy Using Multi-Text Features and Machine Learning

Prognosis Essay Scoring and Article Relevancy Using Multi-Text Features and Machine Learning

A Large Scale Quantitative Exploration of Modeling Strategies for Content Scoring

Proceedings of the Workshop on Speech-Centric Natural Language Processing

Contact Info

Product

Resources

About