Context-based Automated Scoring of Complex Mathematical Responses

Cahill, Aoife; Fife, James; Riordan, Brian; Vajpayee, Avijit; Galochkin, Dmytro

doi:10.18653/v1/2020.bea-1.19

Cited by 4 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the assessment of content and context in student texts frequently uses features such as word n-grams (i.e. chunks of n adjacent word tokens) (Riordan et al, 2020;Cahill et al, 2020) and Part-of-speech (POS) n-grams (Phandi et al, 2015;Kumar et al, 2020). Where assessment takes into account the task prompt to ensure that the student's text is relevant to the prompt, word overlap between the student text and the prompt has been used as a feature set (Phandi et al, 2015;Nguyen & Litman, 2018;Kumar et al, 2020).…”

Section: Feature Sets and Modelsmentioning

confidence: 99%

“…We refer to this task as student free-text evaluation. Depending on the task, students' texts can range from short answers to question prompts that consist of few phrases (Maharjan & Rus, 2019) to short answers consisting of multiple sentences or a paragraph (Cahill et al, 2020), and finally to fully-fledged essays (Dong et al, 2017;Gong et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

Bai

Stede

2022

Int J Artif Intell Educ

View full text Add to dashboard Cite

Recent years have seen increased interests in applying the latest technological innovations, including artificial intelligence (AI) and machine learning (ML), to the field of education. One of the main areas of interest to researchers is the use of ML to assist teachers in assessing students’ work on the one hand and to promote effective self-tutoring on the other hand. In this paper, we present a survey of the latest ML approaches to the automated evaluation of students’ natural language free-text, including both short answers to questions and full essays. Existing systematic literature reviews on the subject often emphasise an exhaustive and methodical study selection process and do not provide much detail on individual studies or a technical background to the task. In contrast, we present an accessible survey of the current state-of-the-art in student free-text evaluation and target a wider audience that is not necessarily familiar with the task or with ML-based text analysis in natural language processing (NLP). We motivate and contextualise the task from an application perspective, illustrate popular feature-based and neural model architectures and present a selection of the latest work in the area. We also remark on trends and challenges in the field.

show abstract

Section: Feature Sets and Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

Bai

Stede

2022

Int J Artif Intell Educ

View full text Add to dashboard Cite

show abstract

“…Intelligent gamebased learning systems (Lester et al, 2013;Richey et al, 2021) present significant benefits for practicing math concepts in smart spaces (Pires et al, 2019;Sun et al, 2021), specifically for early childhood education (Skene et al, 2022). Adapting NLP techniques to build various educational applications has been an appealing area of research for quite some time (Meurers, 2012;Blanchard et al, 2015;Lende and Raghuwanshi, 2016;Taghipour and Ng, 2016;Raamadhurai et al, 2019;Cahill et al, 2020;Ghosh et al, 2020). To slightly narrow down on these applications, building conversational agents for the smart education has been widely studied in the community (Graesser et al, 2004;Litman and Silliman, 2004;Kerry et al, 2009;Roos, 2018;Winkler and Söllner, 2018;Palasundram et al, 2019;Winkler et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

“…Investigating Artificial Intelligence (AI) systems that can help children in their learning process has been a challenging yet exciting area of research (Chassignol et al, 2018;Zhai et al, 2021). Utilizing Natural Language Processing (NLP) for building educational games and applications has gained popularity in the past decade (Lende and Raghuwanshi, 2016;Cahill et al, 2020). Game-based learning systems can offer significant advantages in teaching fundamental math concepts interactively, especially for younger students (Skene et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

NLU for Game-based Learning in Real: Initial Evaluations

Okur¹,

Sahay²,

Nachman³

2022

Preprint

View full text Add to dashboard Cite

Intelligent systems designed for play-based interactions should be contextually aware of the users and their surroundings. Spoken Dialogue Systems (SDS) are critical for these interactive agents to carry out effective goal-oriented communication with users in real-time. For the real-world (i.e., in-the-wild) deployment of such conversational agents, improving the Natural Language Understanding (NLU) module of the goal-oriented SDS pipeline is crucial, especially with limited task-specific datasets. This study explores the potential benefits of a recently proposed transformer-based multi-task NLU architecture, mainly to perform Intent Recognition on small-size domain-specific educational game datasets. The evaluation datasets were collected from children practicing basic math concepts via play-based interactions in game-based learning settings. We investigate the NLU performances on the initial proof-of-concept game datasets versus the real-world deployment datasets and observe anticipated performance drops in-the-wild. We have shown that compared to the more straightforward baseline approaches, Dual Intent and Entity Transformer (DIET) architecture (Bunk et al., 2020) is robust enough to handle real-world data to a large extent for the Intent Recognition task on these domain-specific in-the-wild game datasets.

show abstract

“…Investigating intelligent systems to assist children in their learning process has been an attractive area of research (Jia et al, 2020). Employing Natural Language Processing (NLP) for building educational applications has also gained popularity in the past decade (Meurers, 2012;Lende and Raghuwanshi, 2016;Taghipour and Ng, 2016;Cahill et al, 2020). Game-based learning environments can enhance significant benefits in teaching basic math concepts interactively, particularly for young learners (Skene et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System

Okur¹,

Sahay²,

Nachman³

2022

Preprint

View full text Add to dashboard Cite

Contextually aware intelligent agents are often required to understand the users and their surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning process. Within such complex frameworks, Spoken Dialogue Systems (SDS) are crucial building blocks to handle efficient task-oriented communication with children in game-based learning settings. We are working towards a multimodal dialogue system for younger kids learning basic math concepts. Our focus is on improving the Natural Language Understanding (NLU) module of the task-oriented SDS pipeline with limited datasets. This work explores the potential benefits of data augmentation with paraphrase generation for the NLU models trained on small task-specific datasets. We also investigate the effects of extracting entities for conceivably further data expansion. We have shown that paraphrasing with model-in-the-loop (MITL) strategies using small seed data is a promising approach yielding improved performance results for the Intent Recognition task.

show abstract

Context-based Automated Scoring of Complex Mathematical Responses

Cited by 4 publications

References 8 publications

A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

NLU for Game-based Learning in Real: Initial Evaluations

Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System

Contact Info

Product

Resources

About