A Prompt-Aware Neural Network Approach to Content-Based Scoring of Non-Native Spontaneous Speech

Ye, Qian; Ubale, Rutuja; Mulholland, M. Robert; Evanini, Keelan; Wang, Xinhao

doi:10.1109/slt.2018.8639697

Cited by 15 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Craighead et al (2020) explored text-based auxiliary tasks and train models in a multi-task manner using speech transcription and found the L1 prediction task to benefit the scoring performance. Recent work also explored specific aspects of speech scoring like response content scoring, where, the features from the transcription of response are modeled with a respective question to learn the relevance of response (Yoon and Lee 2019;Qian et al 2018). Qian et al (2019) build over the work done by Qian et al (2018) and model acoustic cues, prompt, and grammar features to improve scoring performance.…”

Section: Related Workmentioning

confidence: 99%

“…Recent work also explored specific aspects of speech scoring like response content scoring, where, the features from the transcription of response are modeled with a respective question to learn the relevance of response (Yoon and Lee 2019;Qian et al 2018). Qian et al (2019) build over the work done by Qian et al (2018) and model acoustic cues, prompt, and grammar features to improve scoring performance. Singla et al (2021a) in a recent work, use speech and text transformers (Shah et al 2021) to score candidate speech.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency

Bamdev¹,

Grover²,

Kumar³

et al. 2021

Preprint

View full text Add to dashboard Cite

English proficiency assessments have become a necessary metric for filtering and selecting prospective candidates for both academia and industry. With the rise in demand for such assessments, it has become increasingly necessary to have the automated human-interpretable results to prevent inconsistencies and ensure meaningful feedback to the second language learners. Feature-based classical approaches have been more interpretable in understanding what the scoring model learns. Therefore, in this work, we utilize classical machine learning models to formulate a speech scoring task as both a classification and a regression problem, followed by a thorough study

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency

Bamdev¹,

Grover²,

Kumar³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Linguistic features have also attracted research interests. As in [12], a prompt-aware feature was proposed for spontaneous speech evaluation. A context-aware GOP was proposed in [13] to incorporate phone transition factor and phone duration factor into the calculation of GOP.…”

Section: Introductionmentioning

confidence: 99%

“…Long short-term memory recurrent network (LSTM) was adopted in pronunciation assessment [19,20]. More recently, attention mechanism has also been applied [21,15,12] to speech evaluation. These studies have presented promising improvement on speech evaluation performance in the language specific tasks.…”

Section: Introductionmentioning

confidence: 99%

Multilingual Speech Evaluation: Case Studies on English, Malay and Tamil

Zhang¹,

Shi

Chen

2021

Preprint

View full text Add to dashboard Cite

Speech evaluation is an essential component in computerassisted language learning (CALL). While speech evaluation on English has been popular, automatic speech scoring on low resource languages remains challenging. Work in this area has focused on monolingual specific designs and handcrafted features stemming from resource-rich languages like English. Such approaches are often difficult to generalize to other languages, especially if we also want to consider suprasegmental qualities such as rhythm. In this work, we examine three different languages that possess distinct rhythm patterns: English (stresstimed), Malay (syllable-timed), and Tamil (mora-timed). We exploit robust feature representations inspired by music processing and vector representation learning. Empirical validations show consistent gains for all three languages when predicting pronunciation, rhythm and intonation performance.

show abstract

“…In the automated scoring area, several researchers have explored the use of diverse neural networks for essay scoring (Farag et al, 2018;Alikaniotis et al, 2016;Dong and Zhang, 2016) and spontaneous speech scoring (Chen et al, 2018a;Qian et al, 2018a,b) and they achieved comparable or superior performance to the sophisticated linguistic feature-based system. In particular, Qian et al (2018b) trained an automated scoring model covering the content aspect and achieved a further improvement over the generic model without content modeling.…”

Section: Introductionmentioning

confidence: 99%

Content Modeling for Automated Oral Proficiency Scoring System

Yoon¹,

Lee²

2019

Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

We developed an automated oral proficiency scoring system for non-native English speakers' spontaneous speech. Automated systems that score holistic proficiency are expected to assess a wide range of performance categories, and the content is one of the core performance categories. In order to assess the quality of the content, we trained a Siamese convolutional neural network (Siamese CNN) to model the semantic relationship between key points generated by experts and a test response. The correlation between human scores and Siamese CNN scores was comparable to human-human agreement (r = 0.63), and it was higher than the baseline content features. The inclusion of Siamese CNN-based feature to the existing state-of-the-art automated scoring model achieved a small but statistically significant improvement. However, the new model suffered from score inflation for long atypical responses with serious content issues. We investigated the reasons of this score inflation by analyzing the associations with linguistic features and identifying areas strongly associated with the score errors.

show abstract

A Prompt-Aware Neural Network Approach to Content-Based Scoring of Non-Native Spontaneous Speech

Cited by 15 publications

References 21 publications

Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency

Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency

Multilingual Speech Evaluation: Case Studies on English, Malay and Tamil

Content Modeling for Automated Oral Proficiency Scoring System

Contact Info

Product

Resources

About