Neural Approaches to Automated Speech Scoring of Monologue and Dialogue Responses

Ye, Qian; Lange, Patrick; Evanini, Keelan; Pugh, Robert; Ubale, Rutuja; Mulholland, M. Robert; Wang, Xinhao

doi:10.1109/icassp.2019.8683717

Cited by 19 publications

(10 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The final output of the RNN can be the output by the units at the last time step as well as the sequence of outputs for the entire time series. Attention is a mechanism proposed for the RNNs and is state-of-the-art for classification in most speech processing tasks ( Qian et al, 2019 ; Ubale et al, 2019 ). To employ attention in RNNs, outputs for all-time steps by a single RNN unit are collapsed by weighted averaging while the weights are learned automatically during training.…”

Section: Taxonomymentioning

confidence: 99%

A review of social background profiling of speakers from speech accents

Humayun,

Shuja,

Abas

2024

PeerJ Computer Science

View full text Add to dashboard Cite

Social background profiling of speakers is heavily used in areas, such as, speech forensics, and tuning speech recognition for accuracy improvement. This article provides a survey of recent research in speaker background profiling in terms of accent classification and analyses the datasets, speech features, and classification models used for the classification tasks. The aim is to provide a comprehensive overview of recent research related to speaker background profiling and to present a comparative analysis of the achieved performance measures. Comprehensive descriptions of the datasets, speech features, and classification models used in recent research for accent classification have been presented, with a comparative analysis made on the performance measures of the different methods. This analysis provides insights into the strengths and weaknesses of the different methods for accent classification. Subsequently, research gaps have been identified, which serve as a useful resource for researchers looking to advance the field.

show abstract

Section: Taxonomymentioning

confidence: 99%

A review of social background profiling of speakers from speech accents

Humayun,

Shuja,

Abas

2024

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Recent work also explored specific aspects of speech scoring like response content scoring, where, the features from the transcription of response are modeled with a respective question to learn the relevance of response (Yoon and Lee 2019;Qian et al 2018). Qian et al (2019) build over the work done by Qian et al (2018) and model acoustic cues, prompt, and grammar features to improve scoring performance. Singla et al (2021a) in a recent work, use speech and text transformers (Shah et al 2021) to score candidate speech.…”

Section: Related Workmentioning

confidence: 99%

Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency

Bamdev¹,

Grover²,

Kumar³

et al. 2021

Preprint

View full text Add to dashboard Cite

English proficiency assessments have become a necessary metric for filtering and selecting prospective candidates for both academia and industry. With the rise in demand for such assessments, it has become increasingly necessary to have the automated human-interpretable results to prevent inconsistencies and ensure meaningful feedback to the second language learners. Feature-based classical approaches have been more interpretable in understanding what the scoring model learns. Therefore, in this work, we utilize classical machine learning models to formulate a speech scoring task as both a classification and a regression problem, followed by a thorough study

show abstract

“…This high demand leads to a shortage of capable examiners, especially for spoken language assessment. Significant progress has been made in applying various techniques from speech processing, machine learning (ML) and Artificial Intelligence (AI) to automate the language assessment process [1,2,3,4]. With ML/AI based systems a concern, however, arises about potential bias within the system.…”

Section: Introductionmentioning

confidence: 99%

Analysing Bias in Spoken Language Assessment Using Concept Activation Vectors

Wei

Gales

Knill

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

A significant concern with deep learning based approaches is that they are difficult to interpret, which means detecting bias in network predictions can be challenging. Concept Activation Vectors (CAVs) have been proposed to address this problem. These use representations -perturbations of activation function outputs -of interpretable concepts to analyse how the network is influenced by the concept. This work applies CAVs to assess bias in a spoken language assessment (SLA) system, a regression task. One of the challenges with SLA is the wide range of concepts that can introduce bias in training data, for example L1, age, acoustic conditions, and particular human graders, or the grading instructions. Simply generating large quantities of expert marked data to check for all forms of bias is impractical. This paper uses CAVs applied to the training data to identify concepts that might be of concern, allowing a more targeted dataset to be collected to assess bias. The ability of CAVs to detect bias is assessed on the BULATS speaking test using both a standard system and a system to which bias was artificially introduced. A strong bias identified by CAVs on the training data matches the bias observed in expert marked held-out test data.

show abstract

Neural Approaches to Automated Speech Scoring of Monologue and Dialogue Responses

Cited by 19 publications

References 18 publications

A review of social background profiling of speakers from speech accents

A review of social background profiling of speakers from speech accents

Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency

Analysing Bias in Spoken Language Assessment Using Concept Activation Vectors

Contact Info

Product

Resources

About