Improved pronunciation features for construct-driven assessment of non-native spontaneous speech

Chen, Lei; Zechner, Klaus; Xi, Xiaoming

doi:10.3115/1620754.1620819

Cited by 35 publications

(37 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, speaking rate and pause profile have been suggested as useful features in several studies and even in commercial speech-scoring product implementations (Franco e t al., 2010; Bernstein, Moere, & Cheng, 2010). A widely used feature for verifying pronunciation is the Goodness of Pronunciation (GOP) tool and its derived features (Witt, 1999;Franco et al, 2010;Chen, Zechner, & Xi, 2009). In particular, following the feature extraction method described in Chen et al…”

Section: Speech Featuresmentioning

confidence: 99%

Designing An Automated Assessment of Public Speaking Skills Using Multimodal Cues

Chen¹,

Feng²,

Leong³

et al. 2016

Learning Analytics

Self Cite

View full text Add to dashboard Cite

Traditional assessments of public speaking skills rely on human scoring. We report an initial study on the development of an automated scoring model for public speaking performances using multimodal technologies. Task design, rubric development, and human rating were conducted according to standards in educational assessment. An initial corpus of 17 speakers with 4 speaking tasks was collected using audio, video, and 3D motion capturing devices. A scoring model based on basic features in the speech content, speech delivery, and hand, body, and head movements significantly predicts human rating, suggesting the feasibility of using multimodal technologies in the assessment of public speaking skills.

show abstract

Section: Speech Featuresmentioning

confidence: 99%

Designing An Automated Assessment of Public Speaking Skills Using Multimodal Cues

Chen¹,

Feng²,

Leong³

et al. 2016

Learning Analytics

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is due to the fact that for most speakers different aspects of proficiency tend to be correlated. For example, more fluent speakers also achieve higher ROUGE scores (the correlation between ROUGE and pronunciation accuracy (Chen et al, 2009) is r = 0.62). As a result, a model which measures only one aspect of performance such as fluency may sometimes reach near optimal performance and adding further predictors leads to a relatively small gain.…”

Section: Metricmentioning

confidence: 99%

Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications

Dell’Orletta⋄

Wieling

Cimino

et al. 2014

View full text Add to dashboard Cite

ii IntroductionThe field of NLP and education has matured dramatically since the first workshop in 1997, where the primary focus was on grammatical error detection and correction. As a community we have continued to improve existing capabilities and to identify and generate innovative and creative methods. Automated writing evaluation systems are now commercially viable, and are used to score millions of test-taker essays on high-stakes assessments. The educational and assessment landscape, especially in the United States, continues to foster a strong interest and high demand that furthers the state-of-the-art in automated writing evaluation capabilities, expanding the analysis of written responses to writing genres beyond those typicallyfound on standardized assessments. Much of the current demand for creative new educational applications results from the development of the Common Core State Standards Initiative (CCSSI). The goal of CCSSI is to ensure college-and workplace-readiness. The CCSSI describes what K-12 students should be learning with regard to reading, writing, speaking, listening, and media and technology.Major advances in speech technology have made it possible to include speech in both assessment and Intelligent Tutoring Systems (ITS). These advances have made it possible for spoken constructed responses are now being evaluated. Consistent with this, there is also a renewed interest in spoken dialog for instruction and assessment. Relative to continued innovation, the explosive growth of mobile applications has increased interest in game-based assessment.In the past few years, the use of NLP in educational applications gained visibility outside of the Computational Linguistics (CL) community. First, the Hewlett Foundation reached out to public and private sectors by sponsoring two competitions (both inspired by the CCSSI): one for automated essay scoring, and one for scoring of short response items. The motivation driving these competitions was to engage the larger scientific community in this enterprise. Massive Open Online Courses (MOOCs) are now also beginning to incorporate automated writing scoring systems to manage the thousands of writing assignments that can be generated in a single MOOC course. Another breakthrough for educational applications within the CL community is the large number of shared task competitions in the last few years. There have been four shared tasks on grammatical error correction, with the most recent edition hosted at CoNLL 2014. In 2013, there was a SemEval Shared Task on Student Response Analysis and one on Native Language Identification (hosted at the 2013 edition of this workshop). All of these competitions increased the visibility of the research space for using NLP to build educational applications.As a community, we continue to improve existing capabilities and to identify and generate innovative ways to use NLP in applications for writing, reading, speaking, critical thinking, curriculum development, and assessment. Steady growth in the development of NLP-b...

show abstract

“…For the non-native speech corpus, the ASR based vowel duration extraction uses a two-pass method [5]: first the utterance is recognized using a non-native speech acoustic model (AM); then a native speech AM is used for forced alignment. This study also examines the performance of the vowel duration features when human transcriptions are used as input to the forced alignment instead of the ASR hypotheses; in this case, the ASR step is bypassed.…”

Section: Pronunciation Scoring 31 Vowel Duration Extraction and Normmentioning

confidence: 99%

“…In a simple approach, [4] correlated mean raw segment duration values with pronunciation scores (although this metric was also highly correlated with speaking rate). [5,6] calculated the average deviations between a non-native speaker's segments and mean segment durations trained on a native speaker corpus. In another approach, [2,6] used the average log probability of segment durations based on native speaker distributions of durations for each phone (after normalizing for rate of speech).…”

Section: Introductionmentioning

confidence: 99%

Gaussian Mixture Modeling of vowel durations for automated assessment of non-native speech

Sun

Evanini

2011

2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper investigates using Gaussian Mixture Model (GMM) based vowel duration features for automated assessment of non-native speech. Two different types of models were compared: a single GMM trained on a reference corpus of native speech and separate GMMs for different proficiency levels trained on a large corpus of scored non-native speech. 13 vowel categories were evaluated separately (after normalization by rate of speech), and a multiple regression model was used to evaluate the performance of all vowel categories combined. Experiments on an English language proficiency assessment show that the non-native speech GMMs outperform the native speech GMMs, and that all 13 vowels have significant correlations with human scores when the non-native speech GMMs are used. The multiple regression combination obtained correlations with human scores of 0.71 when transcriptions were used to extract the vowel durations and 0.64 when the Automatic Speech Recognition (ASR) output was used. The experiments demonstrate that the vowel duration feature based on non-native speech GMMs is a useful predictor of L2 proficiency and is robust to different datasets and situations.

show abstract

Improved pronunciation features for construct-driven assessment of non-native spontaneous speech

Cited by 35 publications

References 6 publications

Designing An Automated Assessment of Public Speaking Skills Using Multimodal Cues

Designing An Automated Assessment of Public Speaking Skills Using Multimodal Cues

Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications

Gaussian Mixture Modeling of vowel durations for automated assessment of non-native speech

Contact Info

Product

Resources

About