Automatic Pronunciation Error Detection and Feedback Generation for CALL Applications

Ai, Renlong

doi:10.1007/978-3-319-20609-7_17

Cited by 8 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous research (Singh and Harun 2020;Alyammahi et al 2021) has shown that clear and concise communication can lead to an increase in productivity, efficiency, and understanding in the workplace. Increasingly, organizations are using automated speech scoring techniques to screen job candidates as well as to provide computer-assisted language learning (CALL) (Ai 2015). Typically, speech scoring systems comprise measurements of different competencies including Pronunciation, Fluency, Automatic Listening, etc.…”

Section: Introductionmentioning

confidence: 99%

“…Previous work in automated speech scoring (Witt and Young 1997;Ai 2015;Evanini and Wang 2013) have examined phone level scores derived from GMM-HMM (Gaussian Mixture Model − Hidden Markov Model) based speech recognizer outputs. With the proliferation of deep learning techniques, more recent studies (Ying 2019;Hu et al 2015;Sudhakara et al 2019) have used acoustic models trained using a Deep Neural Network to improve mispronunciation detection & diagnosis (MDD).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Building a Language-Independent Speech Scoring Assessment

Gupta,

Unnam,

Yadav

et al. 2024

AAAI

View full text Add to dashboard Cite

Automatic speech scoring is crucial in language learning, providing targeted feedback to language learners by assessing pronunciation, fluency, and other speech qualities. However, the scarcity of human-labeled data for languages beyond English poses a significant challenge in developing such systems. In this work, we propose a Language-Independent scoring approach to evaluate speech without relying on labeled data in the target language. We introduce a multilingual speech scoring system that leverages representations from the wav2vec 2.0 XLSR model and a force-alignment technique based on CTC-Segmentation to construct speech features. These features are used to train a machine learning model to predict pronunciation and fluency scores. We demonstrate the potential of our method by predicting expert ratings on a speech dataset spanning five languages - English, French, Spanish, German and Portuguese, and comparing its performance against Language-Specific models trained individually on each language, as well as a jointly-trained model on all languages. Results indicate that our approach shows promise as an initial step towards a universal language independent speech scoring.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Towards Building a Language-Independent Speech Scoring Assessment

Gupta,

Unnam,

Yadav

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…It has been shown to help people practice and improve their pronunciation skills [6][7][8]. CAPT consists of two components: an automated pronunciation evaluation component [9][10][11] and a feedback component [12]. The automated pronunciation evaluation component is responsible for detecting pronunciation errors in spoken speech, for example, for detecting words pronounced incorrectly by the speaker.…”

Section: Introductionmentioning

confidence: 99%

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Korzekwa,

Lorenzo-Trueba,

Drugman

et al. 2022

Preprint

View full text Add to dashboard Cite

The research community has long studied computer-assisted pronunciation training (CAPT) methods in non-native speech. Researchers focused on studying various model architectures, such as Bayesian networks and deep learning methods, as well as on the analysis of different representations of the speech signal. Despite significant progress in recent years, existing CAPT methods are not able to detect pronunciation errors with high accuracy (only 60% precision at 40%-80% recall). One of the key problems is the low availability of mispronounced speech that is needed for the reliable training of pronunciation error detection models. If we had a generative model that could mimic non-native speech and produce any amount of training data, then the task of detecting pronunciation errors would be much easier. We present three innovative techniques based on phoneme-to-phoneme (P2P), text-to-speech (T2S), and speech-to-speech (S2S) conversion to generate correctly pronounced and mispronounced synthetic speech. We show that these techniques not only improve the accuracy of three machine learning models for detecting pronunciation errors but also help establish a new state-of-the-art in the field. Earlier studies have used simple speech generation techniques such as P2P conversion, but only as an additional mechanism to improve the accuracy of pronunciation error detection. We, on the other hand, consider speech generation to be the first-class method of detecting pronunciation errors. The effectiveness of these techniques is assessed in the tasks of detecting pronunciation and lexical stress errors. Non-native English speech corpora of German, Italian, and Polish speakers are used in the evaluations. The best proposed S2S technique improves the accuracy of detecting pronunciation errors in AUC metric by 41% from 0.528 to 0.749 compared to the state-of-the-art approach.

show abstract

Classification of causes of speech recognition errors using attention-based bidirectional long short-term memory and modulation spectrum

Santoso

Makino

2019

2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)

View full text Add to dashboard Cite

Automatic Pronunciation Error Detection and Feedback Generation for CALL Applications

Cited by 8 publications

References 2 publications

Towards Building a Language-Independent Speech Scoring Assessment

Towards Building a Language-Independent Speech Scoring Assessment

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Classification of causes of speech recognition errors using attention-based bidirectional long short-term memory and modulation spectrum

Contact Info

Product

Resources

About