Automatic pronunciation assessment for Mandarin Chinese

Abstract

This paper presents the algorithms used in a prototypical software system for automatic pronunciation assessment of Mandarin Chinese. The system uses forced alignment of HMM (Hidden Markov Models) to identify each syllable and the corresponding log probability for phoneme assessment, through a ranking-based confidence measure. The pitch vector of each syllable is then sent to a GMM (Gaussian Mixture Model) for tone recognition and assessment. We also compute the similarity of scores for intensity and rhythm between the target and test utterances. All four scores for phoneme, tone, intensity, and rhythm are parametric functions with certain free parameters. The overall scoring function was then formulated as a linear combination of these four scoring functions of phoneme, tone, intensity, and rhythm. Since there are both linear and nonlinear parameters involved in the overall scoring function, we employ the downhill Simplex search to fine-tune these parameters in order to approximate the scoring results obtained from a human expert. The experimental results demonstrate that the system can give consistent scores that are close to those of a human's subjective evaluation.

Experimental Investigation of Mandarin Pronunciation Quality Assessment System

“…Therefore, the mandarin automatic pronunciation quality assessment system includes three subsystems, and the block diagram is shown in figure 1. For the tone, we use a pre-trained Gaussian Mixture Model (GMM) classifier based on tone recognition technology to assess the tone accuracy, and its detail can be found in [12] [13]. For consonants and vowels, we use the traditional ASR techniques of Hidden Markov Model (HMM) and Viterbi search, which is the kernel of the system.…”

“…Therefore, the performance of pronunciation quality assessment system mainly depends on how precise is the confidence computed for the speech segments. Many researches about CALL systems based on ASR framework focus on the pronunciation quality assessment of speech segment [5][6] [7][8] [9] [11] [12] [13], and seek to explore a more efficient confidence of pronunciation quality. So far, the confidences include log likelihood score, log posteriori probability, segmental classification score, segmental duration score, fluency score, and so on.…”

“…Based on the theoretical framework of posterior probability, this algorithm has made some reasonable assumptions, and gained significant improvements in the computational complexity. So, GOP as a simplified algorithm of phoneme posterior probability is widely used in computeraided pronunciation quality assessment system [3][4][5] [12].…”

An Effective CALL System for Strongly Accented Mandarin Speech

“…The first two parts are evaluated using the ASR techniques of HMM and Viterbi search [7], which is the core of the system. The tone quality is evaluated using a Gaussian Mixture Model (GMM) classifier, which is not the focus of this paper, but can be found in [8] [10]. A block diagram of the system is shown in Fig.1.…”

“…The computer Aided Language Learning (CALL) system [1]- [8]can automatically score the quality of human speech ,and provide continuous feedback to learners without requiring the sole attention from the teacher. So it facilitates self study and encourage interactive use of the language rather than rote learning.…”