A New Acoustic-Based Pronunciation Distance Measure

Abstract

We present an acoustic distance measure for comparing pronunciations, and apply the measure to assess foreign accent strength in American-English by comparing speech of non-native American-English speakers to a collection of native American-English speakers. An acoustic-only measure is valuable as it does not require the time-consuming and error-prone process of phonetically transcribing speech samples which is necessary for current edit distance-based approaches. We minimize speaker variability in the data set by employing speaker-based cepstral mean and variance normalization, and compute word-based acoustic distances using the dynamic time warping algorithm. Our results indicate a strong correlation of r = −0.71 (p < 0.0001) between the acoustic distances and human judgments of native-likeness provided by more than 1,100 native American-English raters. Therefore, the convenient acoustic measure performs only slightly lower than the state-of-the-art transcription-based performance of r = −0.77. We also report the results of several small experiments which show that the acoustic measure is not only sensitive to segmental differences, but also to intonational differences and durational differences. However, it is not immune to unwanted differences caused by using a different recording device.

Neural Representations for Modeling Variation in Speech

“…Additionally, finegrained pronunciation differences that are relevant for studying accented speech may not be captured by using a set of discrete symbols (Mermelstein, 1976;Duckworth et al, 1990;Cucchiarini, 1996). Bartelds et al (2020) therefore introduced an acoustic-only measure for comparing pronuncia-tions. In their method, they represented accented speech as 39-dimensional Mel-frequency cepstral coefficients (MFCCs), which were used to compute acoustic-based non-native-likeness ratings between non-native and native speakers of English.…”

“…This result was close to, but still not equal to the performance of a phonetic transcriptionbased approach (which showed a correlation of r = −0.77). Bartelds et al (2020) also conducted several small-scale experiments to investigate whether more fine-grained characteristics of human speech were captured compared to the phonetic transcription-based pronunciation difference measure. Their results showed that the acousticonly measure captured segmental differences, intonational differences, and durational differences, but that the method was not invariant to characteristics of the recording device.…”

“…Consequently, in this paper, we employ these self-supervised neural models to create an automatically determined acoustic-only pronunciation difference measure, and investigate whether this results in improved performance compared to the MFCC-based approach of Bartelds et al (2020) and the phonetic transcription-based approach of Wieling et al (2014).…”

“…We evaluate the performance of these algorithms using two different datasets. The first is identical to the dataset used by Bartelds et al (2020) and Wieling et al (2014). The second is a new dataset which only focuses on accented speech from a single group of (Dutch) non-native speakers for which human native-likeness judgements are also available.…”

“…To understand which aspects of pronunciation variation the neural models can capture, we conduct several additional small-scale experiments, in line with those of Bartelds et al (2020).…”