Automatically Measuring L2 Speech Fluency without the Need of ASR: A Proof-of-concept Study with Japanese Learners of French

Abstract

This research work investigates the possibility of using automatic acoustic measures to assess speech fluency in the context of second language (L2) acquisition. To this end, three experts rated speech recordings of Japanese learners of French who were instructed to read aloud a 21-sentence-long text. A Forward-Backward Divergence Segmentation (FBDS) algorithm was used to segment speech recordings (sentences) into acoustically homogeneous units at a subphonemic scale. The FBDS processing results were used-along with more classic measures such as raw percentage of speech and length/standard deviation of silent pauses-to estimate speech rate and regularity of speech rate, while a formant tracking algorithm was used to estimate speech fluidity (i.e., quality of coarticulation). A step-by-step multiple linear regression was finally computed to predict the experts' mean fluency ratings. Results show that FBDS-derived measures, raw percentage of speech, and standard deviation of the first formant curve derivative can be combined together to calculate accurate estimates of speakers' fluency scores (R = .92; P < .001). As only low-level signal features were used in the study, the method could also be relevant for the assessment of speakers of other target languages, as well as for the assessment of disordered speech.

Prediction of L2 speech proficiency based on multi-level linguistic features

“…Since the use of such systems has limitations, as they are dependent on the language for which the models have been trained, new methods have recently been developed to measure fluency more automatically and independently of the target language. For example, the algorithm presented by [8] can be mentioned, resulting from pilot work on the automatic assessment of phonetic fluency of Japanese learners of French in reading task [5,9]. This work relies on the forward-backward divergence segmentation method [10] based on the detection of breaks in the energy trajectory of the speech signal over time and allows, in addition, to compute variables from pseudo-syllables [11] and silent pauses, such as speech rate or percentage of speech.…”

“…This study is part of a joint research project between the ALAIA 5 (Foreign Language Learning Assisted by Artificial Intelligence) laboratory and the project "From corpus to target data as steps for automatic assessment of L2 speech: L2 French phonological lexicon of Japanese learners" 6 , concerning the automatic assessment of the oral production of Japanese learners of French, partly relying on the use of the CLIJAF corpus 7 .…”

“…The difficulties of assessing oral production differ from the difficulties of assessing written production, in that the former also relies on the phonetic-phonological level. However, most studies addressing the automatic evaluation of L2 oral skills focus on a single linguistic level, such as pronunciation [2], fluency [3] or lexical richness [4], and uses relatively constrained production tasks, such as reading out loud [5] or repetition [2].…”

Optimized Prediction of Fluency of L2 English Based on Interpretable Network Using Quantity of Phonation and Quality of Pronunciation

“…cessing Following a previous study [14], envelopebased syllable detection was used, which is provided as Praat script [6]. Then, speaking rate was calculated as speaking rate = #syllables total duration of phonation…”

Predicting lexical skills from oral reading with acoustic measures

“…Previous work has exploited fluency metrics such as speaking rate and pause lengths and frequencies to predict nonnative adult speaker proficiency in communication settings [14]. Fontan et al [15] used low level signal features to estimate speech rate and its regularity in order to predict human ratings of fluency for Japanese learners of French. In Deng et al [16] measures based on the count and duration of morae (syllable like units) are used to predict fluency levels of recordings of spontaneous speech, however, the features were extracted from a manual transcription.…”