Improved Acoustic Modelling for Automatic Literacy Assessment of Children

Abstract

Automatic literacy assessment of children is a complex task that normally requires carefully annotated data. This paper focuses on a system for the assessment of reading skills, aiming to detection of a range of fluency and pronunciation errors. Naturally, reading is a prompted task, and thereby the acquisition of training data for acoustic modelling should be straightforward. However, given the prominence of errors in the training set and the importance of labelling them in the transcription, a lightly supervised approach to acoustic modelling has better chances of success. A method based on weighted finite state transducers is proposed, to model specific prompt corrections, such as repetitions, substitutions, and deletions, as observed in real recordings. Iterative cycles of lightly-supervised training are performed in which decoding improves the transcriptions and the derived models. Improvements are due to increasing accuracy in phone-to-sound alignment and in the training data selection. The effectiveness of the proposed methods for relabelling and acoustic modelling is assessed through experiemnts on the CHOREC corpus, in terms of sequence error rate and alignment accuracy. Improvements over the baseline of up to 60% and 23.3% respectively are observed.

ASR-Based Evaluation and Feedback for Individualized Reading Practice

“…For English, there is a long history of developing ASR for reading assessment and instruction [5]- [8]. For Dutch, research on this topic [9]- [12] has been limited. Ideally, an Automatic Reading Tutor, should be able to monitor children while they read aloud and help when they encounter difficulties [6].…”

Lattice-Based Lightly-Supervised Acoustic Model Training

“…The resulting best path is used as training data. A related approach was taken by Nicolao et al [8], in which they propose to use a transducer to model variations in children's speech. In Olcoz et al [12] the authors propose to correct for word-boundary errors and insertions in a lightly supervised alignment.…”

“…There is, however, a large amount of available data that has inaccurate and partial transcripts. Examples include traditional broadcast data [1,2,3,4], YouTube data [5], medical data [6], crowdsourced data [7], and children's data [8].…”

“…Most of these methods make use of a decode of the data made with a language model (LM) biased towards the data itself [9,10]. They typically proceed by either filtering the data at various levels of granularity [1,5,11,2], or by some combination or error correction algorithm [4,12,13,14,7,8,15,16]. The benefit of the latter techniques is that they can often maintain more data than through filtering, while filtering is perhaps most appropriate with large amounts of data such that false rejects is a nonissue.…”

“…The benefit of the latter techniques is that they can often maintain more data than through filtering, while filtering is perhaps most appropriate with large amounts of data such that false rejects is a nonissue. New acoustic models can be trained on the filtered or corrected transcriptions, and sometimes the process is repeated, yielding increasingly improved models [17,8].…”