UltraSuite: A Repository of Ultrasound and Acoustic Data from Child Speech Therapy Sessions

Abstract

We introduce UltraSuite, a curated repository of ultrasound and acoustic data, collected from recordings of child speech therapy sessions. This release includes three data collections, one from typically developing children and two from children with speech sound disorders. In addition, it includes a set of annotations, some manual and some automatically produced, and software tools to process, transform and visualise the data.

Synchronising Audio and Ultrasound by Learning Cross-Modal Embeddings

“…Uncorrelated segments: Speech therapy data contains interactions between the therapist and patient. The audio therefore contains speech from both speakers, while the ultrasound captures only the patient's tongue [16]. As a result, parts of the recordings will consist of completely uncorrelated audio and ultrasound.…”

“…This allows us to control how the model is trained and verify its performance using ground truth synchronisation offsets. We use Ul-traSuite 2 : a repository of ultrasound and acoustic data gathered from child speech therapy sessions [16]. We used all three datasets from the repository: UXTD (recorded with typically developing children), and UXSSD and UPX (recorded with children with speech sound disorders).…”

Speaker-independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech

“…Although ultrasound imaging is becoming less expensive to acquire, there is still a lack of large publicly available databases to evaluate automatic processing methods. The UltraSuite Repository [20], which we use in this work, helps alleviate this issue, but it still does not compare to standard speech recognition or image classification databases, which contain hundreds of hours of speech or millions of images.…”

“…We use the Ultrax Typically Developing dataset (UXTD) from the publicly available UltraSuite repository 1 [20]. This dataset contains synchronized acoustic and ultrasound data from 58 typically developing children, aged 5-12 years old (31 female, 27 male).…”

“…This dataset contains synchronized acoustic and ultrasound data from 58 typically developing children, aged 5-12 years old (31 female, 27 male). The data was aligned at the phone-level, according to the methods described in [20,26]. For this work, we discarded the acoustic data and focused only on the B-Mode ultrasound images capturing a midsaggital view of the tongue.…”