End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model

Abstract

In this paper, we propose an end-to-end (E2E) neural network model to detect autism spectrum disorder (ASD) from children’s voices without explicitly extracting the deterministic features. In order to obtain the decisions for discriminating between the voices of children with ASD and those with typical development (TD), we combined two different feature-extraction models and a bidirectional long short-term memory (BLSTM)-based classifier to obtain the ASD/TD classification in the form of probability. We realized one of the feature extractors as the bottleneck feature from an autoencoder using the extended version of the Geneva minimalistic acoustic parameter set (eGeMAPS) input. The other feature extractor is the context vector from a pretrained wav2vec2.0-based model directly applied to the waveform input. In addition, we optimized the E2E models in two different ways: (1) fine-tuning and (2) joint optimization. To evaluate the performance of the proposed E2E models, we prepared two datasets from video recordings of ASD diagnoses collected between 2016 and 2018 at Seoul National University Bundang Hospital (SNUBH), and between 2019 and 2021 at a Living Lab. According to the experimental results, the proposed wav2vec2.0-based E2E model with joint optimization achieved significant improvements in the accuracy and unweighted average recall, from 64.74% to 71.66% and from 65.04% to 70.81%, respectively, compared with a conventional model using autoencoder-based BLSTM and the deterministic features of the eGeMAPS.

Voice as a Biomarker of Pediatric Health: A Scoping Review

“…Acoustic features [13][14][15] and phonetic features [16] were extracted to train machine learning algorithms in classifying children with intellectual disabilities. A majority of the included studies centered on training machine learning algorithms to classify children with autism spectrum disorder (and Down Syndrome [9]) using acoustic features [9,[17][18][19][20][21], vocal features [22,23], voice prosody features [24], pre-linguistic vocal features [25], and speech features [26,27]. In particular, Wu et al (2019) [21] focused on acoustic features of crying sounds in children of 2 to 3 years of age, while Pokorny et al (2017) [28] concentrated on pre-linguistic vocal features in 10-month-old babies.…”

“…Miodonska 2016 [60] Szklanny 2019 [70] Woloshuk 2018 [61] Singapore [2] Balamurali 2021 [52] Hee 2019 [46] South Korea [2] Lee 2020 [19] Lee 2022 [20] Sri Lanka [2] Kariyawasam 2019 [32] Wijesinghe 2019 [27] Sweden [1] Pokorny 2017 [28] Turkey [1] Satar 2022 [38] United Kingdom [1] Alharbi 2018 [51] USA [12] Asgari 2021 [22] Chi 2022 [26] Cho 2019 [17] Ji 2021 [35] Ji 2019 [36] MacFarlane 2022 [23] Manigault 2022 [67] McGinnis 2019 [63] Onu 2019 [37] Sadeghian 2015 [49] Suthar 2022 [50] VanDam 2015 [58] Appendix C…”

“…A table listing the condition group and condition type being analyzed by each study included in the scoping review and their associated reference number. Chi 2022 [26] Cho 2019 [17] Deng 2017 [18] Jayasree 2021 [9] Lee 2020 [19] Lee 2022 [20] MacFarlane 2022 [23] Nakai 2017 [24] Pokorny 2017 [28] Wijesinghe 2019 [27] Wu 2019 [21] Dyslexia Kariyawasam 2019 [32] Ribeiro 2020 [33] Intellectual Disability…”

Knowledge-driven speech features for detection of Korean-speaking children with autism spectrum disorder*

“…These abnormal speech patterns of ASD children have motivated researchers to focus on speech signals to detect ASD (Asgari et al, 2021;Beccaria et al, 2022;Cho et al, 2019;Lee et al, 2023;MacFarlane et al, 2022;Mohanta et al, 2020). It would reduce the time and effort spent for diagnosis of ASD and could be easily utilized for screening of the disorder.…”

“…It would reduce the time and effort spent for diagnosis of ASD and could be easily utilized for screening of the disorder. Automated ASD detection models have been developed based on hand-crafted features (Asgari et al, 2021;Cho et al, 2019;MacFarlane et al, 2022;Mohanta et al, 2020), or, recently, deep learning (Lee et al, 2023). Hand-crafted features were to reflect the characteristics specific to the dataset used, which resulted in difficulty in generalizing to other datasets.…”