HypernasalityNet: Deep recurrent neural network for automatic hypernasality detection

Wang, Xiyue; Yang, Sen; Tang, Ming; Yin, Han; Huang, Hua; He, Ling

doi:10.1016/j.ijmedinf.2019.05.023

Cited by 27 publications

(40 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The maximum use of AI was to study hypernasality. Three studies tried to detect its presence ( 40 , 41 , 43 ) and two classified it according to severity ( 44 , 48 ). They used extracted features of speech as inputs for the classifiers.…”

Section: Resultsmentioning

confidence: 99%

“…These provide encouraging experimental results, but they do not reflect the practical performance of these systems. They characterize speech based on hypernasality ( 40 , 41 , 43 , 44 , 46 , 48 ), identifying phonemes ( 42 , 47 ) and misarticulations ( 39 , 45 ). These studies use features of speech data at a single point in time.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Current Applications of Artificial Intelligence in Cleft Care: A Scoping Review

et al. 2021

View full text Add to dashboard Cite

Objective: This scoping review aims to identify the various areas and current status of the application of artificial intelligence (AI) for aiding individuals with cleft lip and/or palate.Introduction: Cleft lip and/or palate contributes significantly toward the global burden on the healthcare system. Artificial intelligence is a technology that can help individuals with cleft lip and/or palate, especially those in areas with limited access to receive adequate care.Inclusion Criteria: Studies that used artificial intelligence to aid the diagnosis, treatment, or its planning in individuals with cleft lip and/or palate were included.Methodology: A search of the Pubmed, Embase, and IEEE Xplore databases was conducted using search terms artificial intelligence and cleft lip and/or palate. Gray literature was searched using Google Scholar. The study was conducted according to the PRISMA- ScR guidelines.Results: The initial search identified 458 results, which were screened based on title and abstracts. After the screening, removal of duplicates, and a full-text reading of selected articles, 26 publications were included. They explored the use of AI in cleft lip and/or palate to aid in decisions regarding diagnosis, treatment, especially speech therapy, and prediction.Conclusion: There is active interest and immense potential for the use of artificial intelligence in cleft lip and/or palate. Most studies currently focus on speech in cleft palate. Multi-center studies that include different populations, with collaboration amongst academicians and researchers, can further develop the technology.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Current Applications of Artificial Intelligence in Cleft Care: A Scoping Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…More recently, hypernasality evaluation algorithms rely on machine learning; frame-level features such as MFCCs are extracted from the segmented regions to train classifiers like support vector machine and Gaussian mixture models [7,8,9]. In similar vein, convolutional neural networks and recurrent neural networks have also been used to detect hypernasality [10,11].…”

Section: Introductionmentioning

confidence: 99%

An Attention Model for Hypernasality Prediction in Children with Cleft Palate

Mathad

Scherer

Chapman

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Hypernasality refers to the perception of abnormal nasal resonances in vowels and voiced consonants. Estimation of hypernasality severity from connected speech samples involves learning a mapping between the frame-level features and utterance-level clinical ratings of hypernasality. However, not all speech frames contribute equally to the perception of hypernasality. In this work, we propose an attention-based bidirectional long-short memory (BLSTM) model that directly maps the frame-level features to utterance-level ratings by focusing only on specific speech frames carrying hypernasal cues. The models performance is evaluated on the Americleft database containing speech samples of children with cleft palate and clinical ratings of hypernasality. We analyzed the attention weights over broad phonetic categories and found that the model yields results consistent with what is known in the speech science literature. Further, the correlation between the predicted and perceptual rating is found to be significant (r = 0.684, p < 0.001) and better than conventional BLSTMs trained using frame-wise and last-frame approaches.

show abstract

“…In the past there have been many proposals to evaluate HN based on acoustic information. While the results in terms of accuracy are generally excellent, most studies have used only a limited number of utterance types, such as sustained vowels [6][7][8][9][10][11]. This is not clearly compatible with standard clinical recommendations [2], which strongly recommend that patients are evaluated using a variety of phonemes and utterances with varying complexity (e.g., CAPS-A protocol, [2,12]).…”

Section: Introductionmentioning

confidence: 99%

“…Classification algorithms include Random Forest (RF), Support Vector Machine (SVM) or Artificial Neural Network (ANN). Acoustic features used in previous studies include, among others, Mel Frequency Cepstrum Coefficients (MFCCs), the Voice Low Tone to High Tone Ratio (VLTHTR) and the vowel formants and their bandwidth [6][7][8][9][10][11]. In most studies using this approach the fragments analyzed were either sustained vowels [6,8] or vowel fragments that had been annotated manually in words or sentences [7,[9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Which Utterance Types Are Most Suitable to Detect Hypernasality Automatically?

et al. 2021

View full text Add to dashboard Cite

Automatic tools to detect hypernasality have been traditionally designed to analyze sustained vowels exclusively. This is in sharp contrast with clinical recommendations, which consider it necessary to use a variety of utterance types (e.g., repeated syllables, sustained sounds, sentences, etc.) This study explores the feasibility of detecting hypernasality automatically based on speech samples other than sustained vowels. The participants were 39 patients and 39 healthy controls. Six types of utterances were used: counting 1-to-10 and repetition of syllable sequences, sustained consonants, sustained vowel, words and sentences. The recordings were obtained, with the help of a mobile app, from Spain, Chile and Ecuador. Multiple acoustic features were computed from each utterance (e.g., MFCC, formant frequency) After a selection process, the best 20 features served to train different classification algorithms. Accuracy was the highest with syllable sequences and also with some words and sentences. Accuracy increased slightly by training the classifiers with between two and three utterances. However, the best results were obtained by combining the results of multiple classifiers. We conclude that protocols for automatic evaluation of hypernasality should include a variety of utterance types. It seems feasible to detect hypernasality automatically with mobile devices.

show abstract

HypernasalityNet: Deep recurrent neural network for automatic hypernasality detection

Cited by 27 publications

References 41 publications

Current Applications of Artificial Intelligence in Cleft Care: A Scoping Review

Current Applications of Artificial Intelligence in Cleft Care: A Scoping Review

An Attention Model for Hypernasality Prediction in Children with Cleft Palate

Which Utterance Types Are Most Suitable to Detect Hypernasality Automatically?

Contact Info

Product

Resources

About