Improving Generalisation to New Speakers in Spoken Dialogue State Tracking

Casanueva, Iñigo; Hain, Thomas; Green, Phil

doi:10.21437/interspeech.2016-404

Cited by 3 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experiments are performed within the context of a voice-enabled control system designed to help speakers with dysarthria to interact with their home devices Casanueva et al, 2016). The user can interact with the system in a mixed initiative way, speaking single-word commands from a total set of 36.…”

Section: Methodsmentioning

confidence: 99%

Using phone features to improve dialogue state tracking generalisation to unseen states

Casanueva

Hain

Nicolao

et al. 2016

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Self Cite

View full text Add to dashboard Cite

The generalisation of dialogue state tracking to unseen dialogue states can be very challenging. In a slot-based dialogue system, dialogue states lie in discrete space where distances between states cannot be computed. Therefore, the model parameters to track states unseen in the training data can only be estimated from more general statistics, under the assumption that every dialogue state will have the same underlying state tracking behaviour. However, this assumption is not valid. For example, two values, whose associated concepts have different ASR accuracy, may have different state tracking performance. Therefore, if the ASR performance of the concepts related to each value can be estimated, such estimates can be used as general features. The features will help to relate unseen dialogue states to states seen in the training data with similar ASR performance. Furthermore, if two phonetically similar concepts have similar ASR performance, the features extracted from the phonetic structure of the concepts can be used to improve generalisation. In this paper, ASR and phonetic structurerelated features are used to improve the dialogue state tracking generalisation to unseen states of an environmental control system developed for dysarthric speakers.

show abstract

Section: Methodsmentioning

confidence: 99%

Using phone features to improve dialogue state tracking generalisation to unseen states

Casanueva

Hain

Nicolao

et al. 2016

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Self Cite

View full text Add to dashboard Cite

show abstract

“…The mode of meaning included the specific speech inputs into the proposed models. Studies that solely used speech features [15,[24][25][26][27][28][29][30][31][32][33][34][35] were heavily speaker-dependent on their approach and tended to lean more toward intelligibility of the dysarthric speaker than their comprehensibility.…”

Section: Mode Of Meaning Extraction Usedmentioning

confidence: 99%

“…The studies that used vector encoding [15,[38][39][40][41][42][43]50,51] used NLU-based models, such long short-term memory neural networks or combinations of gated recurrent unit and convolutional neural networks to achieve the tasks of dialogue assessment in dysarthric speech, language understanding, and semantic pattern tracking.…”

Section: Nature Of Speech Representations Usedmentioning

confidence: 99%

“…However, a majority of the reviewed studies (n=17; 57% of the studies) involved audio data that were primarily sourced. These studies [15,[23][24][25]27,29,30,32,[34][35][36][42][43][44][45][46]50] foremost curated the data and augmented their input data representations to accommodate features that ordinary speech representations would not have accommodated in their natural form. Further models for speech processing were developed and these representations were used to train the models.…”

Section: Databases Usedmentioning

confidence: 99%

“…Therefore, this study reviewed models and methodologies that are used in comprehending meaning from audio dysarthric speech by means of speech-event representations, otherwise known as situational projections. Following the fact that familiarization formed the core paradigm of the reviewed works as a technique that affords listeners an opportunity to retune their stored linguistic representations [15,16], the following key research question arose: following the unique characteristics of each speaker, how do the reviewed models generalize over new dysarthric speech given the specificity of the speaker? This study discusses the approaches used in generalizing speaker-listener underpinnings in relation to familiarization in light of ontology extraction as a technique of interest.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Models and Approaches for Comprehension of Dysarthric Speech Using Natural Language Processing: Systematic Review

Alaka¹,

Kasamani²

2023

JMIR Rehabil Assist Technol

View full text Add to dashboard Cite

Background Speech intelligibility and speech comprehension for dysarthric speech has attracted much attention recently. Dysarthria is characterized by irregularities in the speed, strength, pitch, breath control, range, steadiness, and accuracy of muscle movements required for articulatory aspects of speech production. Objective This study examined the contributions made by other studies involved in dysarthric speech comprehension. We focused on the modes of meaning extraction used in generalizing speaker-listener underpinnings in light of semantic ontology extraction as a desired technique, applied method types, speech representations used, and databases sourced from. Methods This study involved a systematic literature review using 7 electronic databases: Cochrane Database of Systematic Reviews, Web of Science Core Collection, Scopus, PubMed, ACM, IEEE Xplore, and Google Scholar. The main eligibility criterion was the extraction of meaning from dysarthric speech using natural language processing or understanding approaches to improve on dysarthric speech comprehension. In total, out of 834 search results, 30 studies that matched the eligibility requirements were acquired following screening by 2 independent reviewers, with a lack of consensus being resolved through joint discussion or consultation with a third party. In order to evaluate the studies’ methodological quality, the risk of bias assessment was based on the Cochrane risk-of-bias tool version 2 (RoB2) with 23 of the studies (77%) registering low risk of bias and 7 studies (33%) raising some concern over the risk of bias. The overall quality assessment of the study was done using TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis). Results Following a review of 30 primary studies, this study revealed that the reviewed studies focused on natural language understanding or clinical approaches, with an increase in proposed solutions from 2020 onwards. Most studies relied on speaker-dependent speech features, while others used speech patterns, semantic knowledge, or hybrid approaches. The prevalent use of vector representation aligned with natural language understanding models, while Mel-frequency cepstral coefficient representation and no representation approaches were applied in neural networks. Hybrid representation studies aimed to reconstruct dysarthric speech or improve comprehension. Comprehensive databases, like TORGO and UA-Speech, were commonly used in combination with other curated databases, while primary data was preferred for specific or unique research objectives. Conclusions We found significant gaps in dysarthric speech comprehension characterized by the lack of inclusion of important listener or speech-independent features in the speech representations, mode of extraction, and data sources used. Further research is therefore proposed regarding the formulation of models that accommodate listener and speech-independent features through semantic ontologies that will be useful in the inclusion of key features of listener and speech-independent features for meaning extraction of dysarthric speech.

show abstract

Models and Approaches for Comprehension of Dysarthric Speech Using Natural Language Processing: Systematic Review (Preprint)

Alaka¹,

Kasamani²

2022

Preprint

View full text Add to dashboard Cite

UNSTRUCTURED In the past decade, Natural Language Processing has attracted significant attention in medical sciences. This includes speech intelligibility and partially speech comprehension, particularly in Dysarthric speech. Dysarthria is characterized by irregularities in the speed, strength, pitch, breath control, range, steadiness, and accuracy of muscle movements required for articulatory aspects of speech production. This study examined the contributions made by other studies involved in Dysarthric speech comprehension; in relation to the modes of meaning extraction used, applied method types, speech representations used, and databases sourced from. This study followed a systematic review approach to review and map 27 related studies. The findings of this study indicated a significant gap in the inclusion of listener and speech independent features, majorly attributed to the non-robust speech representations used. An index was formulated to illustrate the mappings of the reviewed literature and as such, further research is proposed regarding to the formulation of semantic ontologies that will be useful in the inclusion key features of listener and speech independent features for meaning extraction of dysarthric speech.

show abstract

Improving Generalisation to New Speakers in Spoken Dialogue State Tracking

Abstract: This is a repository copy of Improving generalisation to new speakers in spoken dialogue state tracking.

Cited by 3 publications

References 26 publications

Using phone features to improve dialogue state tracking generalisation to unseen states

Using phone features to improve dialogue state tracking generalisation to unseen states

Models and Approaches for Comprehension of Dysarthric Speech Using Natural Language Processing: Systematic Review

Models and Approaches for Comprehension of Dysarthric Speech Using Natural Language Processing: Systematic Review (Preprint)

Contact Info

Product

Resources

About