Development of the Cuhk Elderly Speech Recognition System for Neurocognitive Disorder Detection Using the Dementiabank Corpus

Ye, Zi; Hu, Shoukang; Li, Jinchao; Xie, Xurong; Yu, Jianwei; Xu, Junhao; Xue, Boyang; Liu, Shansong; Liu, Xunying; Meng, Helen

doi:10.1109/icassp39728.2021.9413634

Cited by 25 publications

(24 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the spectro-temporal level differences between impaired speech and normal speech such as overall reduction of speech volume, changes in the spectral envelope shape, weakened formants and slower speaking rate, recent research in this direction has been largely focused on front-end signal processing based techniques including tempo-stretching [69], [70], VTLP [62], and speed perturbation [15], [63], of normal speech recorded from healthy control speakers. The resulting speech data exhibiting certain high-level attributes such as a slower speaking rate and reduced speech volume is then used to augment the limited dysarthric or elderly speech training data [14], [15], [33]. A range of speech augmentation approaches investigated for dysarthric speech recognition [15] suggest the combined use of personalized, speaker dependent (SD) together with speaker independent (SI) speed perturbation factors produces the largest performance improvements.…”

Section: Corpusmentioning

confidence: 99%

See 1 more Smart Citation

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Jin¹,

Deng²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Despite the rapid progress of automatic speech recognition (ASR) technologies targeting normal speech, accurate recognition of dysarthric and elderly speech remains highly challenging tasks to date. It is difficult to collect large quantities of such data for ASR system development due to the mobility issues often found among these users. To this end, data augmentation techniques play a vital role. In contrast to existing data augmentation techniques only modifying the speaking rate or overall shape of spectral contour, fine-grained spectro-temporal differences between dysarthric, elderly and normal speech are modelled using a novel set of speaker dependent (SD) generative adversarial networks (GAN) based data augmentation approaches in this paper. These flexibly allow both: a) temporal or speed perturbed normal speech spectra to be modified and closer to those of an impaired speaker when parallel speech data is available; and b) for non-parallel data, the SVD decomposed normal speech spectral basis features to be transformed into those of a target elderly speaker before being re-composed with the temporal bases to produce the augmented data for state-of-the-art TDNN and Conformer ASR system training. Experiments are conducted on four tasks: the English UASpeech and TORGO dysarthric speech corpora; the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets. The proposed GAN based data augmentation approaches consistently outperform the baseline speed perturbation method by up to 0.91% and 3.0% absolute (9.61% and 6.4% relative) WER reduction on the TORGO and DementiaBank data respectively. Consistent performance improvements are retained after applying LHUC based speaker adaptation.

show abstract

Section: Corpusmentioning

confidence: 99%

“…based assistive technologies more natural alternatives [23], [24] even though speech quality is degraded. To this end, in recent years there has been increasing interest in developing ASR technologies that are suitable for dysarthric [9], [25]- [40] and elderly speech [14], [41]- [46].…”

Section: Introductionmentioning

confidence: 99%

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Jin¹,

Deng²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A 4-gram language model based on the DementiaBank Pitt transcripts, Switchboard and Fisher transcripts and additional text data of 392.4 millions words from the Gigaword collection released by LDC (LDC2011T07) was used in decoding. More details can be found in [75]. The TDNN baseline system was trained with the DementiaBank Pitt data only and its performance was shown in line 7 of Tab.…”

Section: Experiments On Dementiabank Pitt Elderly Speechmentioning

confidence: 99%

Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition

Xie

Liu

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Discriminative training techniques define state-ofthe-art performance for automatic speech recognition systems. However, they are inherently prone to overfitting, leading to poor generalization performance when using limited training data. In order to address this issue, this paper presents a full Bayesian framework to account for model uncertainty in sequence discriminative training of factored TDNN acoustic models. Several Bayesian learning based TDNN variant systems are proposed to model the uncertainty over weight parameters and choices of hidden activation functions, or the hidden layer outputs. Efficient variational inference approaches using a few as one single parameter sample ensure their computational cost in both training and evaluation time comparable to that of the baseline TDNN systems. Statistically significant word error rate (WER) reductions of 0.4%-1.8% absolute (5%-11% relative) were obtained over a state-of-the-art 900 hour speed perturbed Switchboard corpus trained baseline LF-MMI factored TDNN system using multiple regularization methods including Fsmoothing, L2 norm penalty, natural gradient, model averaging and dropout, in addition to i-Vector plus learning hidden unit contribution (LHUC) based speaker adaptation and RNNLM rescoring. The efficacy of the proposed Bayesian techniques is further demonstrated in a comparison against the state-of-theart performance obtained on the same task using the most recent hybrid and end-to-end systems reported in the literature. Consistent performance improvements were also obtained on a 450 hour HKUST conversational Mandarin telephone speech recognition task. On a third cross domain adaptation task requiring rapidly porting a 1000 hour LibriSpeech data trained system to a small DementiaBank elderly speech corpus, the proposed Bayesian TDNN LF-MMI systems outperformed the baseline system using direct weight fine-tuning by up to 2.5% absolute WER reduction.

show abstract

“…This presents a necessity for a scalable approach in screening cognitive impairments among older adults. Current NCD screening and diagnosis tests, such as the Montreal Cognitive Assessment (MoCA) [60], are mainly conducted as in-person tests by clinical professionals [89,100]. In-person assessments face limitations due to various factors, such as limited accessibility to the tests due to the lower mobility of some older adults, and shortages and inter-rater variabilities of clinicians [50].…”

Section: Introductionmentioning

confidence: 99%

“…To overcome these limitations, researchers are working on alternative solutions for NCD screening. Since NCDs are often manifested as communicative impairments, machine learning (ML) algorithms are offering a new type of support for NCD screening [3,13,22,38,47,55,56,66,72,97,100]. Therefore, a potential solution is to integrate speech analytics into a conversational agent (CA) to support highly accessible voice-based web applications that can interact with older adults through spoken language for screening NCD.…”

Section: Introductionmentioning

confidence: 99%

TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening

Ding¹,

Kang²,

HO³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Conversational agents (CAs) have the great potential in mitigating the clinicians' burden in screening for neurocognitive disorders among older adults. It is important, therefore, to develop CAs that can be engaging, to elicit conversational speech input from older adult participants for supporting assessment of cognitive abilities. As an initial step, this paper presents research in developing the backchanneling ability in CAs in the form of a verbal response to engage the speaker. We analyzed 246 conversations of cognitive assessments between older adults and human assessors, and derived the categories of reactive backchannels (e.g. "hmm") and proactive backchannels (e.g. "please keep going"). This is used in the development of TalkTive, a CA which can predict both timing and form of backchanneling during cognitive assessments. The study then invited 36 older adult participants to evaluate the backchanneling feature. Results show that proactive backchanneling is more appreciated by participants than reactive backchanneling. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI; Auditory feedback; Empirical studies in interaction design; Soundbased input / output.

show abstract

Development of the Cuhk Elderly Speech Recognition System for Neurocognitive Disorder Detection Using the Dementiabank Corpus

Cited by 25 publications

References 35 publications

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition

TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening

Contact Info

Product

Resources

About