Information-Weighted Neural Cache Language Models for ASR

Verwimp, Lyan; Pelemans, Joris; hamme, Hugo Van; Wambacq, Patrick

doi:10.1109/slt.2018.8639543

Cited by 6 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given that we used the negative logarithm, words with a higher frequency are reflected in lower scores. Word frequency was also calculated using the 5-gram model by Verwimp et al (2019).…”

Section: Methodsmentioning

confidence: 99%

Exploring neural tracking of acoustic and linguistic speech representations in individuals with post-stroke aphasia

Kries

Clercq

Gillis

et al. 2023

Preprint

View full text Add to dashboard Cite

Aphasia is a communication disorder that affects processing of language at different levels (e.g., acoustic, phonological, semantic). Neural tracking of continuous speech, such as a story, allows to analyze brain responses to acoustic and linguistic properties. Even though neural tracking of speech measured via EEG may be an interesting tool to study aphasia in an ecologically valid way, this method has not yet been investigated in individuals with stroke-induced aphasia. Here, we explored processing of acoustic and linguistic speech representations in individuals with aphasia in the chronic phase after stroke and age-matched healthy controls. We found decreased neural tracking of acoustic and segmentation-related speech representations in individuals with aphasia. In addition, linguistic speech representations at the word-level displayed decreased amplitudes in individuals with aphasia around 200 ms over left-sided frontal electrodes. These results show that there is potential for neural tracking to capture language processing impairments in individuals with aphasia. However, more research is needed to validate these results. Nonetheless, this exploratory study shows that neural tracking of naturalistic, continuous speech presents a powerful tool to study aphasia.

show abstract

“…Given that we used the negative logarithm, words with a higher frequency are reflected in lower scores. Word frequency was also calculated using the 5-gram model by Verwimp et al (2019).…”

Section: Methodsmentioning

confidence: 99%

Exploring neural tracking of acoustic and linguistic speech representations in individuals with post-stroke aphasia

Kries

Clercq

Gillis

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Word surprisal was calculated as the negative logarithm of the conditional probability of the considered word given the four preceding words. This conditional probability was obtained with a 5-gram model created by Verwimp et al (2019) for the Dutch material and by Yilmaz et al (2016) for the Frisian material. Using these models, word frequency was obtained by calculating the negative probability of the uni-gram probability of the word.…”

Section: Methodsmentioning

confidence: 99%

Heard or understood? Neural tracking of language features in a comprehensible story, an incomprehensible story and a word list

Gillis

Vanthornhout

Francart

2022

Preprint

View full text Add to dashboard Cite

Speech comprehension is a complex neural process on which relies on activation and integration of multiple brain regions. In the current study, we evaluated whether speech comprehension can be investigated by neural tracking. Neural tracking is the phenomenon in which the brain responds in a time-locked fashion to specific features of the speech. These features can be acoustic, i.e., acoustic tracking, or derived from the content of the speech using language properties, i.e., language tracking. We evaluated whether neural tracking of speech differs between a comprehensible story, an incomprehensible story, and a word list. We evaluated the neural responses to speech of 19 participants (6 men). No significant difference regarding acoustic tracking was found. However, significant language tracking was only found for the comprehensible story. The most prominent effect was visible to word surprisal, a language feature at the word level. The neural response to word surprisal showed a prominent negativity between 300 and 400~ms, similar to the N400 in evoked response paradigms. This N400 was significantly more negative when the story was be comprehended, i.e., when words could be integrated in the context of previous words. These results show that language tracking can capture the effect of speech comprehension.

show abstract

“…When predicting the next word wt, the cache probability is computed based on the inner product between the current hidden state ht and the cache hidden state hj−1. Verwimp et al [21] assume that ht captures the previous context, and the cache probability based on the similarity between the current context and the context of cache words better influence the next word prediction.…”

Section: Embedding Extraction Methodsmentioning

confidence: 99%

History Utterance Embedding Transformer LM for Speech Recognition

Deng

Cheng

Miao

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

History utterances contain rich contextual information; however, better extracting information from the history utterances and using it to improve the language model (LM) is still challenging. In this paper, we propose the history utterance embedding Transformer LM (HTLM), which includes an embedding generation network for extracting contextual information contained in the history utterances and a main Transformer LM for current prediction. In addition, the two-stage attention (TSA) is proposed to encode richer contextual information into the embedding of history utterances (h-emb) while supporting GPU parallel training. Furthermore, we combine the extracted h-emb and embedding of current utterance (c-emb) through the dot-product attention and a fusion method for HTLM's current prediction. Experiments are conducted on the HKUST dataset and achieve a 23.4% character error rate (CER) on the test set. Compared with the baseline, the proposed method yields 12.86 absolute perplexity reduction and 0.8% absolute CER reduction.

show abstract

Information-Weighted Neural Cache Language Models for ASR

Cited by 6 publications

References 11 publications

Exploring neural tracking of acoustic and linguistic speech representations in individuals with post-stroke aphasia

Exploring neural tracking of acoustic and linguistic speech representations in individuals with post-stroke aphasia

Heard or understood? Neural tracking of language features in a comprehensible story, an incomprehensible story and a word list

History Utterance Embedding Transformer LM for Speech Recognition

Contact Info

Product

Resources

About