Language model fusion for streaming end to end speech recognition

Cabrera, Rodrigo; Liu, Xiaofeng; Ghodsi, Mohammadreza; Matteson, Zebulun; Weinstein, Eugene; Kannan, Anjuli

doi:10.48550/arxiv.2104.04487

Cited by 3 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The contextual fusion layer integrates information from the entity models into the pre-trained LM by linearly interpolating their final layer output representations based on the utterance context 4 . The fusion layer represents the entity models and the pre-trained LM using a class embedding matrix W c ∈ R N +1×d , where N is the number of entity models.…”

Section: Contextual Fusion Layermentioning

confidence: 99%

“…Our language models can be directly used for shallow fusion [38,4] and n-best hypothesis rescoring [14,36] in seq2seq based speech recognition systems. Our approach can also be extended to integrate entity models directly into the decoders of these systems.…”

Section: Future Workmentioning

confidence: 99%

“…We do not have absolute position information while training out entity models on entity catalogues 3. Note that k + 1 here does not represent the time dimension, but rather represent the k + 1 different outputs obtained by changing the utterance context length from 0 to k 4. This is slightly different from Eq.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Towards Continual Entity Learning in Language Models for Conversational Agents

Gadde,

Bulyko

2021

Preprint

View full text Add to dashboard Cite

Neural language models (LM) trained on diverse corpora are known to work well on previously seen entities, however, updating these models with dynamically changing entities such as place names, song titles and shopping items requires re-training from scratch and collecting full sentences containing these entities. We aim to address this issue, by introducing entity-aware language models (EALM), where we integrate entity models trained on catalogues of entities into the pretrained LMs. Our combined language model adaptively adds information from the entity models into the pre-trained LM depending on the sentence context. Our entity models can be updated independently of the pre-trained LM, enabling us to influence the distribution of entities output by the final LM, without any further training of the pre-trained LM. We show significant perplexity improvements on task-oriented dialogue datasets, especially on long-tailed utterances, with an ability to continually adapt to new entities (to an extent).Preprint. Under review.

show abstract

Section: Contextual Fusion Layermentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Towards Continual Entity Learning in Language Models for Conversational Agents

Gadde,

Bulyko

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…End-to-end ASR models [38], as opposed to traditional Gaussian mixture models, have been increasingly gaining popularity since end-to-end models consist of less componentshence, reducing maintenance costs. However, integration of external LMs into [5,21,29], and personalization of [11,33,34], end-toend systems remains an active research area. With respect to LM, Neural Network LMs (NNLM) [1] have gained popularity within ASR [12,30,43].…”

Section: Beyond Irmentioning

confidence: 99%

Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants

Gysel

Tsagkias

Pusateri

et al. 2020

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

We focus on improving the effectiveness of a Virtual Assistant (VA) in recognizing emerging entities in spoken queries. We introduce a method that uses historical user interactions to forecast which entities will gain in popularity and become trending, and it subsequently integrates the predictions within the Automated Speech Recognition (ASR) component of the VA. Experiments show that our proposed approach results in a 20% relative reduction in errors on emerging entity name utterances without degrading the overall recognition quality of the system.

show abstract

The construction of Bert fusion model of speech recognition and sensing for South China electricity charge service scenario

Wu,

Zheng

2023

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Electric charge service and management is an important part of electric power work. The effective recovery of the electric charge relates to the smooth development of daily work and continuous improvement of the operation and management of power supply enterprises. With the large-scale implementation of the card prepayment system, the problem of electricity customers defaulting on electricity charges has been solved to a large extent, but some large electricity users still fail to pay electricity charges on time. Therefore, under the current situation of power grid development, it is still necessary to strengthen the service and management of electricity charges to promote efficient recovery of electricity charges. Speech recognition technology has increasingly become the focus of research institutions at home and abroad. People are committed to enabling machines to understand human speech instructions and hope to control the machine through speech. The research and development of speech recognition will greatly facilitate people's lives shortly. The development of 5G technology and the proposal of 6G technology make the interconnection of all things not only a hope but also a reality. To realize the interconnection of all things, one of the key technical breakthroughs is the development of a new human–computer interaction sensing system. Under the guidance of relevant theories and methods, this paper systematically analyzes the user structure, electricity charge recovery management and service system, existing problems and causes in South China, and clarifies the necessity of design and application of electricity charge service system in South China power supply companies. The experimental data and empirical analysis results show that the optimized Bert fusion model can provide more digital support for the power supply companies in South China in terms of electricity charge recovery efficiency, management level system improvement, and electricity charge service.

show abstract

Language model fusion for streaming end to end speech recognition

Cited by 3 publications

References 17 publications

Towards Continual Entity Learning in Language Models for Conversational Agents

Towards Continual Entity Learning in Language Models for Conversational Agents

Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants

The construction of Bert fusion model of speech recognition and sensing for South China electricity charge service scenario

Contact Info

Product

Resources

About