2022
DOI: 10.1109/taslp.2022.3153268
|View full text |Cite
|
Sign up to set email alerts
|

ARoBERT: An ASR Robust Pre-Trained Language Model for Spoken Language Understanding

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
3
1
1

Citation Types

0
5
0

Year Published

2023
2023
2024
2024

Publication Types

Select...
7
1
1

Relationship

0
9

Authors

Journals

citations
Cited by 15 publications
(5 citation statements)
references
References 40 publications
0
5
0
Order By: Relevance
“…SLU is an essential task for machines to infer correct semantic meaning (e.g., intent detection, slot filling) from human speech Zhou et al, 2020;Cheng et al, 2023b,d,e;. Traditionally, it can be solved by fine-tuning NLP models (especially PLMs) with the ASR hypothesis as input (Wang et al, 2022a). However, the ASR hypothesis often contains errors caused by ASR systems.…”
Section: Related Workmentioning
confidence: 99%
“…SLU is an essential task for machines to infer correct semantic meaning (e.g., intent detection, slot filling) from human speech Zhou et al, 2020;Cheng et al, 2023b,d,e;. Traditionally, it can be solved by fine-tuning NLP models (especially PLMs) with the ASR hypothesis as input (Wang et al, 2022a). However, the ASR hypothesis often contains errors caused by ASR systems.…”
Section: Related Workmentioning
confidence: 99%
“…The Transformer model [3,[19][20][21][22][23] is modeled and applied to natural language processing tasks using only self-attentive mechanisms.…”
Section: Transformermentioning
confidence: 99%
“…Pre-training models have shown great promise in natural language processing, with the Transformer model [1] proposing an encoder-decoder architecture based solely on the self-attention mechanism, enabling the construction of large-scale models that can be pretrained on vast amounts of data. Language models [2][3][4] can be broadly categorized into two types: autoregressive language modeling and autoencoder language modeling. autoregressive language models, such as ELMO [5], GPT [6], and T5 [7], predict the next possible word based on the preceding context, making them well-suited for generative tasks.…”
Section: Introductionmentioning
confidence: 99%
“…Since transformer architecture was first introduced in a 2017 paper titled "Attention is All You Need" by Vaswani et al, which used a self-attention mechanism to compute contextualized word embeddings. Large Language Model or language model with transformers have been applied in numerous NLU tasks across a wide range of business areas or industries, such as building chatbots [9,10,11], improving product recommendations [12], analyzing financial reports or news articles [13,14] as well as the increasing LLM applications in healthcare [15], which enhanced the efficiency of medical resources allocation and provided appropriate medical services to patients. Its exploration in USMLE patient notes automatic scoring has also achieved significant…”
Section: Introductionmentioning
confidence: 99%