Investigating the Effects of Word Substitution Errors on Sentence Embeddings

Voleti, Rohit; Liss, Julie; Berisha, Visar

doi:10.1109/icassp.2019.8683367

Cited by 12 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performing a pairwise comparison on the slopes of the similarity values, between our sensitivity analysis for Hungarian and a similar one for English [29], our results are quite satisfactory. The BOW (regardless of the usage of the stop words), SIF and uSIF are more robust (around rel.…”

Section: Results and Discussion 41 Sts-related Resultsmentioning

confidence: 65%

On the Effects of Automatic Transcription and Segmentation Errors in Hungarian Spoken Language Processing

Tündik

Kaszás

Szaszák

2019

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

Emerging Artificial Intelligence (AI) technology has brought machines to reach an equal or even superior level compared to human capabilities in several fields; nevertheless, among many other fields, making a computer able to understand human language still remains a challenge. When dealing with speech understanding, Automatic Speech Recognition (ASR) is used to generate transcripts, which are processed with text-based tools targeting Spoken Language Understanding (SLU). Depending on the ASR quality (which further depends on speech quality, the complexity of the topic, environment etc.), transcripts contain errors, which propagate further into the processing pipeline. Subjective tests show on the other hand, that humans understand quite well ASR-closed captions, despite the word and punctuation errors. Through word embedding based semantic parsing, the present paper is interested in quantifying the semantic bias introduced by ASR error propagation. As a special use case, speech summarization is also evaluated with regard to ASR error propagation. We show, that despite the higher word error rates seen with the highly inflectional Hungarian, the semantic space suffers least impact than the difference in Word Error Rate would suggest.

show abstract

Section: Results and Discussion 41 Sts-related Resultsmentioning

confidence: 65%

On the Effects of Automatic Transcription and Segmentation Errors in Hungarian Spoken Language Processing

Tündik

Kaszás

Szaszák

2019

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

show abstract

“…When using the AMI ASR (A1) transcripts instead of the manual transcripts, the decrease in ROUGE is around 2-3%. This relatively small drop is likely because even at 30% WER, the sentence/utterance embedding similarity between a manual source and an ASR source is about 0.70-0.85% [33,34]. This system achieves higher all ROUGE measures than extractive method CoreRank [6], and when compared to abstractive method Top-icSeg (without visual signals) [12] our system achieves higher ROUGE-2 and ROUGE-L although lower ROUGE-1.…”

Section: Modelmentioning

confidence: 80%

Abstractive Spoken Document Summarization Using Hierarchical Model with Multi-Stage Attention Diversity Optimization

Manakul

Gales

Wang³

2020

Interspeech 2020

View full text Add to dashboard Cite

ive summarization is a standard task for written documents, such as news articles. Applying summarization schemes to spoken documents is more challenging, especially in situations involving human interactions, such as meetings. Here, utterances tend not to form complete sentences and sometimes contain little information. Moreover, speech disfluencies will be present as well as recognition errors for automated systems. For current attention-based sequence-to-sequence summarization systems, these additional challenges can yield a poor attention distribution over the spoken document words and utterances, impacting performance. In this work, we propose a multi-stage method based on a hierarchical encoder-decoder model to explicitly model utterance-level attention distribution at training time; and enforce diversity at inference time using a unigram diversity term. Furthermore, multitask learning tasks including dialogue act classification and extractive summarization are incorporated. The performance of the system is evaluated on the AMI meeting corpus. The inclusion of both training and inference diversity terms improves performance, outperforming current state-of-the-art systems in terms of ROUGE scores. Additionally, the impact of ASR errors, as well as performance on the multitask learning tasks, is evaluated.

show abstract

“…The advantages of using the R-TLM structure are as follows. First, during the rescoring stage at test-time, there usually exists word errors in the transcriptions of past utterances, and the LSTM layer in the R-TLM structure provides LSTM hidden state representations which are more robust against such errors [31]. Second, the R-TLM provides complementary history representations from both the LSTM and Transformer-XL to the attention-based representation, and increases the network capability.…”

Section: R-tlm Structurementioning

confidence: 99%

Transformer Language Models with LSTM-Based Cross-Utterance Information Representation

Sun

Zhang

Woodland

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The effective incorporation of cross-utterance information has the potential to improve language models (LMs) for automatic speech recognition (ASR). To extract more powerful and robust crossutterance representations for the Transformer LM (TLM), this paper proposes the R-TLM which uses hidden states in a long short-term memory (LSTM) LM. To encode the cross-utterance information, the R-TLM incorporates an LSTM module together with a segmentwise recurrence in some of the Transformer blocks. In addition to the LSTM module output, a shortcut connection using a fusion layer which bypasses the LSTM module is also investigated. The proposed system was evaluated on the AMI meeting corpus, the Eval2000 and the RT03 telephone conversation evaluation sets. The best R-TLM achieved 0.9%, 0.6% and 0.8% absolute WER reductions over the single-utterance TLM baseline, and 0.5%, 0.3%, 0.2% absolute WER reductions over a strong cross-utterance TLM baseline on the AMI evaluation set, Eval2000 and RT03 respectively. Improvements on Eval2000 and RT03 were further supported by significance tests. R-TLMs were found to have better LM scores on words where recognition errors are more likely to occur. The R-TLM WER can be further reduced by interpolation with an LSTM-LM.

show abstract

Investigating the Effects of Word Substitution Errors on Sentence Embeddings

Cited by 12 publications

References 18 publications

On the Effects of Automatic Transcription and Segmentation Errors in Hungarian Spoken Language Processing

On the Effects of Automatic Transcription and Segmentation Errors in Hungarian Spoken Language Processing

Abstractive Spoken Document Summarization Using Hierarchical Model with Multi-Stage Attention Diversity Optimization

Transformer Language Models with LSTM-Based Cross-Utterance Information Representation

Contact Info

Product

Resources

About