Improving Readability for Automatic Speech Recognition Transcription

Abstract

Modern Automatic Speech Recognition (ASR) systems can achieve high performance in terms of recognition accuracy. However, a perfectly accurate transcript still can be challenging to read due to grammatical errors, disfluency, and other noises common in spoken communication. These readable issues introduced by speakers and ASR systems will impair the performance of downstream tasks and the understanding of human readers. In this work, we present a task called ASR post-processing for readability (APR) and formulate it as a sequence-to-sequence text generation problem. The APR task aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of speakers. We further study the APR task from the benchmark dataset, evaluation metrics, and baseline models: first, to address the lack of task-specific data, we propose a method to construct a dataset for the APR task by using the data collected for grammatical error correction. Second, we utilize metrics adapted or borrowed from similar tasks to evaluate model performance on the APR task. Lastly, we use several typical or adapted pre-trained models as the baseline models for the APR task. Furthermore, we fine-tune the baseline models on the constructed dataset and compare their performance with a traditional pipeline method in terms of proposed evaluation metrics. Experimental results show that all the fine-tuned baseline models perform better than the transitional pipeline method and our adapted RoBERTa model outperforms the pipeline method by 4.95 and 6.63 BLEU points on two test sets respectively. The human evaluation and case study further reveal the ability of the proposed model to improve the readability of ASR transcripts.

Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation

“…Although speaking is faster than typing or writing, it can be timeconsuming to correct errors caused by speech artifacts, such as disfluencies and repetitions. Modern speech and NLP research has made much progress in cleaning disfluency and speech recognition errors [17,19,30], mostly for post-processing. Researchers have also proposed multimodal interaction methods to reduce editing effort, especially on mobile devices.…”

Automated Generation of Clinical Reports Using Sensing Technologies with Deep Learning Techniques

“…On the other hand, ASR is recognized as a subfield of NLP aimed at enabling computers to transcribe spoken language into text. This field has also been intensively explored for the development of various applications such as virtual assistants [10] and efficient human transcription systems [11].…”

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

“…However, the results indicate acceptable performance when using NER and dependency parsing through open-source and hybrid NLP models. The performance of the pipeline may increase over time with improvements in automatic speech recognition and text prediction and suggestion methods (methods that also use NLP models that are not covered within the scope of this study) [29][30][31]. However, in this study, the pipeline performance was potentially affected by the transcription errors or typing errors existing in the data set (n=16, 18% of 87 notes had at least one error; errors have not been corrected to contain real-world data features).…”