A New ASR Evaluation Measure and Minimum Bayes-Risk Decoding for Open-domain Speech Understanding

Glass

IEEE/ACM Trans. Audio Speech Lang. Process.

et al. 2015

101

Spoken content retrieval refers to directly indexing and retrieving spoken content based on the audio rather than text descriptions. This potentially eliminates the requirement of producing text descriptions for multimedia content for indexing and retrieval purposes, and is able to precisely locate the exact time the desired information appears in the multimedia. Spoken content retrieval has been very successfully achieved with the basic approach of cascading automatic speech recognition (ASR) with text information retrieval: after the spoken content is transcribed into text or lattice format, a text retrieval engine searches over the ASR output to find desired information. This framework works well when the ASR accuracy is relatively high, but becomes less adequate when more challenging real-world scenarios are considered, since retrieval performance depends heavily on ASR accuracy. This challenge leads to the emergence of another approach to spoken content retrieval: to go beyond the basic framework of cascading ASR with text retrieval in order to have retrieval performances that are less dependent on ASR accuracy. This overview article is intended to provide a thorough overview of the concepts, principles, approaches, and achievements of major technical contributions along this line of investigation. This includes five major directions: 1) Modified ASR for Retrieval Purposes: cascading ASR with text retrieval, but the ASR is modified or optimized for spoken content retrieval purposes; 2) Exploiting the Information not present in ASR outputs: to try to utilize the information in speech signals inevitably lost when transcribed into phonemes and words; 3) Directly Matching at the Acoustic Level without ASR: for spoken queries, the signals can be directly matched at the acoustic level, rather than at the phoneme or word levels, bypassing all ASR issues; 4) Semantic Retrieval of Spoken Content: trying to retrieve spoken content that is semantically related to the query, but not necessarily including the query terms themselves; 5) Interactive Retrieval and Efficient Presentation of the Retrieved Objects: with efficient presentation of the retrieved objects, an interactive retrieval process incorporating user actions may produce better retrieval results and user experiences.Index Terms-Spoken content retrieval, spoken term detection, query by example, semantic retrieval, joint optimization, pseudo-relevance feedback, graph-based random walk, unsupervised acoustic pattern discovery, query expansion, interactive retrieval, summarization, key term extraction.

Section: B Summarization Title Generation and Key Term Extraction Fmentioning

confidence: 99%

Spoken Content Retrieval—Beyond Cascading Speech Recognition with Text Retrieval

Glass

IEEE/ACM Trans. Audio Speech Lang. Process.

et al. 2015

101

“…However, such an error may not significantly change the meaning of the sentence and in fact may be sufficiently correct for clinical use. This can be partly mitigated by using relative word importance to re-weight the final metric accordingly 79,80 . However, this still measures word-level equivalence rather than sentence-level resemblance 81 .…”

Section: Measures Of Automatic Speech Recognition Performancementioning

confidence: 99%

Assessing the accuracy of automatic speech recognition for psychotherapy

et al. 2020

Accurate transcription of audio recordings in psychotherapy would improve therapy effectiveness, clinician training, and safety monitoring. Although automatic speech recognition software is commercially available, its accuracy in mental health settings has not been well described. It is unclear which metrics and thresholds are appropriate for different clinical use cases, which may range from population descriptions to individual safety monitoring. Here we show that automatic speech recognition is feasible in psychotherapy, but further improvements in accuracy are needed before widespread use. Our HIPAA-compliant automatic speech recognition system demonstrated a transcription word error rate of 25%. For depression-related utterances, sensitivity was 80% and positive predictive value was 83%. For clinician-identified harm-related sentences, the word error rate was 34%. These results suggest that automatic speech recognition may support understanding of language patterns and subgroup variation in existing treatments but may not be ready for individual-level safety surveillance.

“…For ASR of morphologically-rich languages, some more adequate metrics can be applied: Letter/Character Error Rate (LER or CER) (Kurimo et al, 2006a,b), Phone Error Rate (PER), Syllable Error Rate (SylER) (Huang et al, 2000) or Morpheme Error Rate (Ablimit et al, 2010). There exist also some other measures, such as Inflectional Word Error Rate (IWER) (Bhanuprasad and Svenson, 2008;Karpov et al, 2011), Speaker Attributed Word Error Rate (NIST, 2009), Weighted Word Error Rate (WWER) (Nanjo and Kawahara, 2005), etc.…”

Section: Evaluating Asr Performancementioning

confidence: 99%

Automatic speech recognition for under-resourced languages: A survey

Besacier

Barnard

Karpov

et al. 2014

Speech Communication

407

195

Speech processing for under-resourced languages is an active field of research, which has experienced significant progress during the past decade. We propose, in this paper, a survey that focuses on automatic speech recognition (ASR) for these languages. The definition of under-resourced languages and the challenges associated to them are first defined. The main part of the paper is a literature review of the recent (last 8 years) contributions made in ASR for under-resourced languages. Examples of past projects and future trends when dealing with under-resourced languages are also presented. We believe that this paper will be a good starting point for anyone interested to initiate research in (or operational development of) ASR for one or several under-resourced languages. It should be clear, however, that many of the issues and approaches presented here, apply to speech technology in general (text-to-speech synthesis for instance).