Phonetically-oriented word error alignment for speech recognition error analysis in speech translation

Ruiz, Nicholas; Federico, Marcello

doi:10.1109/asru.2015.7404808

Cited by 12 publications

(10 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Two key features are the use of a digital speech pronouncing dictionary for automated derivation of the phonemes from stimuli and responses, and the modification of the Levenshtein minimum edit distance via dynamic programming for automated alignment of phonemes. Traditionally, speech pronouncing dictionaries have been used in speech recognition research for purposes such as aligning phonemes in speech-to-text translation ( 38 ). Here, the open source CMUDict is used for aligning phonemes in text-to-text comparison.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Visualization of Speech Perception Analysis via Phoneme Alignment: A Pilot Study

et al. 2022

View full text Add to dashboard Cite

Objective: Speech tests assess the ability of people with hearing loss to comprehend speech with a hearing aid or cochlear implant. The tests are usually at the word or sentence level. However, few tests analyze errors at the phoneme level. So, there is a need for an automated program to visualize in real time the accuracy of phonemes in these tests.Method: The program reads in stimulus-response pairs and obtains their phonemic representations from an open-source digital pronouncing dictionary. The stimulus phonemes are aligned with the response phonemes via a modification of the Levenshtein Minimum Edit Distance algorithm. Alignment is achieved via dynamic programming with modified costs based on phonological features for insertion, deletions and substitutions. The accuracy for each phoneme is based on the F1-score. Accuracy is visualized with respect to place and manner (consonants) or height (vowels). Confusion matrices for the phonemes are used in an information transfer analysis of ten phonological features. A histogram of the information transfer for the features over a frequency-like range is presented as a phonemegram.Results: The program was applied to two datasets. One consisted of test data at the sentence and word levels. Stimulus-response sentence pairs from six volunteers with different degrees of hearing loss and modes of amplification were analyzed. Four volunteers listened to sentences from a mobile auditory training app while two listened to sentences from a clinical speech test. Stimulus-response word pairs from three lists were also analyzed. The other dataset consisted of published stimulus-response pairs from experiments of 31 participants with cochlear implants listening to 400 Basic English Lexicon sentences via different talkers at four different SNR levels. In all cases, visualization was obtained in real time. Analysis of 12,400 actual and random pairs showed that the program was robust to the nature of the pairs.Conclusion: It is possible to automate the alignment of phonemes extracted from stimulus-response pairs from speech tests in real time. The alignment then makes it possible to visualize the accuracy of responses via phonological features in two ways. Such visualization of phoneme alignment and accuracy could aid clinicians and scientists.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Figure 5 shows examples of alignments and phoneme F1-scores for four challenging stimulus-response pairs. The first two are examples of the consequences of insertion and deletion [see Table 4 from ( 38 )]. The third example is one of phonemic ambiguity but with different alignments caused by one substitution.…”

Section: Methodsmentioning

confidence: 99%

Visualization of Speech Perception Analysis via Phoneme Alignment: A Pilot Study

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Training the Error Model: We use the seed set S to train E. Using a phone-aware edit distance algorithm as in [18], we align the phoneme representationp i of each hypothesisŷ i with the phoneme representation p i of the corresponding reference y i . Using these alignments, we obtain the error sequence e i such that e j i = 0 if the token aligned with p i at position j is the same as p j i .…”

Section: The Error Detection Modelmentioning

confidence: 99%

Error-Driven Fixed-Budget ASR Personalization for Accented Speakers

Awasthi

Kansal

Sarawagi

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We consider the task of personalizing ASR models while being constrained by a fixed budget on recording speaker specific utterances. Given a speaker and an ASR model, we propose a method of identifying sentences for which the speaker's utterances are likely to be harder for the given ASR model to recognize. We assume a tiny amount of speakerspecific data to learn phoneme-level error models which help us select such sentences. We show that speaker's utterances on the sentences selected using our error model indeed have larger error rates when compared to speaker's utterances on randomly selected sentences. We find that fine-tuning the ASR model on the sentence utterances selected with the help of error models yield higher WER improvements in comparison to fine-tuning on an equal number of randomly selected sentence utterances. Thus, our method provides an efficient way of collecting speaker utterances under budget constraints for personalizing ASR models.

show abstract

“…Voice over IP professionals and telecommunications professionals generally use the word IVR. Also, voice response is used sometimes [9].…”

Section: Related Workmentioning

confidence: 99%

Automatic Speech Recognition in Mobile Customer Care Service

Sahu¹

2015

IJNPME

View full text Add to dashboard Cite

In the project an automatic speech system is used in mobile customer care services. In existing mobile customer care services, customer have to wait for 4 to 5 minutes to get into the option what they want to inquire. Based on the requirement, we go for filtering the incoming calls. Persons who require particular data are dynamically move to speech recognition system that identifies the type of the enquiry chosen. Speech recognition is the one which dynamically identifies the individual speaking based on analyzing the speech waves. It helps in identifying the voice of the speaker to know the recognized user. It also helps in accessing services like telephone banking, mobile shopping, database services and securing the information which is confidential.

show abstract

Phonetically-oriented word error alignment for speech recognition error analysis in speech translation

Cited by 12 publications

References 9 publications

Visualization of Speech Perception Analysis via Phoneme Alignment: A Pilot Study

Visualization of Speech Perception Analysis via Phoneme Alignment: A Pilot Study

Error-Driven Fixed-Budget ASR Personalization for Accented Speakers

Automatic Speech Recognition in Mobile Customer Care Service

Contact Info

Product

Resources

About