Testing the ability of speech recognizers to measure the effectiveness of encoding algorithms for digital speech transmission

Chernick, C. Michael; Leigh, Stefan D.; Mills, Kevin L.; Toense, R.

doi:10.1109/milcom.1999.821447

Cited by 10 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[3] and [7] have shown that a frequency-dependent gating or modulation of noise, which has low quality, can be highly intelligible. Thus, while ASR performance can be predicted by quality to some extent, the relationship is imperfect and indirect [8], [9].…”

mentioning

confidence: 99%

Evaluating Source Separation Algorithms With Reverberant Speech

Mandel

Bressler

Shinn‐Cunningham

et al. 2010

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-This paper examines the performance of several source separation systems on a speech separation task for which human intelligibility has previously been measured. For anechoic mixtures, automatic speech recognition (ASR) performance on the separated signals is quite similar to human performance. In reverberation, however, while signal separation has some benefit for ASR, the results are still far below those of human listeners facing the same task. Performing this same experiment with a number of oracle masks created with a priori knowledge of the separated sources motivates a new objective measure of separation performance, the Direct-path, Early echo, and Reverberation, of the Target and Masker (DERTM), which is closely related to the ASR results. This measure indicates that while the non-oracle algorithms successfully reject the direct-path signal from the masking source, they reject less of its reverberation, explaining the disappointing ASR performance.

show abstract

mentioning

confidence: 99%

Evaluating Source Separation Algorithms With Reverberant Speech

Mandel

Bressler

Shinn‐Cunningham

et al. 2010

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Chernick et al [4], [3] have evaluated the performance of a speech recognizer with the DoD-CELP [15] codec. Our study differs from theirs in four aspects.…”

Section: Related Workmentioning

confidence: 99%

“…This is detailed in section III-A. Finally, our evaluation also includes human recognition performance, whereas Chernick et al [4], [3] studies machine recognition performance only.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Speech recognition performance as an effective perceived quality predictor

Jiang

Schulzrinne

IEEE 2002 Tenth IEEE International Workshop on Quality of Service (Cat. No.02EX564)

View full text Add to dashboard Cite

Abstract-Determining the perceived quality of packet audio under packet loss usually requires human-based Mean Opinion Score (MOS) listening tests. We propose a new MOS estimation method based on machine speech recognition. Its automated, machine-based nature facilitates real-time monitoring of transmission quality without the need to conduct time-consuming listening tests. Our evaluation of this new method shows that it can use the word recognition ratio metric to reliably predict perceived quality.In particular, we find that although the absolute word recognition ratio of a speech recognizer may vary depending on the speaker, the relative word recognition ratio, obtained by dividing the absolute word recognition ratio with its own value at 0% loss, is speaker-independent. Therefore the relative word recognition ratio is well suited as a universal, speaker-independent MOS predictor.Finally we have also conducted human-based word recognition tests and examined its relationship with machine-based recognition results. Our analysis shows that they are correlated although not very linearly. Also we find that human-based word recognition ratio does not degrade significantly once packet loss is large ( 10%).

show abstract

Human and Automatic Speaker Recognition over Telecommunication Channels

Gallardo¹

2016

T-Labs Series in Telecommunication Services

View full text Add to dashboard Cite

The automatic detection of people's identity from their voices is part of modern telecommunication services. This generally requires the telephone transmission of the speech to remote servers that perform the recognition task. The transmission may introduce severe distortions that degrade the system performance and hence represents one of the major challenges speech technologies are currently facing.Similarly, humans also cope with the difficulty of reliably identifying talkers from speech transmitted over communication channels, particularly if the utterance heard is of short duration. The present research work addresses the evaluation of the human and of the automatic performances under different channel distortions caused by bandwidth limitation, codecs, and electro-acoustic user interfaces, among other impairments. Its main contribution is the demonstration of the benefits of communication channels of extended bandwidth, together with an insight into how speaker-specific characteristics of speech are preserved through different transmissions. It provides sufficient motivation for considering speaker recognition as a criterion for the migration from narrowband to enhanced bandwidths, such as wideband and super-wideband.

show abstract

Testing the ability of speech recognizers to measure the effectiveness of encoding algorithms for digital speech transmission

Cited by 10 publications

References 13 publications

Evaluating Source Separation Algorithms With Reverberant Speech

Evaluating Source Separation Algorithms With Reverberant Speech

Speech recognition performance as an effective perceived quality predictor

Human and Automatic Speaker Recognition over Telecommunication Channels

Contact Info

Product

Resources

About