Comparing human and automatic speech recognition in simple and complex acoustic scenes

Spille, Constantin; Kollmeier, Birger; Meyer, Bernd T.

doi:10.1016/j.csl.2018.04.003

Cited by 36 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With recent advances in automatic speech recognition (ASR), there has been an increased interest in comparing human speech recognition (HSR) with ASR (Rader et al, 2015;Stolcke and Droppo, 2017;Spille and Meyer, 2017;Kell et al, 2018;Spille et al, 2018;Hu et al, 2020;Kollmeier et al, 2020). Such comparisons can be used to pinpoint existing weaknesses in ASR systems, but as the gap between HSR and ASR tightens further, ASR models may also be used as a proxy for human hearing.…”

Section: Introductionmentioning

confidence: 99%

“…The impressive results of these modelling efforts (even if not universal; Jacob et al 2021) have led to the suggestion that this approach should be extended more widely in neuroscience (Kriegeskorte, 2015; Richards et al, 2019). With recent advances in deep learning-based automatic speech recognition (ASR), there has therefore been an increased interest in comparing human speech recognition (HSR) with ASR (Rader et al, 2015; Stolcke and Droppo, 2017; Spille and Meyer, 2017; Kell et al, 2018; Spille et al, 2018; Hu et al, 2020; Kollmeier et al, 2020). ASR models may provide an opportunity to support or generate new hypotheses about the functioning of HSR.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Psychometrics of Automatic Speech Recognition

Weerts

Clopath²,

Goodman³

2021

Preprint

View full text Add to dashboard Cite

Automatic speech recognition (ASR) software has been suggested as a candidate model of the human auditory system thanks to dramatic improvements in performance in recent years. To test this hypothesis, we compared several state-of-the-art ASR systems to results from humans on a barrage of standard psychoacoustic experiments. While some systems showed qualitative agreement with humans in some tests, in others all tested systems diverged markedly from humans. In particular, none of the models used spectral invariance, temporal fine structure or speech periodicity in a similar way to humans. We conclude that none of the tested ASR systems are yet ready to act as a strong proxy for human speech recognition. However, we note that the more recent systems with better performance also tend to better match human results, suggesting that continued cross-fertilisation of ideas between human and automatic speech recognition may be fruitful. Our software is released as an open-source toolbox to allow researchers to assess future ASR systems or add additional psychoacoustic measures.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The Psychometrics of Automatic Speech Recognition

Weerts

Clopath²,

Goodman³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Despite that some of the latest ASR systems outperform human performance on specific tasks [14,22], ASR performance is clearly inferior to human auditory system under challenging acoustic conditions, i.e. when input speech is clipped, spectrally modulated, bandpass filtered, or masked by noise [15,23]. The present study is focused on the effect of LTR speech on E2E ASR.…”

Section: Locally-time Reversed Speechmentioning

confidence: 99%

Data Augmentation with Locally-time Reversed Speech for Automatic Speech Recognition

Ng,

Lee

2021

Preprint

View full text Add to dashboard Cite

Psychoacoustic studies have shown that locally-time reversed (LTR) speech, i.e., signal samples time-reversed within a short segment, can be accurately recognised by human listeners. This study addresses the question of how well a state-of-the-art automatic speech recognition (ASR) system would perform on LTR speech. The underlying objective is to explore the feasibility of deploying LTR speech in the training of end-to-end (E2E) ASR models, as an attempt to data augmentation for improving the recognition performance. The investigation starts with experiments to understand the effect of LTR speech on general-purpose ASR. LTR speech with reversed segment duration of 5 ms -50 ms is rendered and evaluated. For ASR training data augmentation with LTR speech, training sets are created by combining natural speech with different partitions of LTR speech. The efficacy of data augmentation is confirmed by ASR results on speech corpora in various languages and speaking styles. ASR on LTR speech with reversed segment duration of 15 ms -30 ms is found to have lower error rate than with other segment duration. Data augmentation with these LTR speech achieves satisfactory and consistent improvement on ASR performance.

show abstract

“…D IRECTION Of Arrival (DOA) estimation and Sound Source Localization with microphone arrays has been widely investigated and used in different applications, such as robot audition [1], [2], acoustic characterization [3], speech recognition [4], [5] or teleconference systems [6]. Most of the techniques in the literature can be roughly classified into i) Time Difference Of Arrival (TDOA) based techniques, which first use the Generalized Cross-Correlation (GCCs) functions [7] to estimate the TDOA and then compute the most reliable DOA for them, ii) beamforming based techniques, such as SRP-PHAT [8], [9], which search the direction that maximizes the power of the output of a beamformer, and iii) subspace techniques, such as Multiple Signal Classification (MUSIC) [10], [11], based on the eigenstructure of the narrowband cross-correlation matrices.…”

Section: Introductionmentioning

confidence: 99%

Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks

Diaz-Guerra,

Miguel,

Beltran

2020

Preprint

View full text Add to dashboard Cite

In this paper, we present a new sound source DOA estimation and tracking system based on the well known SRP-PHAT algorithm and a three-dimensional Convolutional Neural Network. It uses SRP-PHAT power maps as input features of a fully convolutional causal architecture that uses 3D convolutional layers to accurately perform the tracking of a sound source even in highly reverberant scenarios where most of the state of the art techniques fail. Unlike previous methods, since we do not use bidirectional recurrent layers and all our convolutional layers are causal in the time dimension, our system is feasible for realtime applications and it provides a new DOA estimation for each new SRP-PHAT map. To train the model, we introduce a new procedure to simulate random trajectories as they are needed during the training, equivalent to an infinite-size dataset with high flexibility to modify its acoustical conditions such as the reverberation time. We use both acoustical simulations on a large range of reverberation times and the actual recordings of the LOCATA dataset to prove the robustness of our system and its good performance even using low-resolution SRP-PHAT maps.

show abstract

Comparing human and automatic speech recognition in simple and complex acoustic scenes

Cited by 36 publications

References 34 publications

The Psychometrics of Automatic Speech Recognition

The Psychometrics of Automatic Speech Recognition

Data Augmentation with Locally-time Reversed Speech for Automatic Speech Recognition

Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks

Contact Info

Product

Resources

About