Audio quality evaluation by experienced and inexperienced listeners

Schinkel-Bielefeld, Nadja; Lotze, Netaya; Nagel, Frederik

doi:10.1121/1.4799190

Cited by 15 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of the first user study indicate that it is impossible to comprehend the target transcription of adversarial perturbations and only the original transcription is recognized by human listeners. The second part of the listening test is a MUSHRA test [46] in order to rate the quality of different algorithm setups. The results show that the psychoacoustic model greatly increases the quality of the adversarial examples.…”

Section: Simon and Garfunkel The Sound Of Silencementioning

confidence: 99%

See 1 more Smart Citation

Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding

Schönherr¹,

Kohls²,

Zeiler³

et al. 2019

Proceedings 2019 Network and Distributed System Security Symposium

200

157

View full text Add to dashboard Cite

Voice interfaces are becoming accepted widely as input methods for a diverse set of devices. This development is driven by rapid improvements in automatic speech recognition (ASR), which now performs on par with human listening in many tasks. These improvements base on an ongoing evolution of deep neural networks (DNNs) as the computational core of ASR. However, recent research results show that DNNs are vulnerable to adversarial perturbations, which allow attackers to force the transcription into a malicious output.In this paper, we introduce a new type of adversarial examples based on psychoacoustic hiding. Our attack exploits the characteristics of DNN-based ASR systems, where we extend the original analysis procedure by an additional backpropagation step. We use this backpropagation to learn the degrees of freedom for the adversarial perturbation of the input signal, i.e., we apply a psychoacoustic model and manipulate the acoustic signal below the thresholds of human perception. To further minimize the perceptibility of the perturbations, we use forced alignment to find the best fitting temporal alignment between the original audio sample and the malicious target transcription. These extensions allow us to embed an arbitrary audio input with a malicious voice command that is then transcribed by the ASR system, with the audio signal remaining barely distinguishable from the original signal. In an experimental evaluation, we attack the state-of-the-art speech recognition system Kaldi and determine the best performing parameter and analysis setup for different types of input. Our results show that we are successful in up to 98 % of cases with a computational effort of fewer than two minutes for a ten-second audio file. Based on user studies, we found that none of our target transcriptions were audible to human listeners, who still understand the original speech content with unchanged accuracy.

show abstract

Section: Simon and Garfunkel The Sound Of Silencementioning

confidence: 99%

“…In the second part of the study, we have conducted a Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) test, which is commonly used to rate the quality of audio signals [46].…”

Section: B Mushra Testmentioning

confidence: 99%

Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding

Schönherr¹,

Kohls²,

Zeiler³

et al. 2019

Proceedings 2019 Network and Distributed System Security Symposium

200

157

View full text Add to dashboard Cite

show abstract

“…The participants were employees of Fraunhofer IIS or the International Audio Laboratories Erlangen at the time of the experiments, and can be considered expert listeners in audio quality evaluations [35]. However, they were not trained in VR-specific listening tests, and they were naive about the specific conditions under examination here.…”

Section: Subjective Experiments Overviewmentioning

confidence: 99%

Head-Related Transfer Functions for Dynamic Listeners in Virtual Reality

2021

View full text Add to dashboard Cite

In dynamic virtual reality, visual cues and motor actions aid auditory perception. With multimodal integration and auditory adaptation effects, generic head-related transfer functions (HRTFs) may yield no significant disadvantage to individual HRTFs regarding accurate auditory perception. This study compares two individual HRTF sets against a generic HRTF set by way of objective analysis and two subjective experiments. First, auditory-model-based predictions examine the objective deviations in localization cues between the sets. Next, the HRTFs are compared in a static subjective (N=8) localization experiment. Finally, the localization accuracy, timbre, and overall quality of the HRTF sets are evaluated subjectively (N=12) in a six-degrees-of-freedom audio-visual virtual environment. The results show statistically significant objective deviations between the sets, but no perceived localization or overall quality differences in the dynamic virtual reality.

show abstract

“…However, this value can only give a rough approximation, and we are more interested in the judgment of human listeners. Therefore, we have conducted a Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) test [54], a commonly used test to assess the quality of audio stimuli. This enables us to understand the practical impact of the modifications on the attack in more detail and study how humans perceive our method.…”

Section: Listening Testsmentioning

confidence: 99%

Dompteur: Taming Audio Adversarial Examples

Eisenhofer¹,

Schönherr²,

Frank³

et al. 2021

Preprint

View full text Add to dashboard Cite

Adversarial examples seem to be inevitable. These specifically crafted inputs allow attackers to arbitrarily manipulate machine learning systems. Even worse, they often seem harmless to human observers. In our digital society, this poses a significant threat. For example, Automatic Speech Recognition (ASR) systems, which serve as hands-free interfaces to many kinds of systems, can be attacked with inputs incomprehensible for human listeners. The research community has unsuccessfully tried several approaches to tackle this problem.In this paper we propose a different perspective: We accept the presence of adversarial examples against ASR systems, but we require them to be perceivable by human listeners. By applying the principles of psychoacoustics, we can remove semantically irrelevant information from the ASR input and train a model that resembles human perception more closely. We implement our idea in a tool named DOMPTEUR 1 and demonstrate that our augmented system, in contrast to an unmodified baseline, successfully focuses on perceptible ranges of the input signal. This change forces adversarial examples into the audible range, while using minimal computational overhead and preserving benign performance. To evaluate our approach, we construct an adaptive attacker, which actively tries to avoid our augmentations and demonstrate that adversarial examples from this attacker remain clearly perceivable. Finally, we substantiate our claims by performing a hearing test with crowd-sourced human listeners.

show abstract

Audio quality evaluation by experienced and inexperienced listeners

Cited by 15 publications

References 0 publications

Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding

Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding

Head-Related Transfer Functions for Dynamic Listeners in Virtual Reality

Dompteur: Taming Audio Adversarial Examples

Contact Info

Product

Resources

About