A survey on deep reinforcement learning for audio-based applications

Following the success of Natural Language Processing (NLP) transformers pretrained via self-supervised learning, similar models have been proposed recently for speech processing such as Wav2Vec2, HuBERT and UniSpeech-SAT. An interesting yet unexplored area of application of these models is Spoken Dialogue Systems, where the users' audio signals are typically just mapped to word-level features derived from an Automatic Speech Recogniser (ASR), and then processed using NLP techniques to generate system responses. This paper reports a comprehensive comparison of dialogue policies trained using ASR-based transcriptions and extended with the aforementioned audio processing transformers in the DSTC2 task. Whilst our dialogue policies are trained with supervised and policy-based deep reinforcement learning, they are assessed using both automatic task completion metrics and a human evaluation. Our results reveal that using audio embeddings is more beneficial than detrimental in most of our trained dialogue policies, and that the benefits are stronger for supervised learning than reinforcement learning.

show abstract

“…This is new in the area of dialogue policy learning. We refer the readers to [42] for a more-in-depth analysis of RL in spoken dialogue systems.…”

Section: Related Workmentioning

confidence: 99%

Audio Embedding-Aware Dialogue Policy Learning

Zorrilla

Torres

Cuayáhuitl

2023

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Deep learning algorithms have been used in various fields [ 1 , 2 , 3 ]. In the field of computer vision, deep learning algorithms are used to perform multi-modal learning to obtain useful information from images and texts, images and speech, and images and sensor signals [ 1 ].…”

Section: Introductionmentioning

confidence: 99%

“…In the field of computer vision, deep learning algorithms are used to perform multi-modal learning to obtain useful information from images and texts, images and speech, and images and sensor signals [ 1 ]. In the field of audio analysis, deep learning algorithms are used for automatic speech recognition, audio enhancement, and audio generation [ 2 ]. In the field of natural language processing, deep learning algorithms are used to perform sentiment analysis and machine translation [ 3 ].…”

Section: Introductionmentioning

confidence: 99%

Data Valuation Algorithm for Inertial Measurement Unit-Based Human Activity Recognition

Kim

Lee

2022

Sensors

View full text Add to dashboard Cite

This paper proposes a data valuation algorithm for inertial measurement unit-based human activity recognition (IMU-based HAR) data based on meta reinforcement learning. Unlike previous studies that received feature-level input, the algorithm in this study added a feature extraction structure to the data valuation algorithm, and it can receive raw-level inputs and achieve excellent performance. As IMU-based HAR data are multivariate time-series data, the proposed algorithm incorporates an architecture capable of extracting both local and global features by inserting a transformer encoder after the one-dimensional convolutional neural network (1D-CNN) backbone in the data value estimator. In addition, the 1D-CNN-based stacking ensemble structure, which exhibits excellent efficiency and performance on IMU-based HAR data, is used as a predictor to supervise model training. The Berg balance scale (BBS) IMU-based HAR dataset and the public datasets, UCI-HAR, WISDM, and PAMAP2, are used for performance evaluation in this study. The valuation performance of the proposed algorithm is observed to be excellent on IMU-based HAR data. The rate of discovering corrupted data is higher than 96% on all datasets. In addition, classification performance is confirmed to be improved by the suppression of discovery of low-value data.

show abstract

“…Deep RL has mostly been applied in applications like as gaming [25], [38], [39], recommendation systems [40], and robotics [41]. With the popularity of deep RL, the use of RL for speech-based applications has increased [42].…”

Section: Introductionmentioning

confidence: 99%

Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning

Rajapakshe¹,

Rana²,

Khalifa³

2022

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning has been a popular training paradigm as deep learning has gained popularity in the field of machine learning. Domain adaptation allows us to transfer knowledge learnt by a model across domains after a phase of training. The inability to adapt an existing model to a real-world domain is one of the shortcomings of current domain adaptation algorithms. We present a deep reinforcement learning-based strategy for adapting a pre-trained model to a newer domain while interacting with the environment and collecting continual feedback. This method was used on the Speech Emotion Recognition task, which included both cross-corpus and cross-language domain adaption schema. Furthermore, it demonstrates that in a real-world environment, our approach outperforms the supervised learning strategy by 42% and 20% in cross-corpus and cross-language schema, respectively.

show abstract

A survey on deep reinforcement learning for audio-based applications

Cited by 51 publications

References 361 publications

Audio Embedding-Aware Dialogue Policy Learning

Audio Embedding-Aware Dialogue Policy Learning

Data Valuation Algorithm for Inertial Measurement Unit-Based Human Activity Recognition

Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning

Contact Info

Product

Resources

About