State-of-the-Art Analysis of Deep Learning-Based Monaural Speech Source Separation Techniques

Target sound separation (TSS) aims to separate specific sounds of interest, like a speech or a musical instrument, from complex acoustic environments with multiple overlapping sounds. In realistic scenarios, the important sounds that we want to hear can differ depending on transitions in the surrounding acoustic scene. This study addresses the problem of acoustic-scene-aware TSS, which separates predefined sets of target sounds considered significant for the current acoustic environment. Predefined sets of target sounds were determined beforehand based on the expected acoustic scenes. For example, the sound of a bicycle bell is predefined as the target sound in a park scene and separated from a mixture of various sounds. As a solution, we propose a novel approach called Acoustic-SCene-Aware Target sound separation with sound Embedding Refinement (SCATER). It refines pre-trained sound embeddings into acoustic-sceneaware representations to guide the separation of specific target sounds based on the surrounding scene. SCATER adopts a multiple instance learning-based acoustic scene classification system for rapid response to scene changes. The refined sound embeddings serve as cues for the TSS model, enabling the separation of different target sounds across various acoustic scenes. Experimental results demonstrate the superiority of SCATER over an approach that combines sound separation and scene classification separately.

Section: Introductionmentioning

confidence: 99%

Acoustic-Scene-Aware Target Sound Separation With Sound Embedding Refinement

Kim,

Chang

2024

“…The primary objective is to differentiate and recover the individual speeches by leveraging the available perceptual data, i.e., BSS of audio files [13]. As evident from the observations, the development of a robust framework capable of effectively separating speech and music has the potential to yield substantial benefits across numerous lucrative applications [14], [15].…”

Section: Introductionmentioning

confidence: 99%

“…One prominent signal processing technique in this context is BSS. Various BSS algorithms and architectures are investigated, considering their potential to improve interference management, channel estimation, beamforming, and resource allocation in these next-generation wireless networks [4], [15], [16], [17], [18], [19], [20].…”

Section: Introductionmentioning

confidence: 99%

A Robust Hybrid Neural Network Architecture for Blind Source Separation of Speech Signals Exploiting Deep Learning

Ansari,

Alnajjar,

Khater

et al. 2023

In the contemporary era, blind source separation has emerged as a highly appealing and significant research topic within the field of signal processing. The imperative for the integration of blind source separation techniques within the context of beyond fifth-generation and sixth-generation networks arises from the increasing demand for reliable and efficient communication systems that can effectively handle the challenges posed by high-density networks, dynamic interference environments, and the coexistence of diverse signal sources, thereby enabling enhanced signal extraction and separation for improved system performance. Particularly, audio processing presents a critical domain where the challenge lies in effectively handling files containing a mixture of human speech, silence, and music. Addressing this challenge, speech separation systems can be regarded as a specialized form of human speech recognition or audio signal classification systems that are leveraged to separate, identify, or delineate segments of audio signals encompassing human speech. In various applications such as volume reduction, quality enhancement, detection, and identification, the need arises to separate human speech by eliminating silence, music, or environmental noise from the audio signals. Consequently, the development of robust methods for accurate and efficient speech separation holds paramount importance in optimizing audio signal processing tasks. This study proposes a novel three-way neural network architecture that incorporates transfer learning, a pre-trained dual-path recurrent neural network, and a transformer. In addition to learning the time series associated with audio signals, this network possesses the unique capability of direct context-awareness for modeling the speech sequence within the transformer framework. A comprehensive array of simulations is meticulously conducted to evaluate the performance of the proposed model, which is benchmarked with seven prominent state-of-the-art deep learning-based architectures. The results obtained from these evaluations demonstrate notable advancements in multiple objective metrics. Specifically, our proposed solution showcases an average improvement of 4.60% in terms of short-time objective intelligibility, 14.84% in source-to-distortion ratio, and 9.87% in scale-invariant signal-to-noise ratio. These extraordinary advancements surpass those achieved by the nearest rival, namely the dual-path recurrent neural network time-domain audio separation network, firmly establishing the superiority of our proposed model's performance.

“…Conventional deep-learning-based speech separation technologies are reviewed in [1], [2]. [3] describes the comprehensive outlook of the state-of-the-art technologies for speech separation and presents a detailed performance comparison. However, these approaches have not yet reached the levels of separation capability of auditory systems.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Sparse Approximation for Monaural Overlapped Speech Separation From Auditory Perspective

Sekiguchi,

Narusue,

Morikawa

2023

Neuroscience suggests that the sparse behavior of a neural population underlies the mechanisms of the auditory system for monaural overlapped speech separation. This study investigates leveraging sparse approximation to improve speech separation in a conventional deep learning algorithm. We develop a combined model that embeds a sparse approximation algorithm, a multilayered iterative soft thresholding algorithm (ML-ISTA), into a conventional time-domain-based speech separation algorithm, Conv-TasNet. Adopting ML-ISTA is a crucial enabler for the embedding process and helps avoid solving a bi-level optimization problem comprising sparse approximation and speech separation. ML-ISTA performs sparse approximation through forward calculations, thereby eliminating the optimization of sparse approximation. The combined model is trained with WSJ0-2mix, the Wall Street Journal English corpus for two-speaker mixed speech without noisy or reverberant interference, to clarify the proposed method's performance. The model demonstrates that sparse approximation improves separation performance regardless of the approximation setting. The peak performance of the model exceeds that of Conv-TasNet by 1.1% to 4.7% in four speech quality criteria. Moreover, sparse approximation accelerates the combined model performance gain at the early stages of learning relative to Conv-TasNet. The primary novelty of the study is embedding the sparse approximation algorithm, ML-ISTA, into a deep-learning-based speech separation framework and the experimental proof of improved separation performance in the proposed algorithm.