A Survey on Low-Latency DNN-Based Speech Enhancement

Drgas, Szymon

doi:10.3390/s23031380

Cited by 7 publications

(5 citation statements)

References 118 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…11,12 These approaches, however, are computationally expensive and not yet well-suited for low-power real-time applications. 13…”

Section: Introductionmentioning

confidence: 99%

“…11,12 These approaches, however, are computationally expensive and not yet well-suited for low-power real-time applications. 13 We recently developed a biologically oriented sound segregation algorithm 14 (BOSSA), which is designed to separate competing sounds based on differences in spatial location. Taking its inspiration from binaural auditory system, this algorithm requires only two input signals, and does not sacrifice spatial cues.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A brain-inspired algorithm improves “cocktail party” listening for individuals with hearing loss

Boyd,

Best,

Sen

2024

Preprint

View full text Add to dashboard Cite

Selective listening in competing-talker situations (restaurants, parties, etc.) is an extraordinarily difficult task for many people. For individuals with hearing loss, this difficulty can be so extreme that it seriously impedes communication and participation in daily life. Directional filtering is one of the only proven ways to improve speech understanding in competition, and most hearing devices now incorporate some kind of directional technology, although real-world benefits are modest, and many approaches fail in competing-talker situations. We recently developed a biologically inspired algorithm that is capable of very narrow spatial tuning and can isolate one talker from a mixture of talkers. The algorithm is based on a hierarchical network model of the auditory system, in which binaural sound inputs drive populations of neurons tuned to specific spatial locations and frequencies, and the spiking responses of neurons in the output layer are reconstructed into audible waveforms. Here we evaluated the algorithm in a group of adults with sensorineural hearing loss, using a challenging competing-talker task. The biologically inspired algorithm led to robust intelligibility gains under conditions in which a standard beamforming approach failed. The results provide compelling support for the potential benefits of biologically inspired algorithms for assisting individuals with hearing loss in cocktail party situations.

show abstract

“…11,12 These approaches, however, are computationally expensive and not yet well-suited for low-power real-time applications. 13…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A brain-inspired algorithm improves “cocktail party” listening for individuals with hearing loss

Boyd,

Best,

Sen

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Noise reduction can also be regarded as increasing speech intelligibility [2,3]. There has been extensive recent interest in the emerging area of machine learning-based speech enhancement techniques [4][5][6][7]. A particularly important application of speech intelligibility is its use in signal processors for cochlear implants (CIs) used by hearing-impaired listeners [8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Experimental Investigation of Acoustic Features to Optimize Intelligibility in Cochlear Implants

Henry,

Parsi,

Glavin

et al. 2023

Sensors

View full text Add to dashboard Cite

Although cochlear implants work well for people with hearing impairment in quiet conditions, it is well-known that they are not as effective in noisy environments. Noise reduction algorithms based on machine learning allied with appropriate speech features can be used to address this problem. The purpose of this study is to investigate the importance of acoustic features in such algorithms. Acoustic features are extracted from speech and noise mixtures and used in conjunction with the ideal binary mask to train a deep neural network to estimate masks for speech synthesis to produce enhanced speech. The intelligibility of this speech is objectively measured using metrics such as Short-time Objective Intelligibility (STOI), Hit Rate minus False Alarm Rate (HIT-FA) and Normalized Covariance Measure (NCM) for both simulated normal-hearing and hearing-impaired scenarios. A wide range of existing features is experimentally evaluated, including features that have not been traditionally applied in this application. The results demonstrate that frequency domain features perform best. In particular, Gammatone features performed best for normal hearing over a range of signal-to-noise ratios and noise types (STOI = 0.7826). Mel spectrogram features exhibited the best overall performance for hearing impairment (NCM = 0.7314). There is a stronger correlation between STOI and NCM than HIT-FA and NCM, suggesting that the former is a better predictor of intelligibility for hearing-impaired listeners. The results of this study may be useful in the design of adaptive intelligibility enhancement systems for cochlear implants based on both the noise level and the nature of the noise (stationary or non-stationary).

show abstract

“…The low-computational cost is a property desired in all systems, but most crucial for on-device deployment on smartphones or hearing aids. These translate into algorithmic latency and hardware latency that both add up to the overall processing latency [10,11]. The implementation aspects of speech enhancement are important parts of current research.…”

Section: Introductionmentioning

confidence: 99%

“…The frame is predicted based on current and previous inputs and overlap-added with previous predictions for final enhanced speech signal generation. The recent challenges that focus on real-time speech enhancement, such as Interspeech 2020 Deep Noise Suppression (DNS) challenge [16] and Clarity [17], put tight requirements on the maximum processing time and allowed look-ahead limited to respectively 40 ms and 5 ms. A comprehensive survey on recent advancements in low-latency speech enhancement can be found in [11].…”

Section: Introductionmentioning

confidence: 99%

Causal Signal-Based DCCRN with Overlapped-Frame Prediction for Online Speech Enhancement

Bartolewska¹,

Kacprzak²,

Kowalczyk³

2023

Interspeech 2023

View full text Add to dashboard Cite

The aim of speech enhancement is to improve speech signal quality and intelligibility from a noisy microphone signal. In many applications, it is crucial to enable processing with small computational complexity and minimal requirements regarding access to future signal samples (look-ahead). This paper presents signal-based causal DCCRN that improves online single-channel speech enhancement by reducing the required look-ahead and the number of network parameters. The proposed modifications include complex filtering of the signal, application of overlapped-frame prediction, causal convolutions and deconvolutions, and modification of the loss function. Results of performed experiments indicate that the proposed model with overlapped signal prediction and additional adjustments, achieves similar or better performance than the original DC-CRN in terms of various speech enhancement metrics, while it reduces the latency and network parameter number by around 30%.

show abstract

A Survey on Low-Latency DNN-Based Speech Enhancement

Cited by 7 publications

References 118 publications

A brain-inspired algorithm improves “cocktail party” listening for individuals with hearing loss

A brain-inspired algorithm improves “cocktail party” listening for individuals with hearing loss

Experimental Investigation of Acoustic Features to Optimize Intelligibility in Cochlear Implants

Causal Signal-Based DCCRN with Overlapped-Frame Prediction for Online Speech Enhancement

Contact Info

Product

Resources

About