Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition

Gong, Rong; Quillen, Carl; Sharma, Dushyant; Goderre, Andrew; Laínez, José Enrique García; Milanovic, Ljubomir

doi:10.21437/interspeech.2021-1190

Cited by 11 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We showed that self attention-based algorithms reach state-ofthe-art performance without requiring handcrafted feature extraction. Furthermore, the original self-attention channel combinator (SACC) [22] slightly outperforms MVDR beamformer without requiring to estimate speech signal statistics. The SACC model was also extended in the complex space to preserve all of the information in the STFT.…”

Section: Discussionmentioning

confidence: 99%

“…beamforming [15]). In order to automatically combine multiple channels, we apply the SACC method [22] as a feature extractor for the OSD task. This method is also extended in the complex space (cSACC) to preserve all of the STFT information during the weight-estimation procedure.…”

Section: Multichannel Feature Extractionmentioning

confidence: 99%

“…Since channels combination can lead to a large variation in the data, MVN is applied to the combined STFT for each frequency bin, and converted to the log-mel scale using F = 64 filters as in [22]. The process of the SACC algorithm is presented in Figure 1.…”

Section: Self-attention Channel Combinatormentioning

confidence: 99%

“…The SACC combination-weights can be visualized as a function of the speaker direction of arrival (DOA) [22]. However, since weights belong to the real space, the lack of information about the phase limits their interpretation.…”

Section: Self-attention Channel Combinatormentioning

confidence: 99%

“…speaker location, noise statistics). On neural approaches, Gong et al [22] proposed the Self Attention Channel Combinator (SACC) which learns how to combine channels using self-attention [23]. Combination-weights are learned on the magnitude of the multichannel Short-Time Fourier Transform (STFT) of the input signal.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Microphone Array Channel Combination Algorithms for Overlapped Speech Detection

Mariotte¹,

Larcher²,

Montrésor³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

Overlapped speech occurs when multiple speakers are simultaneously active. This may lead to severe performance degradation in automatic speech processing systems such as speaker diarization. Overlapped speech detection (OSD) aims at detecting time segments in which several speakers are simultaneously active. Recent deep neural network architectures have shown impressive results in the close-talk scenario. However, performance tends to deteriorate in the context of distant speech. Microphone arrays are often considered under these conditions to record signals including spatial information. This paper investigates the use of the self-attention channel combinator (SACC) system as a feature extractor for OSD. This model is also extended in the complex space (cSACC) to improve the interpretability of the approach. Results show that distant OSD performance with self-attentive models gets closer to the nearfield condition. A detailed analysis of the cSACC combinationweights is also conducted showing that the self-attention module focuses attention on the speakers' direction.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Multichannel Feature Extractionmentioning

confidence: 99%

Section: Self-attention Channel Combinatormentioning

confidence: 99%

Section: Self-attention Channel Combinatormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations