Detecting the locus of auditory attention based on the spectro-spatial-temporal analysis of EEG

Abstract

Objective. Auditory Attention Decoding (AAD) determines which speaker the listener is focusing on by analyzing his/her EEG. CNN was adopted to extract Spectro-Spatial-Feature (SSF) from short-time-interval of EEG to detect auditory spatial attention without stimuli. However, the following factors are not considered in SSF-CNN scheme. i) Single-band frequency analysis cannot represent the EEG pattern precisely. ii) The power cannot represent the EEG feature related to the dynamic patterns of the attended auditory stimulus. iii) The temporal feature of EEG representing the relationship between EEG and attended stimulus is not extracted. To solve these problems, SSF-CNN scheme was modified. Approach. i) Multiple-frequency bands, but not a single alpha frequency band, of EEG, were analyzed to represent the EEG pattern more precisely. ii) Differential Entropy (DE), but not power, was extracted from each frequency band to represent the disorder degree of EEG, which was related to the dynamic patterns of the attended auditory stimulus. iii) CNN and Convolutional-Long-Short-Term-Memory (ConvLSTM) were combined to extract spectro-spatial-temporal features from the 3-D descriptor sequence constructed based on the topographical activity maps of multiple-frequency bands. Main results. Experimental results on KUL, DTU, and PKU with 0.1s, 1s, 2s, and 5s decision windows demonstrated that: i) The proposed model outperformed SSF-CNN and state-of-the-art AAD models. Specifically, when the auditory stimulus was unavailable, AAD accuracy could be enhanced by at least 3:25%, 3:96%, and 5:08% on KUL, DTU, and PKU, respectively, compared with the baselines. And, on KUL, the longer decision window corresponded to lower enhancement, while on both DTU and PKU, the longer decision window corresponded to higher enhancement, except for two cases when decision window length was 2s on PKU or 5s on DTU. ii) Each modification contributed to the performance enhancement. Significance. DE feature, multi-band frequency analysis, and ConvLSTM-based temporal analysis help to enhance AAD accuracy.

Electroencephalogram-based Multiclass Auditory Attention Decoding of Attended Speaker Direction

“…A convolutional neural network (CNN) model [7] was first proposed to directly extract patterns from EEG signals, which achieves better performance than rule-based methods and is more robust to the change of decision window length. The spatial-temporal attention network (STAnet) [18] and the spectro-spatial-temporal convolutional recurrent network (CRN) [17] were proposed to further improve the directional focus decoding accuracy. The STAnet is designed to dynamically assign differentiated weights to EEG channels.…”

“…In this paper, we elaborate on our newly released database that consists of 15 alternative speaker directions and enhance the capability of neural networks to multiclass decoding of directional focus of attended speakers by integrating the learnable spatial mapping module into the CNN baseline model [7] and the state-of-the-art CRN model [17].…”

“…Subsequently, a manual inspection is conducted to identify and eliminate additional abnormal segments within the EEG signal. Before being fed into the DNN model, the EEG signals are split into four subbands through four bandpass filters: the delta band (1-4 Hz), theta band (4-8 Hz), alpha band (8-13 Hz), and beta band (13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) Hz). The dataset is available at https://ieee-dataport.org/documents/nju-auditory-attention-dec oding-dataset.…”

“…The rule-based methods, such as the filter bank common spatial patterns filter (FB-CSP) [16] and the Riemannian geometry-based classifier (RGC) [8], depend on second-order statistics to decode the attended direction. DNN approaches utilize the convolution [7], recurrent [17], and self-attention [18] structures to directly extract the attended direction from EEG signals. The FB-CSP utilizes optimized filters to maximize the energy contrast of filtered signals between different directional focus classes [16].…”

Attention-guided graph structure learning network for EEG-enabled auditory attention detection

“…Building on this motivation, Geirnaert et al have implemented a common spatial pattern (CSP)-based approach [8] and a Riemannian geometry-based approach [9] for auditory spatial attention detection. Moreover, deep learning techniques, especially convolutional neural networks (CNNs), have shown remarkable performance in EEG-based AAD tasks [10][11][12][13][14].…”