Dynamic selective auditory attention detection using RNN and reinforcement learning

Geravanchizadeh, Masoud; Roushan, Hossein

doi:10.1101/2021.02.18.431748

Cited by 4 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In video abnormal behavior detection, the deep learning method has made a lot of contributions. For example, recurrent neural network (RNN) [8], convolutional neural networks (CNN) [9], and LSTM network [10] are all applied to video abnormal behavior detection. Autoencoder (AE) can detect abnormal behavior by using reconstruction error information.…”

Section: Related Workmentioning

confidence: 99%

Intelligent Transport Surveillance Memory Enhanced Method for Detection of Abnormal Behavior in Video

Zhang

2022

Journal of Advanced Transportation

View full text Add to dashboard Cite

The purpose is to build a better intelligent transport platform and improve the performance of surveillance video abnormal behavior detection systems under rapid progress of science and technology, to process large-scale traffic surveillance video data. Autoencoder (AE) can detect abnormal behavior by using reconstruction error information. However, it cannot decode some abnormal codes well, so an AE based on memory needs improvement. The objective of this research is to propose a model where abnormal surveillance video can be handled. Therefore, a self-coding method based on memory enhancement is proposed. The steps are as follows: different abnormal behavior detection system algorithms are analyzed at first. The characteristics of three different methods, namely, the original autoencoder (AE), recurrent neural network, and convolutional neural network, are compared. Then, a memory module is proposed to enhance the automatic encoder to reduce the reconstruction error of normal samples and increase the reconstruction error of abnormal samples. The effect image is obtained by Laplace transform and convolution for the image with low definition, and the image with noise is processed by guided filtering. Finally, different methods are used for experimental comparison. Experiments show that, on the dataset Avenue, the frame-level result of the method proposed is about 2% higher than that of the optimal ConvLSTM in the comparison method; on the Ped1 and Ped2 datasets, it is also about 3% higher than ConvLSTM. The comparison of different methods shows that the effect of the method proposed is the best. The self-coding traffic surveillance video abnormal behavior detection system based on memory enhancement is designed with a modular structure and it uses the self-coding method based on memory enhancement. The effectiveness of the proposed method in the real scene is verified by comparing the performance of different methods in the same data set (Xia and Li, 2021).

show abstract

Section: Related Workmentioning

confidence: 99%

Intelligent Transport Surveillance Memory Enhanced Method for Detection of Abnormal Behavior in Video

Zhang

2022

Journal of Advanced Transportation

View full text Add to dashboard Cite

show abstract

“…One option is to train a model that treats the time varying pupil and EEG measures of effort as additional physiological channels that can be concatenated to the EEG data. In this case, nonlinear models such as a recurrent neural network, could leverage its capacity to retain a running memory to merge various physiological measures that operate on different time scales to produce an attended prediction (Geravanchizadeh & Roushan, 2021). A state‐modeling approach could also be used to update the attended talker state when an marker of effortful listening has taken place (Miran et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort

Haro

Rao

Quatieria

et al. 2022

Eur J of Neuroscience

View full text Add to dashboard Cite

Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control.We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial-audio cues. A least-squares, electroencephalography (EEG)-based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5-s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5-s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [p ¼ 0:034] and minimum parietal alpha power measure [p ¼ 0:016]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization (p ¼ 0:0428). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech-based features.

show abstract

“…A recurrent neural network (RNN) has a recurrent hidden state with an activation at each time that depends on the one from the previous time. RNN based algorithms have been developed in time series forecasting applications 37 – 39 . Due to infinite lookback windows, RNNs are vulnerable to long-range dependencies in the data, and the gradients tend to either vanish or explode 40 .…”

Section: Methodsmentioning

confidence: 99%

Neural network ensemble model for prediction of erythrocyte sedimentation rate (ESR) using partial least squares regression

Lee

Hong

Song

et al. 2022

Sci Rep

View full text Add to dashboard Cite

The erythrocyte sedimentation rate (ESR) is a non-specific blood test for determining inflammatory conditions. However, the long measurement time (60 min) to obtain ESR is an obstacle for a prompt evaluation. In this study, to reduce the measurement time of ESR, deep neural networks (DNNs) were applied to the sedimentation tendency of blood samples. DNNs using multilayer perceptron (MLP), long short-term memory (LSTM), and gated recurrent unit (GRU) were assessed and compared to determine a suitable length of time for the input sequence. To avoid overfitting, a stacking ensemble learning was adopted, which combines multiple models by using a meta model. Four meta models were compared: mean, median, least absolute shrinkage and selection operator, and partial least squares regression (PLSR) schemes. From the empirical results, LSTM and GRU models have better prediction than MLP over sequence lengths of 5 to 20 min. The decrease in $$\overline{\mathrm{MAPE} }$$ MAPE ¯ and $$\overline{\mathrm{RMSE} }$$ RMSE ¯ of GRU and LSTM was attenuated after a sequence length of 15 min, so the input sequence length is determined as 15 min. In terms of the meta model, the statistical comparison suggests that GRU combined with PLSR (GRU–PLSR) is the best case. Then, the GRU–PLSR was tested for prediction of ESR data obtained from periodontitis patients to check its applicability to a specific disease. The Bland–Altman plot shows acceptable agreement between measured and predicted ESR values. Based on the results, the GRU–PLSR can predict ESR with improved performance within 15 min and has potential applicability to ESR data with inflammatory and non-inflammatory conditions.

show abstract

Dynamic selective auditory attention detection using RNN and reinforcement learning

Cited by 4 publications

References 47 publications

Intelligent Transport Surveillance Memory Enhanced Method for Detection of Abnormal Behavior in Video

Intelligent Transport Surveillance Memory Enhanced Method for Detection of Abnormal Behavior in Video

EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort

Neural network ensemble model for prediction of erythrocyte sedimentation rate (ESR) using partial least squares regression

Contact Info

Product

Resources

About