Driver confusion status detection using recurrent neural networks

Hori, Chiori; Watanabe, Shinji; Hori, Takaaki; Harsham, Bret; Hershey, JohnR.; Koji, Yusuke; Fujii, Yoichi; Furumoto, Yuki

doi:10.1109/icme.2016.7552966

Cited by 11 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most repeated AUs of frames sequence (belong to the same video clip) are selected depending on the presence or absence of each AU in the frames sequence of the video clips. The proposed system recognizes 18 facial AUs (AU 1, 2, 4, 5,6,7,9,10,12,14,15,17,20,23,25,26,28, and 45) some of these AUs have no effect on confusion detection processes. Based on the result of the proposed AUs detection system using collected datasets (for confused and not confused responses) show that some AUs not affected during the confused and not confused response approximately remain in the same state during all interviews.…”

Section: B Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Detection of confusion behavior using a facial expression based on different classification algorithms

Yasser¹,

Bassam²,

Abbas³

2021

ETJ

View full text Add to dashboard Cite

Confusion detection systems (CDSs) that need Noninvasive, mobile, and cost-effective methods use facial expressions as a technique to detect confusion. In previous works, the technology that the system used represents a major gap between this proposed CDS and other systems. This CDS depends on the Facial Action Coding System (FACS) that is used to extract facial features. The FACS shows the motion of the facial muscles represented by Action Units (AUs); the movement is represented with one facial muscle or more. Seven AUs are used as possible markers for detecting confusion that has been implemented in the form of a single vector of facial action; the AUs that have been used in this work are AUs 4, 5, 6, 7, 10, 12, and 23. The database used to calculate the performance of the proposed CDS is gathered from 120 participants (91males, 29 females), between the ages of 18-45. Four types of classification algorithms are used as individuals; these classifiers are (VG-RAM), (SVM), Logistic Regression and Quadratic Discriminant classifiers. The best success rate was found when using Logistic Regression and Quadratic Discriminant. This work introduces different classification techniques to detect confusion by collecting an actual database that can be used to evaluate the performance for every CDS employing facial expressions and selecting appropriate facial features.

show abstract

Section: B Feature Extractionmentioning

confidence: 99%

“…-- [22] driver's behavior and the traffic conditions 11 -- [2] EEG data 10 73.3% [23] EEG data 17 71.3% [10] Action units AU25, 26 and 27.…”

mentioning

confidence: 99%

Detection of confusion behavior using a facial expression based on different classification algorithms

Yasser¹,

Bassam²,

Abbas³

2021

ETJ

View full text Add to dashboard Cite

show abstract

“…In a different driving simulator study, bagging ensembles composed of Long Short-Term Memory (LSTM) neural networks have been used to successfully predict lane departures and lateral driving speed [2]. Other recurrent neural network (RNN) architectures have employed successfully in assessments of driving performance in simulators that also track the driver's on-screen gaze [13,22]. Successes in these similar driver classification tasks motivated our decision to employ a deep neural network incorporating an LSTM layer in some of our experiments.…”

Section: Introductionmentioning

confidence: 99%

Identifying On-road Scenarios Predictive of ADHD usingDriving Simulator Time Series Data

Grethlein¹,

Sladek²,

Ontañón³

2021

Preprint

View full text Add to dashboard Cite

In this paper we introduce a novel algorithm called Iterative Section Reduction (ISR) to automatically identify sub-intervals of spatiotemporal time series that are predictive of a target classification task. Specifically, using data collected from a driving simulator study, we identify which spatial regions (dubbed sections) along the simulated routes tend to manifest driving behaviors that are predictive of the presence of Attention Deficit Hyperactivity Disorder (ADHD). Identifying these sections is important for two main reasons: (1) to improve predictive accuracy of the trained models by filtering out non-predictive time series sub-intervals, and (2) to gain insights into which on-road scenarios (dubbed events) elicit distinctly different driving behaviors from patients undergoing treatment for ADHD versus those that are not. Our experimental results show both improved performance over prior efforts (+10% accuracy) and good alignment between the predictive sections identified and scripted on-road events in the simulator (negotiating turns and curves).

show abstract

“…Recent advances in neural network models have led a wide application of the RNN for modeling sequential data [5]- [10]. RNN has been applied in the field of driving behavior studies for detection of driver confusion status with Long Short Term Memory (LSTM), a variant of the RNN [11], and prediction of driver action with the bidirectional recurrent neural network [12]. However, RNN and its variants including LSTM [13] and the Gated Recurrent Unit (GRU) [14] do not consider missing values nor intrinsic noise in the data which are unavoidable in data from sensors in noisy, naturalistic driving situations.…”

Section: Introductionmentioning

confidence: 99%

Denoising Recurrent Neural Networks for Classifying Crash-Related Events

Park

Seonwoo

Kim

et al. 2020

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

With detailed sensor and visual data from automobiles, a data-driven model can learn to classify crash-related events during a drive. We propose a neural network model accepting time-series vehicle sensor data and forward-facing videos as input for learning classification of crash-related events and varying types of such events. To elaborate, a novel recurrent neural network structure is introduced, namely, denoising gated recurrent unit with decay, in order to deal with time-series automobile sensor data with missing value and noises. Our model detects crash and near-crash events based on a large set of time-series data collected from naturalistic driving behavior. Furthermore, the model classifies those events involving pedestrians, a vehicle in front, or a vehicle on either side. The effectiveness of our model is evaluated with more than two thousand 30-s clips from naturalistic driving behavior data. The results show that the model, including sensory encoder with denoising gated recurrent unit with decay, visual encoder, and attention mechanism, outperforms gated recurrent unit with decay, gated CNN, and other baselines not only in event classification and but also in event-type classification.

show abstract

Driver confusion status detection using recurrent neural networks

Cited by 11 publications

References 7 publications

Detection of confusion behavior using a facial expression based on different classification algorithms

Detection of confusion behavior using a facial expression based on different classification algorithms

Identifying On-road Scenarios Predictive of ADHD usingDriving Simulator Time Series Data

Denoising Recurrent Neural Networks for Classifying Crash-Related Events

Contact Info

Product

Resources

About