Open Set Driver Activity Recognition

Roitberg, Alina; Ma, Chaoxiang; Haurilet, Monica; Stiefelhagen, Rainer

doi:10.1109/iv47402.2020.9304605

Cited by 19 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We cannot capture all possible driver behaviors in the training data, then the model, developed for closed set recognition, will be quickly exposed to uncertain situations and put the driver in disturbing and potentially dangerous situations. In [178], the task of open set driver activity recognition is introduced to address this issue.…”

Section: Applications Of Open Set Recognitionmentioning

confidence: 99%

A Survey on Open Set Recognition

Mahdavi,

Carvalho

2021

Preprint

View full text Add to dashboard Cite

Recognition (OSR) is about dealing with unknown situations that were not learned by the models during training. In this paper, we provide a survey of existing works about OSR and distinguish their respective advantages and disadvantages to help out new researchers interested in the subject. The categorization of OSR models is provided along with an extensive summary of recent progress. Additionally, the relationships between OSR and its related tasks including multi-class classification and novelty detection are analyzed. It is concluded that OSR can appropriately deal with unknown instances in the real-world where capturing all possible classes in the training data is not practical. Lastly, applications of OSR are highlighted and some new directions for future research topics are suggested.

show abstract

Section: Applications Of Open Set Recognitionmentioning

confidence: 99%

A Survey on Open Set Recognition

Mahdavi,

Carvalho

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Several works have studied uncertainty estimation in the context of novelty detection [10], [15], [25], [27]. A Bayesian approach has been used in a framework for recognizing activity classes which were not present during training [25].…”

Section: B Identifying Model Misclassificationsmentioning

confidence: 99%

Uncertainty-sensitive Activity Recognition: A Reliability Benchmark and the CARING Models

Roitberg

Haurilet

Martínez

et al. 2021

2020 25th International Conference on Pattern Recognition (ICPR)

Self Cite

View full text Add to dashboard Cite

Beyond assigning the correct class, an activity recognition model should also be able to determine, how certain it is in its predictions. We present the first study of how well the confidence values of modern action recognition architectures indeed reflect the probability of the correct outcome and propose a learning-based approach for improving it. First, we extend two popular action recognition datasets with a reliability benchmark in form of the expected calibration error and reliability diagrams. Since our evaluation highlights that confidence values of standard action recognition architectures do not represent the uncertainty well, we introduce a new approach which learns to transform the model output into realistic confidence estimates through an additional calibration network. The main idea of our Calibrated Action Recognition with Input Guidance (CARING) model is to learn an optimal scaling parameter depending on the video representation. We compare our model with the native action recognition networks and the temperature scaling approach -a wide spread calibration method utilized in image classification. While temperature scaling alone drastically improves the reliability of the confidence values, our CARING method consistently leads to the best uncertainty estimates in all benchmark settings.

show abstract

“…There has also been significant research that builds upon cues from driver gaze, hand and foot analysis for making higher level inferences such as driver activity recognition [33]- [37], driver intent or behavior prediction [38]- [44] and driver distraction detection [45]- [50]. Of particular interest is recent work [4], where the authors map driver gaze, hand and foot activity to the driver's observable take-over readiness index (ORI) obtained via subjective ratings assigned by multiple human observers.…”

Section: A Vision Based Driver Behavior Analysismentioning

confidence: 99%

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

Rangesh¹,

Deo²,

Greer³

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding occupant-vehicle interactions by modeling control transitions is important to ensure safe approaches to passenger vehicle automation. Models which contain contextual, semantically meaningful representations of driver states can be used to determine the appropriate timing and conditions for transfer of control between driver and vehicle. However, such models rely on real-world control take-over data from drivers engaged in distracting activities, which is costly to collect. Here, we introduce a scheme for data augmentation for such a dataset. Using the augmented dataset, we develop and train take-over time (TOT) models that operate sequentially on mid and high-level features produced by computer vision algorithms operating on different driver-facing camera views, showing models trained on the augmented dataset to outperform the initial dataset. The demonstrated model features encode different aspects of the driver state, pertaining to the face, hands, foot and upper body of the driver. We perform ablative experiments on feature combinations as well as model architectures, showing that a TOT model supported by augmented data can be used to produce continuous estimates of take-over times without delay, suitable for complex real-world scenarios.

show abstract

Open Set Driver Activity Recognition

Cited by 19 publications

References 30 publications

A Survey on Open Set Recognition

A Survey on Open Set Recognition

Uncertainty-sensitive Activity Recognition: A Reliability Benchmark and the CARING Models

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

Contact Info

Product

Resources

About