Uncertainty-Guided Probabilistic Transformer for Complex Action Recognition

Guo, Hongji; Wang, Hanjing; Ji, Qiang

doi:10.1109/cvpr52688.2022.01942

Cited by 24 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further measure the uncertainty of the forecast, ref. [ 191 ] went beyond the deterministic transformer and created a probabilistic one by capturing the distribution of attention values.…”

Section: Research Methods and Taxonomymentioning

confidence: 99%

Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities

Morshed

Sultana

Alam

et al. 2023

Sensors

View full text Add to dashboard Cite

Human action recognition systems use data collected from a wide range of sensors to accurately identify and interpret human actions. One of the most challenging issues for computer vision is the automatic and precise identification of human activities. A significant increase in feature learning-based representations for action recognition has emerged in recent years, due to the widespread use of deep learning-based features. This study presents an in-depth analysis of human activity recognition that investigates recent developments in computer vision. Augmented reality, human–computer interaction, cybersecurity, home monitoring, and surveillance cameras are all examples of computer vision applications that often go in conjunction with human action detection. We give a taxonomy-based, rigorous study of human activity recognition techniques, discussing the best ways to acquire human action features, derived using RGB and depth data, as well as the latest research on deep learning and hand-crafted techniques. We also explain a generic architecture to recognize human actions in the real world and its current prominent research topic. At long last, we are able to offer some study analysis concepts and proposals for academics. In-depth researchers of human action recognition will find this review an effective tool.

show abstract

Section: Research Methods and Taxonomymentioning

confidence: 99%

Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities

Morshed

Sultana

Alam

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The variant features can predict the multiple possible labels (Yang et al 2023). The probabilistic prediction has been described in the Bayesian language model (Xue et al 2022;Zhang et al 2021) and the complex action recognition model (Guo, Wang, and Ji 2022). Some methods learn probabilistic prediction with the latent variables (Zheng et al 2022;Itkina et al 2020;Pambala, Dutta, and Biswas 2020;Zhang et al 2020).…”

Section: Related Workmentioning

confidence: 99%

Towards Understanding Future: Consistency Guided Probabilistic Modeling for Action Anticipation

Xie,

Shi,

et al. 2024

AAAI

View full text Add to dashboard Cite

Action anticipation aims to infer the action in the unobserved segment (future segment) with the observed segment (past segment). Existing methods focus on learning key past semantics to predict the future, but they do not model the temporal continuity between the past and the future. However, past actions are always highly uncertain in anticipating the unobserved future. The absence of temporal continuity smoothing in the video's past-and-future segments may result in an inconsistent anticipation of future action. In this work, we aim to smooth the global semantics changes in the past and future segments. We propose a Consistency-guided Probabilistic Model (CPM), which focuses on learning the globally temporal probabilistic consistency to inhibit the unexpected temporal consistency. The CPM is deployed on the Transformer architecture, which includes three modules of future semantics estimation, global semantics estimation, and global distribution estimation involving the learning of past-to-future semantics, past-and-future semantics, and semantically probabilistic distributions. To achieve the smoothness of temporal continuity, we follow the principle of variational analysis and describe two probabilistic distributions, i.e., a past-aware distribution and a global-aware distribution, which help to estimate the evidence lower bound of future anticipation. In this study, we maximize the evidence lower bound of future semantics by reducing the distribution distance between the above two distributions for model optimization. Extensive experiments demonstrate that the effectiveness of our method and the CPM achieves state-of-the-art performance on Epic-Kitchen100, Epic-Kitchen55, and EGTEA-GAZE.

show abstract

“…Further, Zhou et al 12 proposed GHRM, a graph-based method that models the high-order relation among sub-activities. By adopting the self-attention mechanism, Guo et al 13 introduced uncertainty-guided probabilistic Transformer. By constructing a majority model for low-uncertainty input and a minority model for high-uncertainty input, the model can well utilize the prediction uncertainty to improve both training and inference.…”

Section: Complex Human Event Recognitionmentioning

confidence: 99%

Video-based complex human event recognition with a probabilistic transformer

Guo,

Aved,

Roller

et al. 2023

Geospatial Informatics XIII

View full text Add to dashboard Cite

Complex human events are high-level human activities that are composed of a set of interacting primitive human actions over time. Complex human event recognition is important for many applications, including security surveillance, healthcare, sports and games. Complex human event recognition requires recognizing not only the constituent primitive actions but also, more importantly, their long range spatiotemporal interactions. To meet this requirement, we propose to exploit the self-attention mechanism in the Transformer to model and capture the long-range interactions among primitive actions. We further extend the conventional Transformer to a probabilistic Transformer in order to quantify the event recognition confidence and to detect anomaly events. Specifically, given a sequence of human 3D skeletons, the proposed model first performs primitive action localization and recognition. The recognized primitive human actions and their features are then fed into the probabilistic Transformer for complex human event recognition. By using a probabilistic attention score, the probabilistic Transformer can not only recognize complex events but also quantify its prediction uncertainty. Using the prediction uncertainty, we further propose to detect anomaly events in an unsupervised manner. We evaluate the proposed probabilistic Transformer on FineDiving dataset and Olympics Sports dataset for both complex event recognition and abnormal event detection. The dataset consists of complex events composed of primitive diving actions. The experimental results demonstrate the effectiveness and superiority of our method against baseline methods.

show abstract

Uncertainty-Guided Probabilistic Transformer for Complex Action Recognition

Cited by 24 publications

References 33 publications

Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities

Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities

Towards Understanding Future: Consistency Guided Probabilistic Modeling for Action Anticipation

Video-based complex human event recognition with a probabilistic transformer

Contact Info

Product

Resources

About