Multi-view region-adaptive multi-temporal DMM and RGB action recognition

Al-Faris, Mahmoud; Chiverton, John; Yang, Yanyan; Ndzi, David

doi:10.1007/s10044-020-00886-5

Cited by 11 publications

(5 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DMMs are extended to include multiple appearance information, similar to e.g., [31]. The important features in the 3D depth and appearance streams are then learned by a series of fine-tuned, pre-trained 3D Convolutional Neural Networks (CNN)s. More details on this action recognition technique can be found in [32].…”

Section: Methodsmentioning

confidence: 99%

Vision Based Dynamic Thermal Comfort Control Using Fuzzy Logic and Deep Learning

Al-Faris¹,

Chiverton

Ndzi

et al. 2021

Applied Sciences

Self Cite

View full text Add to dashboard Cite

A wide range of techniques exist to help control the thermal comfort of an occupant in indoor environments. A novel technique is presented here to adaptively estimate the occupant’s metabolic rate. This is performed by utilising occupant’s actions using computer vision system to identify the activity of an occupant. Recognized actions are then translated into metabolic rates. The widely used Predicted Mean Vote (PMV) thermal comfort index is computed using the adaptivey estimated metabolic rate value. The PMV is then used as an input to a fuzzy control system. The performance of the proposed system is evaluated using simulations of various activities. The integration of PMV thermal comfort index and action recognition system gives the opportunity to adaptively control occupant’s thermal comfort without the need to attach a sensor on an occupant all the time. The obtained results are compared with the results for the case of using one or two fixed metabolic rates. The included results appear to show improved performance, even in the presence of errors in the action recognition system.

show abstract

Section: Methodsmentioning

confidence: 99%

Vision Based Dynamic Thermal Comfort Control Using Fuzzy Logic and Deep Learning

Al-Faris¹,

Chiverton

Ndzi

et al. 2021

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…Some approaches try to predict pose from RGB data and use it in action prediction. In [231], a framework is provided for hierarchical region-adaptive multi-time resolution depth motion map (RAMDMM) and multi-time resolution RGB action recognition system. The suggested approach presents a feature representation method for RGB-D data that allows multi-view and multi-temporal action recognition.…”

Section: Rgb and Depthmentioning

confidence: 99%

Transformers in Action Recognition: A Review on Temporal Modeling

Shabaninia¹,

Nezamabadi‐pour²,

Shafizadegan³

2023

Preprint

View full text Add to dashboard Cite

In vision-based action recognition, spatio-temporal features from different modalities are used for recognizing activities. Temporal modeling is a long challenge of action recognition. However, there are limited methods such as pre-computed motion features, three-dimensional (3D) filters, and recurrent neural networks (RNN) for modeling motion information in deep-based approaches. Recently, transformers' success in modeling long-range dependencies in natural language processing (NLP) tasks has gotten great attention from other domains; including speech, image, and video, to rely entirely on self-attention without using sequence-aligned RNNs or convolutions. Although the application of transformers to action recognition is relatively new, the amount of research proposed on this topic within the last few years is astounding. This paper especially reviews recent progress in deep learning methods for modeling temporal variations. It focuses on action recognition methods that use transformers for temporal modeling, discussing their main features, used modalities, and identifying opportunities and challenges for future research.

show abstract

“…In their method feature vectors are mined from MHIs and SHIs by the GLAC feature descriptor. Al-Faris et al [ 37 ] presented the construction of a multi-view region-adaptive multi-resolution-in-time depth motion map (MV-RAMDMM). They trained several scenes and time resolutions of the region-adaptive depth motion maps (RA-DMMs) by multi-stream 3D convolutional neural networks (CNNs).…”

Section: Related Workmentioning

confidence: 99%

Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences

Bulbul¹,

Tabussum²,

Ali

et al. 2021

Sensors

View full text Add to dashboard Cite

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to L2-regularized Collaborative Representation Classifier (L2-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of L2-CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.

show abstract

Multi-view region-adaptive multi-temporal DMM and RGB action recognition

Cited by 11 publications

References 66 publications

Vision Based Dynamic Thermal Comfort Control Using Fuzzy Logic and Deep Learning

Vision Based Dynamic Thermal Comfort Control Using Fuzzy Logic and Deep Learning

Transformers in Action Recognition: A Review on Temporal Modeling

Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences

Contact Info

Product

Resources

About