Learning a non-linear knowledge transfer model for cross-view action recognition

Rahmani, Hossein; Mian, Ajmal

doi:10.1109/cvpr.2015.7298860

Cited by 135 publications

(143 citation statements)

References 25 publications

Supporting

Mentioning

143

Contrasting

Order By: Relevance

“…At the same time, AUC values of the ROC curves help to understand and compare the ROC curves in a clearer way when they cross each other or nearly close to each other. [32] 12fps 16fps HOPC [20] 0.04fps 0.5fps HPM+TM [18] 22fps 25fps Ours 36fps 28 fps…”

Section: Ntu Rgb-d Human Activity Datasetmentioning

confidence: 99%

View-Invariant Deep Architecture for Human Action Recognition Using Two-Stream Motion and Shape Temporal Dynamics

Dhiman

Vishwakarma

2020

IEEE Trans. on Image Process.

130

View full text Add to dashboard Cite

Human action Recognition for unknown views is a challenging task. We propose a view-invariant deep human action recognition framework, which is a novel integration of two important action cues: motion and shape temporal dynamics (STD). The motion stream encapsulates the motion content of action as RGB Dynamic Images (RGB-DIs) which are processed by the fine-tuned InceptionV3 model. The STD stream learns longterm view-invariant shape dynamics of action using human pose model (HPM) based view-invariant features mined from structural similarity index matrix (SSIM) based key depth human pose frames. To predict the score of the test sample, three types of late fusion (maximum, average and product) techniques are applied on individual stream scores. To validate the performance of the proposed novel framework the experiments are performed using both cross subject and cross-view validation schemes on three publically available benchmarks-NUCLA multi-view dataset, UWA3D-II Activity dataset and NTU RGB-D Activity dataset. Our algorithm outperforms with existing state-of-the-arts significantly that is reported in terms of accuracy, receiver operating characteristic (ROC) curve and area under the curve (AUC).

show abstract

Section: Ntu Rgb-d Human Activity Datasetmentioning

confidence: 99%

View-Invariant Deep Architecture for Human Action Recognition Using Two-Stream Motion and Shape Temporal Dynamics

Dhiman

Vishwakarma

2020

IEEE Trans. on Image Process.

130

View full text Add to dashboard Cite

show abstract

“…4. Specifically, given K human joints with [175] Vector of Joints Conc Lowlv Hand Patsadu et al [176] Vector of Joints Conc Lowlv Hand Huang and Kitani [177] Cost Topology Stat Lowlv Hand Devanne et al [178] Motion Units Conc Manif Hand Wang et al [179] Motion Poselets BoW Body Dict Wei et al [180] Structural Prediction Conc Lowlv Hand Gupta et al [181] 3D Pose w/o Body Parts Conc Lowlv Hand Amor et al [182] Skeleton's Shape Conc Manif Hand Sheikh et al [183] Action Space Conc Lowlv Hand Yilma and Shah [184] Multiview Geometry Conc Lowlv Hand Gong et al [185] Structured Time Conc Manif Hand Rahmani and Mian [186] Knowledge Transfer BoW Lowlv Dict Munsell et al [187] Motion Biometrics Stat Lowlv Hand Lillo et al [188] Composable Activities BoW Lowlv Dict Wu et al [189] Watch-n-Patch BoW Lowlv Dict Gong and Medioni [190] Dynamic Manifolds BoW Manif Dict Han et al [191] Hierarchical Manifolds BoW Manif Dict Slama et al [192,193] Grassmann Manifolds BoW Manif Dict Devanne et al [194] Riemannian Manifolds Conc Manif Hand Huang et al [195] Shape Tracking Conc Lowlv Hand Devanne et al [196] Riemannian Manifolds Conc Manif Hand Zhu et al [197] RNN with LSTM Conc Lowlv Deep Chen et al [198] EnwMi Learning BoW Lowlv Dict Hussein et al [199] Covariance of 3D Joints Stat Lowlv Hand Shahroudy et al [200] MMMP BoW Body Unsup Jung and Hong [201] Elementary Moving Pose BoW Lowlv Dict Evangelidis et al [202] Skeletal Quad Conc Lowlv Hand Azary and Savakis [203] Grassmann Manifolds Conc Manif Hand Barnachon et al [204] Hist. of Action Poses Stat Lowlv Hand Shahroudy et al [205] Feature Fusion BoW Body Unsup Cavazza et al [206] Kernelized-COV Stat Lowlv Hand …”

Section: Representations Based On Raw Joint Positionsmentioning

confidence: 99%

“…Given a new instance, this encoding methodology uses the normalized frequency vector of code occurrence as the final feature vector. Bag-of-words encoding is widely employed by a large number of skeleton-based human representations [174,143,144,210,186,109,150,127,188,189,115,152,190,192,159,147,165,167,179,162,218,219]. According to how the dictionary is learned, the encoding methods can be broadly categorized into two groups, based on clustering or sparse coding.…”

Section: Bag-of-words Encodingmentioning

confidence: 99%

Space-time representation of people based on 3D skeletal data: A review

Han

Reily

Hoff

et al. 2017

Computer Vision and Image Understanding

276

192

View full text Add to dashboard Cite

Spatiotemporal human representation based on 3D visual perception data is a rapidly growing research area. Representations can be broadly categorized into two groups, depending on whether they use RGB-D information or 3D skeleton data. Recently, skeletonbased human representations have been intensively studied and kept attracting an increasing attention, due to their robustness to variations of viewpoint, human body scale and motion speed as well as the realtime, online performance. This paper presents a comprehensive survey of existing space-time representations of people based on 3D skeletal data, and provides an informative categorization and analysis of these methods from the perspectives, including information modality, representation encoding, structure and transition, and feature engineering. We also provide a brief overview of skeleton acquisition devices and construction methods, enlist a number of benchmark datasets with skeleton data, and discuss potential future research directions.

show abstract

“…Domain Adaptation for Action Recognition. Of the several domain shifts in action recognition, only one has received significant research attention, that is the problem of cross-viewpoint (or viewpoint-invariant) action recognition [24,27,31,40,46]. These works focus on adapting to the geometric transformations of a camera but do little to combat other shifts, like changes in environment.…”

Section: Related Workmentioning

confidence: 99%

Multi-Modal Domain Adaptation for Fine-Grained Action Recognition

Munro

Damen

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Fine-grained action recognition datasets exhibit environmental bias, where multiple video sequences are captured from a limited number of environments. Training a model in one environment and deploying in another results in a drop in performance due to an unavoidable domain shift. Unsupervised Domain Adaptation (UDA) approaches have frequently utilised adversarial training between the source and target domains. However, these approaches have not explored the multi-modal nature of video within each domain. In this work we exploit the correspondence of modalities as a self-supervised alignment approach for UDA in addition to adversarial alignment (Fig. 1).We test our approach on three kitchens from our largescale dataset, EPIC-Kitchens [8], using two modalities commonly employed for action recognition: RGB and Optical Flow. We show that multi-modal self-supervision alone improves the performance over source-only training by 2.4% on average. We then combine adversarial training with multi-modal self-supervision, showing that our approach outperforms other UDA methods by 3%.

show abstract

Learning a non-linear knowledge transfer model for cross-view action recognition

Cited by 135 publications

References 25 publications

View-Invariant Deep Architecture for Human Action Recognition Using Two-Stream Motion and Shape Temporal Dynamics

View-Invariant Deep Architecture for Human Action Recognition Using Two-Stream Motion and Shape Temporal Dynamics

Space-time representation of people based on 3D skeletal data: A review

Multi-Modal Domain Adaptation for Fine-Grained Action Recognition

Contact Info

Product

Resources

About