Home Action Genome: Cooperative Compositional Action Understanding

Rai, Nishant; Chen, Haofeng; Ji, Jingwei; Desai, Rishi; Kozuka, Kazuki; Ishizaka, Shun; Adeli, Ehsan; Niebles, Juan Carlos

doi:10.1109/cvpr46437.2021.01103

Cited by 49 publications

(22 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[445] 2019 RGB 700 -650,317 -Kitchen20 [476] 2019 Au 20 -800 -MMAct [356] 2019 RGB,S,Ac,Gyr,etc. 37 20 36,764 4+Egocentric Moments in Time [477] 2019 RGB 339 -∼1,000,000 -Wang et al [478] 2019 WiFi CSI 6 1 1,394 -NTU RGB+D 120 [185] 2019 RGB,S,D,IR 120 106 114,480 155 ETRI-Activity3D [479] 2020 RGB,S,D 55 100 112,620 -EV-Action [357] 2020 RGB,S,D,EMG 20 70 7,000 9 IKEA ASM [480] 2020 RGB,S,D 33 48 16,764 3 RareAct [481] 2020 RGB 122 -905 -BABEL [482] 2021 Mocap 252 -13,220 -HAA500 [483] 2021 RGB 500 -10,000 -HOMAGE [484] 2021 RGB,IR,Ac,Gyr,etc. 75 27 1,752 2∼5 MultiSports [485] 2021 RGB 66 -37,701 -UAV-Human [486] 2021 RGB,S,D,IR,etc.…”

Section: Datasetsmentioning

confidence: 99%

Human Action Recognition From Various Data Modalities: A Review

Sun

Rahmani

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

286

View full text Add to dashboard Cite

Human Action Recognition (HAR) aims to understand human behavior and assign a label to each action. It has a wide range of applications, and therefore has been attracting increasing attention in the field of computer vision. Human actions can be represented using various data modalities, such as RGB, skeleton, depth, infrared, point cloud, event stream, audio, acceleration, radar, and WiFi signal, which encode different sources of useful yet distinct information and have various advantages depending on the application scenarios. Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities. In this paper, we present a comprehensive survey of recent progress in deep learning methods for HAR based on the type of input data modality. Specifically, we review the current mainstream deep learning methods for single data modalities and multiple data modalities, including the fusion-based and the co-learning-based frameworks. We also present comparative results on several benchmark datasets for HAR, together with insightful observations and inspiring future research directions.

show abstract

Section: Datasetsmentioning

confidence: 99%

Human Action Recognition From Various Data Modalities: A Review

Sun

Rahmani

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

286

View full text Add to dashboard Cite

show abstract

“…Labeling work would be painful and error-prone for datasets not collected in controlled settings. Some of them were annotated by hand, like Fine-Gym, UAV-Human, HOMAGE [11,13,28], etc. Other datasets (e.g., Ac-tivityNet, AVA, Babel [12,17,29]) were labeled through commercial crowd-sourcing platforms like Amazon Mechanical Turk (AMT) [30], with a charge for dataset creators.…”

Section: Positive Impactsmentioning

confidence: 99%

“…Human activity recognition is attracting increasing attention. There is a large number of publicly available datasets [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17]. It requires a lot of labor-intensive work to annotate the video datasets.…”

Section: Introductionmentioning

confidence: 99%

HAVPTAT: A Human Activity Video Pose Tracking Annotation Tool

Quan

Bonarini

2022

Software Impacts

View full text Add to dashboard Cite

“…We recognize activities under domain shifts, caused by change of scenery, camera viewpoint or actor, with the aid of sound. 28,31,38,39,45,46,49]. For instance, both Gao et al [18] and Korbar et al [24] reduce the computational cost by previewing the audio track, while Lee et al [25] show that combining visual features with audio can better localize actions.…”

Section: Introductionmentioning

confidence: 99%

Audio-Adaptive Activity Recognition Across Video Domains

Zhang¹,

Doughty²,

Shao³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper strives for activity recognition under domain shift, for example caused by change of scenery or camera viewpoint. The leading approaches reduce the shift in activity appearance by adversarial training and self-supervised learning. Different from these vision-focused works we leverage activity sounds for domain adaptation as they have less variance across domains and can reliably indicate which activities are not happening. We propose an audio-adaptive encoder and associated learning methods that discriminatively adjust the visual feature representation as well as addressing shifts in the semantic distribution. To further eliminate domain-specific features and include domain-invariant activity sounds for recognition, an audio-infused recognizer is proposed, which effectively models the cross-modal interaction across domains. We also introduce the new task of actor shift, with a corresponding audio-visual dataset, to challenge our method with situations where the activity appearance changes dramatically. Experiments on this dataset, EPIC-Kitchens and CharadesEgo show the effectiveness of our approach. Project page: https://xiaobai1217. github.io/DomainAdaptation.

show abstract

Home Action Genome: Cooperative Compositional Action Understanding

Cited by 49 publications

References 22 publications

Human Action Recognition From Various Data Modalities: A Review

Human Action Recognition From Various Data Modalities: A Review

HAVPTAT: A Human Activity Video Pose Tracking Annotation Tool

Audio-Adaptive Activity Recognition Across Video Domains

Contact Info

Product

Resources

About