Exploration of Spatial and Temporal Modeling Alternatives for HOI

Dabral, Rishabh; Sarkar, Srijon; Reddy, Sai Praneeth; Ramakrishnan, Ganesh

doi:10.1109/wacv48630.2021.00233

Cited by 4 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Qi et al [38] expand prior graphical models in DNNs for videos with learnable graph structures and pass messages through GPNN. Dabral et al [6] analyze the effectiveness of GCNs against Convolutional Networks and Capsule Networks for spatial relation learning. Wang et al [53] propose the STIGPN exploiting the parsed graphs to learn spatiotemporal connection development and discover objects existing in a scene.…”

Section: Hoi Recognition In Videosmentioning

confidence: 99%

Geometric Features Informed Multi-person Human-object Interaction Recognition in Videos

Qiao¹,

Men²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Human-Object Interaction (HOI) recognition in videos is important for analyzing human activity. Most existing work focusing on visual features usually suffer from occlusion in the real-world scenarios. Such a problem will be further complicated when multiple people and objects are involved in HOIs. Consider that geometric features such as human pose and object position provide meaningful information to understand HOIs, we argue to combine the benefits of both visual and geometric features in HOI recognition, and propose a novel Two-level Geometric feature-informed Graph Convolutional Network (2G-GCN). The geometric-level graph models the interdependency between geometric features of humans and objects, while the fusion-level graph further fuses them with visual features of humans and objects. To demonstrate the novelty and effectiveness of our method in challenging scenarios, we propose a new multi-person HOI dataset (MPHOI-72). Extensive experiments on MPHOI-72 (multi-person HOI), CAD-120 (single-human HOI) and Bimanual Actions (two-hand HOI) datasets demonstrate our superior performance compared to state-of-the-arts.

show abstract

Section: Hoi Recognition In Videosmentioning

confidence: 99%

Geometric Features Informed Multi-person Human-object Interaction Recognition in Videos

Qiao¹,

Men²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Qi et al [38] expand prior graphical models in DNNs for videos with learnable graph structures and pass messages through GPNN. Dabral et al [6] analyse the effectiveness of GCNs against Convolutional Networks and Capsule Networks for spatial relation learning. Wang et al [53] propose the STIGPN exploiting the parsed graphs to learn spatiotemporal connection development and discover objects existing in a scene.…”

Section: Hoi Recognition In Videosmentioning

confidence: 99%

“…On the other hand, the model needs to consider human dynamics in the video and the shifting orientations of items in the scene in relation to humans [38]. This makes it difficult to directly extend image-based models to video that exploit the region of interest (ROI) features of human-object union [6]. We propose a novel two-level graph to refine the interactive representations; the first graph models the interdependency within the geometric key points of human and objects, and the second graph models the interdependency between the visual features and the learned geometric representations.…”

Section: Geometric Features Informed Hoi Analysismentioning

confidence: 99%

Geometric Features Informed Multi-person Human-Object Interaction Recognition in Videos

Qiao

Men

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Human-Object Interaction (HOI) recognition in videos is important for analysing human activity. Most existing work focusing on visual features usually suffer from occlusion in the real-world scenarios. Such a problem will be further complicated when multiple people and objects are involved in HOIs. Consider that geometric features such as human pose and object position provide meaningful information to understand HOIs, we argue to combine the benefits of both visual and geometric features in HOI recognition, and propose a novel Two-level Geometric feature-informed Graph Convolutional Network (2G-GCN). The geometric-level graph models the interdependency between geometric features of humans and objects, while the fusion-level graph further fuses them with visual features of humans and objects. To demonstrate the novelty and effectiveness of our method in challenging scenarios, we propose a new multi-person HOI dataset (MPHOI-72). Extensive experiments on MPHOI-72 (multi-person HOI), CAD-120 (single-human HOI) and Bimanual Actions (two-hand HOI) datasets demonstrate our superior performance compared to state-of-the-arts.

show abstract

From Category to Scenery: An End-to-End Framework for Multi-person Human-Object Interaction Recognition in Videos

Qiao,

Li,

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Exploration of Spatial and Temporal Modeling Alternatives for HOI

Cited by 4 publications

References 37 publications

Geometric Features Informed Multi-person Human-object Interaction Recognition in Videos

Geometric Features Informed Multi-person Human-object Interaction Recognition in Videos

Geometric Features Informed Multi-person Human-Object Interaction Recognition in Videos

From Category to Scenery: An End-to-End Framework for Multi-person Human-Object Interaction Recognition in Videos

Contact Info

Product

Resources

About