PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection

Liao, Yue; Liu, Si; Wang, Fei; Chen, Yanjie; Qian, Chen; Feng, Jiashi

doi:10.1109/cvpr42600.2020.00056

Cited by 257 publications

(207 citation statements)

References 16 publications

Supporting

Mentioning

207

Contrasting

Order By: Relevance

“…The objective of Human Object Interaction (HOI) detection is to locate humans and objects and to recognise their interactions. Previous studies [32][33][34][35][36][37] show promising results of HOI sensing by decoupling it into the detection and classification of objects. In particular, the results of human and object detection first come from an object detector pre-trained, and then a pair of combined proposals for human objects interaction classification.…”

Section: Human Object Interactionmentioning

confidence: 99%

“…In particular, the results of human and object detection first come from an object detector pre-trained, and then a pair of combined proposals for human objects interaction classification. In recent approaches [38,37,36], a substitute detection problem was introduced, which would indirectly optimise the HOI detection. Firstly, the proposal of interaction was predefined on the basis of human priors.…”

Section: Human Object Interactionmentioning

confidence: 99%

See 1 more Smart Citation

Study on Temperature Variance for SimCLR based Activity Recognition

Kumar¹

2021

Preprint

View full text Add to dashboard Cite

Human Activity Recognition (HAR) is a process to automatically detect human activities based on stream data generated from various sensors, including inertial sensors, physiological sensors, location sensors, cameras, time, and many others. In this paper, we propose a robust SimCLR model for human activity recognition with a temperature variance study. In this work, SimCLR, a contrasting learning technique is optimized via regulating the temperature for visual representations, is incorporated for improving the HAR performance in healthcare.

show abstract

Section: Human Object Interactionmentioning

confidence: 99%

Section: Human Object Interactionmentioning

confidence: 99%

Study on Temperature Variance for SimCLR based Activity Recognition

Kumar¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…More recently, it has become popular to treat object detection as a keypoint estimation task [Law and Deng, 2018; to improve detection efficiency. Following the same idea, IPNet and PPDM [Liao et al, 2020] pose HOI detection as a keypoint detection and grouping problem, which propose to build end-toend HOI detectors. Specifically, PPDM [Liao et al, 2020] contains two parallel branches to predict three points (i.e., human, object and interaction points) and two displacements from the interaction point to its corresponding human and object points.…”

Section: Framework In Hoi Detectionmentioning

confidence: 99%

“…Following the same idea, IPNet and PPDM [Liao et al, 2020] pose HOI detection as a keypoint detection and grouping problem, which propose to build end-toend HOI detectors. Specifically, PPDM [Liao et al, 2020] contains two parallel branches to predict three points (i.e., human, object and interaction points) and two displacements from the interaction point to its corresponding human and object points. Then, human with its corresponding object can be paired using the interaction point and displacements as bridge.…”

Section: Framework In Hoi Detectionmentioning

confidence: 99%

RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection

Yang

Zou

Zhang

et al. 2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Human-Object Interaction (HOI) detection devotes to learn how humans interact with surrounding objects. Latest end-to-end HOI detectors are short of relation reasoning, which leads to inability to learn HOI-specific interactive semantics for predictions. In this paper, we therefore propose novel relation reasoning for HOI detection. We first present a progressive Relation-aware Frame, which brings a new structure and parameter sharing pattern for interaction inference. Upon the frame, an Interaction Intensifier Module and a Correlation Parsing Module are carefully designed, where: a) interactive semantics from humans can be exploited and passed to objects to intensify interactions, b) interactive correlations among humans, objects and interactions are integrated to promote predictions. Based on modules above, we construct an end-to-end trainable framework named Relation Reasoning Network (abbr. RR-Net). Extensive experiments show that our proposed RR-Net sets a new state-of-the-art on both V-COCO and HICO-DET benchmarks and improves the baseline about 5.5% and 9.8% relatively, validating that this first effort in exploring relation reasoning and integrating interactive semantics has brought obvious improvement for end-to-end HOI detection.

show abstract

“…Finally, very recently models built on anchor-free point-based detection frameworks have been proposed to perform HOI detection [27,28]. They treat HOIs as keypoints lying between a human and an object.…”

Section: Related Workmentioning

confidence: 99%

Detecting human—object interaction with multi-level pairwise feature network

Liu

Huang

2020

Comp. Visual Media

View full text Add to dashboard Cite

Human–object interaction (HOI) detection is crucial for human-centric image understanding which aims to infer ⟨human, action, object⟩ triplets within an image. Recent studies often exploit visual features and the spatial configuration of a human–object pair in order to learn the action linking the human and object in the pair. We argue that such a paradigm of pairwise feature extraction and action inference can be applied not only at the whole human and object instance level, but also at the part level at which a body part interacts with an object, and at the semantic level by considering the semantic label of an object along with human appearance and human–object spatial configuration, to infer the action. We thus propose a multi-level pairwise feature network (PFNet) for detecting human–object interactions. The network consists of three parallel streams to characterize HOI utilizing pairwise features at the above three levels; the three streams are finally fused to give the action prediction. Extensive experiments show that our proposed PFNet outperforms other state-of-the-art methods on the V-COCO dataset and achieves comparable results to the state-of-the-art on the HICO-DET dataset.

show abstract

PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection

Cited by 257 publications

References 16 publications

Study on Temperature Variance for SimCLR based Activity Recognition

Study on Temperature Variance for SimCLR based Activity Recognition

RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection

Detecting human—object interaction with multi-level pairwise feature network

Contact Info

Product

Resources

About