HICO: A Benchmark for Recognizing Human-Object Interactions in Images

Chao, Yu-Wei; Wang, Zhan; He, Yugeng; Wang, Jiaxuan; Deng, Jia

doi:10.1109/iccv.2015.122

Cited by 291 publications

(269 citation statements)

References 28 publications

Supporting

Mentioning

262

Contrasting

Order By: Relevance

“…Visual phrase models such as [40] have demonstrated better robustness to the visual diversity of relations than compositional models. However, with the introduction of datasets with a larger vocabulary of objects and predicates [6,23], visual phrase approaches have been facing severe difficulties as most relations have very few training examples. Compositional methods [9,11,17,27,30,33,42], which allow sharing knowledge across triplets, have scaled better but do not cope well with unseen relations.…”

Section: Related Workmentioning

confidence: 99%

Detecting Unseen Visual Relations Using Analogies

Peyre¹,

Šivic²,

Laptev³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

122

146

View full text Add to dashboard Cite

We seek to detect visual relations in images of the form of triplets t = (subject, predicate, object), such as "person riding dog", where training examples of the individual entities are available but their combinations are unseen at training. This is an important set-up due to the combinatorial nature of visual relations : collecting sufficient training data for all possible triplets would be very hard. The contributions of this work are three-fold. First, we learn a representation of visual relations that combines (i) individual embeddings for subject, object and predicate together with (ii) a visual phrase embedding that represents the relation triplet. Second, we learn how to transfer visual phrase embeddings from existing training triplets to unseen test triplets using analogies between relations that involve similar objects. Third, we demonstrate the benefits of our approach on three challenging datasets : on HICO-DET, our model achieves significant improvement over a strong baseline for both frequent and unseen triplets, and we observe similar improvement for the retrieval of unseen triplets with out-ofvocabulary predicates on the COCO-a dataset as well as the challenging unusual triplets in the UnRel dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Detecting Unseen Visual Relations Using Analogies

Peyre¹,

Šivic²,

Laptev³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

122

146

View full text Add to dashboard Cite

show abstract

“…one of the more interesting aspects of video understanding, namely modeling the changes in appearance and semantics of scenes, objects and humans over time [6,13,15,37].…”

Section: Introductionmentioning

confidence: 99%

Detect-and-Track: Efficient Pose Estimation in Videos

Girdhar

Gkioxari

Torresani

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

227

192

View full text Add to dashboard Cite

This paper addresses the problem of estimating and tracking human body keypoints in complex, multi-person video. We propose an extremely lightweight yet highly effective approach that builds upon the latest advancements in human detection [17] and video understanding [5]. Our method operates in two-stages: keypoint estimation in frames or short clips, followed by lightweight tracking to generate keypoint predictions linked over the entire video. For frame-level pose estimation we experiment with Mask R-CNN, as well as our own proposed 3D extension of this model, which leverages temporal information over small clips to generate more robust frame predictions. We conduct extensive ablative experiments on the newly released multi-person video pose estimation benchmark, PoseTrack, to validate various design choices of our model. Our approach achieves an accuracy of 55.2% on the validation and 51.8% on the test set using the Multi-Object Tracking Accuracy (MOTA) metric, and achieves state of the art performance on the ICCV 2017 PoseTrack keypoint tracking challenge [1].

show abstract

“…To the best of our knowledge, our work is the first attempt to apply the attention mechanism to human body part correlations for recognizing humanobject interactions. We evaluate our model on the HICO dataset [5] and the MPII dataset [2]. Our method achieves the state-of-the-art result, and outperforms the previous methods by 10% relatively in mAP on HICO dataset.…”

Section: Introductionmentioning

confidence: 88%

Pairwise Body-Part Attention for Recognizing Human-Object Interactions

Fang

Cao

Tai

et al. 2018

Lecture Notes in Computer Science

110

View full text Add to dashboard Cite

In human-object interactions (HOI) recognition, conventional methods consider the human body as a whole and pay a uniform attention to the entire body region. They ignore the fact that normally, human interacts with an object by using some parts of the body. In this paper, we argue that different body parts should be paid with different attention in HOI recognition, and the correlations between different body parts should be further considered. This is because our body parts always work collaboratively. We propose a new pairwise body-part attention model which can learn to focus on crucial parts, and their correlations for HOI recognition. A novel attention based feature selection method and a feature representation scheme that can capture pairwise correlations between body parts are introduced in the model. Our proposed approach achieved 10% relative improvement (36.1 mAP→ 39.9 mAP) over the state-of-the-art results in HOI recognition on the HICO dataset. We will make our model and source codes publicly available.

show abstract

HICO: A Benchmark for Recognizing Human-Object Interactions in Images

Cited by 291 publications

References 28 publications

Detecting Unseen Visual Relations Using Analogies

Detecting Unseen Visual Relations Using Analogies

Detect-and-Track: Efficient Pose Estimation in Videos

Pairwise Body-Part Attention for Recognizing Human-Object Interactions

Contact Info

Product

Resources

About