Diagnosing Rarity in Human-object Interaction Detection

Kilickaya, Mert; Smeulders, A.W.M.

doi:10.1109/cvprw50498.2020.00460

Cited by 5 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training images are used as the gallery set and the testing set is used as the query set. A unique property of the dataset is that 150 interactions have less than 10 examples in the training set, which means a query can only match very few images within the gallery set, leading to a challenging visual search setup [22]. HICO-DET is only used for evaluation.…”

Section: Datasetsmentioning

confidence: 99%

Structured Visual Search via Composition-aware Learning

Kilickaya¹,

Smeulders²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper studies visual search using structured queries. The structure is in the form of a 2D composition that encodes the position and the category of the objects. The transformation of the position and the category of the objects leads to a continuous-valued relationship between visual compositions, which carries highly beneficial information, although not leveraged by previous techniques. To that end, in this work, our goal is to leverage these continuous relationships by using the notion of symmetry in equivariance. Our model output is trained to change symmetrically with respect to the input transformations, leading to a sensitive feature space. Doing so leads to a highly efficient search technique, as our approach learns from fewer data using a smaller feature space. Experiments on two largescale benchmarks of MS-COCO [29] and HICO-DET [4] demonstrates that our approach leads to a considerable gain in the performance against competing techniques.

show abstract

Section: Datasetsmentioning

confidence: 99%

Structured Visual Search via Composition-aware Learning

Kilickaya¹,

Smeulders²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For visual understanding tasks this process becomes more difficult as it is harder to rely on well-defined visual features such as those generated by SIFT [26] features or convolutional neural networks [43]. However some distribution issues can be attributed to the dataset, as seen in the study [16], exploring HICO-DET and some of the multi-stream models covered in this survey. An attempt at the task of zero-shot recognition and weakly supervised learning is seen by Pyere et al in [28], incorporating sematic language information from large text databases that provide probabilities for the interaction in question.…”

Section: Weakly Supervisedmentioning

confidence: 99%

Human-Object Interaction Detection:A Quick Survey and Examination of Methods

Bergstrom,

Shi

2020

Preprint

View full text Add to dashboard Cite

Human-object interaction detection is a relatively new task in the world of computer vision and visual semantic information extraction. With the goal of machines identifying interactions that humans perform on objects, there are many real-world use cases for the research in this field. To our knowledge, this is the first general survey of the state-of-the-art and milestone works in this field. We provide a basic survey of the developments in the field of human-object interaction detection. Many works in this field use multi-stream convolutional neural network architectures, which combine features from multiple sources in the input image. Most commonly these are the humans and objects in question, as well as a spatial quality of the two. As far as we are aware, there have not been in-depth studies performed that look into the performance of each component individually. In order to provide insight to future researchers, we perform an individualized study that examines the performance of each component of a multi-stream convolutional neural network architectures for human-object interaction detection. Specifically we examine the HORCNN architecture as it is a foundational work in the field. In addition, we provide an in-depth look at the HICO-DET dataset, a popular benchmark in the field of human-object interaction detection. Code and papers can be found at https://github.com/SHI-Labs/Human-Object-Interaction-Detection. CCS CONCEPTS• Computing methodologies → Scene understanding.

show abstract

Human–object interaction recognition based on interactivity detection and multi-feature fusion

Xia

Ding

2023

Cluster Comput

View full text Add to dashboard Cite

Diagnosing Rarity in Human-object Interaction Detection

Cited by 5 publications

References 11 publications

Structured Visual Search via Composition-aware Learning

Structured Visual Search via Composition-aware Learning

Human-Object Interaction Detection:A Quick Survey and Examination of Methods

Human–object interaction recognition based on interactivity detection and multi-feature fusion

Contact Info

Product

Resources

About