OW-DETR: Open-world Detection Transformer

Gupta, Akshita; Narayan, Sanath; Joseph, K J; Khan, Salman A.; Khan, Fahad Shahbaz; Shah, Mubarak

doi:10.48550/arxiv.2112.01513

Cited by 2 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two-stage Bipartite Matching Algorithm: Similar to OW-DETR (Gupta et al 2021), we apply a two-stage bipartite matching algorithm to match the seen pairs and the unknown pairs respectively, as shown in Fig. 3.…”

Section: Learning To Detect Potential Interactive Pairsmentioning

confidence: 99%

End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation

Shen³

et al. 2023

AAAI

View full text Add to dashboard Cite

Most existing Human-Object Interaction (HOI) Detection methods rely heavily on full annotations with predefined HOI categories, which is limited in diversity and costly to scale further. We aim at advancing zero-shot HOI detection to detect both seen and unseen HOIs simultaneously. The fundamental challenges are to discover potential human-object pairs and identify novel HOI categories. To overcome the above challenges, we propose a novel End-to-end zero-shot HOI Detection (EoID) framework via vision-language knowledge distillation. We first design an Interactive Score module combined with a Two-stage Bipartite Matching algorithm to achieve interaction distinguishment for human-object pairs in an action-agnostic manner. Then we transfer the distribution of action probability from the pretrained vision-language teacher as well as the seen ground truth to the HOI model to attain zero-shot HOI classification. Extensive experiments on HICO-Det dataset demonstrate that our model discovers potential interactive pairs and enables the recognition of unseen HOIs. Finally, our method outperforms the previous SOTA under various zero-shot settings. Moreover, our method is generalizable to large-scale object detection data to further scale up the action sets. The source code is available at: https://github.com/mrwu-mac/EoID.

show abstract

Section: Learning To Detect Potential Interactive Pairsmentioning

confidence: 99%

End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation

Shen³

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…Object detection, which entails localizing and identifying objects within an image or a video sequence, is a crucial task in computer vision. Extensive researches have been conducted in this field, leading to the development of numerous works [40,47,11,10,2,14,21]. However, the majority of these works operate under the closed-world assumption, limiting their applicability to a pre-defined set of categories.…”

Section: Introductionmentioning

confidence: 99%

OpenInst: A Simple Query-Based Method for Open-World Instance Segmentation

Wang¹,

Wang²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Open-world instance segmentation has recently gained significant popularitydue to its importance in many real-world applications, such as autonomous driving, robot perception, and remote sensing. However, previous methods have either produced unsatisfactory results or relied on complex systems and paradigms. We wonder if there is a simple way to obtain state-of-the-art results. Fortunately, we have identified two observations that help us achieve the best of both worlds: 1) query-based methods demonstrate superiority over dense proposal-based methods in open-world instance segmentation, and 2) learning localization cues is sufficient for open world instance segmentation. Based on these observations, we propose a simple query-based method named OpenInst for open world instance segmentation. OpenInst leverages advanced query-based methods like QueryInst and focuses on learning localization cues. Notably, OpenInst is an extremely simple and straightforward framework without any auxiliary modules or post-processing, yet achieves state-of-the-art results on multiple benchmarks. Specifically, in the COCO→UVO scenario, OpenInst achieves a mask AR of 53.3, outperforming the previous best methods by 2.0 AR with a simpler structure. We hope that OpenInst can serve as a solid baselines for future research in this area.

show abstract

OW-DETR: Open-world Detection Transformer

Cited by 2 publications

References 22 publications

End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation

End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation

OpenInst: A Simple Query-Based Method for Open-World Instance Segmentation

Contact Info

Product

Resources

About