An End-to-End Transformer Model for 3D Object Detection

Misra, Ishan; Girdhar, Rohit; Joulin, Armand

doi:10.1109/iccv48922.2021.00290

Cited by 377 publications

(222 citation statements)

References 59 publications

Supporting

Mentioning

220

Contrasting

Order By: Relevance

“…Given positions and colors as input, our PointGroup detector (second row) clearly outperforms VoteNet. Using multiview features and normals instead of RGB colors, our Point-Group based detector gives improved detection results of 50.7 mAP@0.5, which outperforms the current state-of-theart detectors [41,63] on the validation set of ScanNet v2 with gains of 3.7 and 2.6 respectively. Also, our Point-Group generates notably better detections for small and thin objects than VoteNet, such as picture ("pic.")…”

Section: Pointgroup Implementation Detailsmentioning

confidence: 94%

D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding

Chen¹,

Wu²,

Nießner³

et al. 2021

Preprint

View full text Add to dashboard Cite

Listener discriminates" this is a black chair . it is against the wall and facing the door. " 4 " this chair is black . it is under the whiteboard facing the round table ."2 " … " 3 " … " 5 " … " 1 Dense captions Detections Figure 1. We introduce D 3 Net, an end-to-end neural speaker-listener architecture that can detect, describe and discriminate. D 3 Net also enables semi-supervised training on Scan-Net data with partially annotated descriptions.

show abstract

Section: Pointgroup Implementation Detailsmentioning

confidence: 94%

D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding

Chen¹,

Wu²,

Nießner³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…As such, we have reached a stage where researchers are moving on to improve performance on even complex computer vision problems, viz. 3D object detection [32], action detection and localization [3] tracking objects across videos, event recognition and scene understanding [33][4], etc. In this chapter, we focus on the task of event/activity recognition in videos, done with the assistance of frame-wise object detection, which enables inter-frame tracking of objects.…”

Section: Video Activity Recognition Assisted By Object Detectionmentioning

confidence: 99%

Recent Trends in 2D Object Detection and Applications in Video Event Recognition

Jana¹,

Mohanta²

2022

Preprint

View full text Add to dashboard Cite

Object detection serves as a significant step in improving performance of complex downstream computer vision tasks. It has been extensively studied for many years now and current state-of-the-art 2D object detection techniques proffer superlative results even in complex images. In this chapter, we discuss the geometry-based pioneering works in object detection, followed by the recent breakthroughs that employ deep learning. Some of these use a monolithic architecture that takes a RGB image as input and passes it to a feed-forward ConvNet or vision Transformer. These methods, thereby predict class-probability and bounding-box coordinates, all in a single unified pipeline. Two-stage architectures on the other hand, first generate region proposals and then feed it to a CNN to extract features and predict object category and boundingbox. We also elaborate upon the applications of object detection in video event recognition, to achieve better fine-grained video classification performance. Further, we highlight recent datasets for 2D object detection both in images and videos, and present a comparative performance summary of various state-of-the-art object detection techniques.

show abstract

“…To handle the second issue, we adopt the set-to-set loss to match the predicted sequences with the ground truths. Unlike existing approaches [3,21,36] that utilize the sum of the classification and regression loss as the cost function for bipartite matching, in this paper, we propose a novel metric to measure the similarity between two sequences. Then we perform bipartite matching by maximizing the global similarity of the prediction and the ground truth set using the proposed metric.…”

Section: Introductionmentioning

confidence: 99%

Point2Seq: Detecting 3D Objects as Sequences

Xue¹,

Mao²,

Niu³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present a simple and effective framework, named Point2Seq, for 3D object detection from point clouds. In contrast to previous methods that normally predict attributes of 3D objects all at once, we expressively model the interdependencies between attributes of 3D objects, which in turn enables a better detection accuracy. Specifically, we view each 3D object as a sequence of words and reformulate the 3D object detection task as decoding words from 3D scenes in an auto-regressive manner. We further propose a lightweight scene-to-sequence decoder that can auto-regressively generate words conditioned on features from a 3D scene as well as cues from the preceding words. The predicted words eventually constitute a set of sequences that completely describe the 3D objects in the scene, and all the predicted sequences are then automatically assigned to the respective ground truths through similarity-based sequence matching. Our approach is conceptually intuitive and can be readily plugged upon most existing 3D-detection backbones without adding too much computational overhead; the sequential decoding paradigm we proposed, on the other hand, can better exploit information from complex 3D scenes with the aid of preceding predicted words. Without bells and whistles, our method significantly outperforms previous anchor-and center-based 3D object detection frameworks, yielding the new state of the art on the challenging ONCE dataset as well as the Waymo Open Dataset. Code is available at https://github.com/ ocNflag/point2seq.

show abstract

An End-to-End Transformer Model for 3D Object Detection

Cited by 377 publications

References 59 publications

D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding

D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding

Recent Trends in 2D Object Detection and Applications in Video Event Recognition

Point2Seq: Detecting 3D Objects as Sequences

Contact Info

Product

Resources

About