Three-dimensional objects are commonly represented as 3D boxes in a pointcloud. This representation mimics the well-studied image-based 2D boundingbox detection, but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. We use a keypoint detector to find centers of objects, and simply regress to other attributes, including 3D size, 3D orientation, and velocity. In our centerbased framework, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. On the nuScenes dataset, our point-based representations perform 3-4 mAP higher than the box-based counterparts for 3D detection, and 6 AMOTA higher for 3D tracking. Our real-time model runs end-to-end 3D detection and tracking at 30 FPS with 54.2 AMOTA and 48.3 mAP while the best single model achieves 60.3 mAP for 3D detection, and 63.8 AMOTA for 3D tracking. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

No abstract

No abstract

No abstract

No abstract