Lite-FPN for Keypoint-based Monocular 3D Object Detection

Abstract

3D object detection with a single image is an essential and challenging task for autonomous driving. Recently, keypoint-based monocular 3D object detection has made tremendous progress and achieved great speed-accuracy trade-off. However, there still exists a huge gap with LIDAR-based methods in terms of accuracy. To improve their performance without sacrificing efficiency, we propose a sort of lightweight feature pyramid network called Lite-FPN to achieve multi-scale feature fusion in an effective and efficient way, which can boost the multi-scale detection capability of keypoint-based detectors. Besides, the misalignment between the classification score and the localization precision is further relieved by introducing a novel regression loss named attention loss. With the proposed loss, predictions with high confidence but poor localization are treated with more attention during the training phase. Comparative experiments based on several state-ofthe-art keypoint-based detectors on the KITTI dataset show that our proposed method achieves significantly higher accuracy and frame rate at the same time. The code and pretrained models will be available at https://github. com/yanglei18/Lite-FPN .

CenterLoc3D: monocular 3D vehicle localization network for roadside surveillance cameras

“…3D feature estimation-based methods. To avoid ambiguous results from geometric constraints and further improve accuracy and efficiency, many new methods [15,17,18,[23][24][25][26][27][28] focus on direct 3D information extraction in images which can be used for 3D bounding box regression by CNNs. Mono3DBox [23] proposes an end-to-end detector that can directly predict the center point of the vehicle bottom in the image.…”

“…Then, vehicle location, dimension, and orientation can be solved by minimizing the reprojection error of 2D-3D bounding boxes. Lite-FPN [27] proposes a light-weight keypoint-based feature pyramid net-work, which contains three parts: backbone, detection head and post-processing. Top-k operation is used in the detection head to connect keypoints and regression sub-network.…”

“…In Fig. 12, we can see that 3D vehicle localization error increases as the distance grows between vehicles and road- MonoGRNet [13] Deep3DBox [19] GS3D [22] RTM3D [25] SMOKE [15] KM3D [28] Lite-FPN [27] Ours side cameras of different scenes in SVLD-3D test set. The Y-axis error is the largest among the X-, Y-, and Z-axis.…”

“…Monocular 3D vehicle detection methods can be divided into the following two categories: (1) geometric constraintbased methods [19][20][21][22]. (2) 3D feature estimation based methods [23][24][25][26][27][28]. Methods of the first category are implemented based on mature 2D detectors, which use 3D bounding boxes or CAD models to represent vehicles.…”