Online Camera LiDAR Fusion and Object Detection on Hybrid Data for Autonomous Driving

Banerjee, Koyel; Notz, Dominik; Windelen, Johannes; Gavarraju, Sumanth; He, Mingkang

doi:10.1109/ivs.2018.8500699

Cited by 65 publications

(42 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, when the detection results of the single-object detection systems are different from each other, the performance can be improved through the proposed MYs-WM by reinforcement through result fusion. The detection result of the car through MYs-WM was improved to 89.83% (IOU = 0.7), and that of the pedestrian to 79.25% (IOU = 0.5), which is higher than that of the Faster R-CNN-based convergence system of [1]. Examples of fusion detection results of MYs-WM in comparison with Y-CM are shown in Figure 8 for cars and Figure 9 for pedestrians.…”

Section: Resultsmentioning

confidence: 91%

“…As mentioned earlier, the proposed system aims to enhance the performance of object detection by fusing all object detection results from Y-CM, Y-DM, and Y-RM through a weighted mean. Performance comparisons with the single-object detection systems (Y-CM, Y-DM, Y-RM) and [1], where Faster R-CNN is applied to VGG-16 [26] structure based on a RGB camera and a LiDAR, were conducted with IOUs of 0.3, 0.5, and 0.7, and the evaluation results are summarized in Table 1 for cars and Table 2 for pedestrians. The results show that Y-CM had the highest detection performance of 87.12% (IOU = 0.7) for cars and 76.62% (IOU = 0.5) for pedestrians among the single-object detection systems, while Y-DM and Y-RM each showed about 16% lower detection performance.…”

Section: Resultsmentioning

confidence: 99%

“…Object detection is a fundamental field of computer vision and an essential component of autonomous driving (AD), which uses sensors to detect the driving environment. Sensing the driving environment is closely related to safety, and research on fusion of sensors with different characteristics is being actively conducted to increase the detection rate of objects in a dynamic environment to secure such safety [1]. However, the development of a robust object detection system that can be operated in various driving environments in real time remains a challenge.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exploring a Multimodal Mixture-Of-YOLOs Framework for Advanced Real-Time Object Detection

Kim¹,

Cho²

2020

Applied Sciences

View full text Add to dashboard Cite

To construct a safe and sound autonomous driving system, object detection is essential, and research on fusion of sensors is being actively conducted to increase the detection rate of objects in a dynamic environment in which safety must be secured. Recently, considerable performance improvements in object detection have been achieved with the advent of the convolutional neural network (CNN) structure. In particular, the YOLO (You Only Look Once) architecture, which is suitable for real-time object detection by simultaneously predicting and classifying bounding boxes of objects, is receiving great attention. However, securing the robustness of object detection systems in various environments still remains a challenge. In this paper, we propose a weighted mean-based adaptive object detection strategy that enhances detection performance through convergence of individual object detection results based on an RGB camera and a LiDAR (Light Detection and Ranging) for autonomous driving. The proposed system utilizes the YOLO framework to perform object detection independently based on image data and point cloud data (PCD). Each detection result is united to reduce the number of objects not detected at the decision level by the weighted mean scheme. To evaluate the performance of the proposed object detection system, tests on vehicles and pedestrians were carried out using the KITTI Benchmark Suite. Test results demonstrated that the proposed strategy can achieve detection performance with a higher mean average precision (mAP) for targeted objects than an RGB camera and is also robust against external environmental changes.

show abstract

Section: Resultsmentioning

confidence: 91%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring a Multimodal Mixture-Of-YOLOs Framework for Advanced Real-Time Object Detection

Kim¹,

Cho²

2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Ishikawa et al [7] estimated the motion of LiDAR and camera separately and determined the intrinsics and extrinsics through motion-tomotion correspondences. Banerjee et al [1] detected the edges of objects in camera frames and calibrated through edge-to-edge correspondences. These approach s not sensitive to the quality of calibration equipment but relies on the segmentation of LiDAR and camera frames.…”

Section: A Lidar-camera Correspondence Collectionmentioning

confidence: 99%

“…The first one is the extrinsic transformation that is the projection model from LiDAR to camera coordinate. This 6-DOF matrix can be expressed as (1).…”

Section: A Calibration Modelsmentioning

confidence: 99%

An Interactive LiDAR to Camera Calibration

Lyu

Bai

Elhousni

et al. 2019

2019 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

Recent progress in the automated driving system (ADS) and advanced driver assistant system (ADAS) has shown that the combined use of 3D light detection and ranging (LiDAR) and the camera is essential for an intelligent vehicle to perceive and understand its surroundings. LiDAR-camera fusion requires precise intrinsic and extrinsic calibrations between the sensors. However, due to the limitation of the calibration equipment and susceptibility to noise, algorithms in existing methods tend to fail in finding LiDAR-camera correspondences in long-range. In this paper, we introduced an interactive LiDAR to camera calibration toolbox to estimate the intrinsic and extrinsic transforms. This toolbox automatically detects the corner of a planer board from a sequence of LiDAR frames and provides a convenient user interface for annotating the corresponding pixels on camera frames. Since the toolbox only detects the top corner of the board, there is no need to prepare a precise polygon planar board or a checkerboard with different reflectivity areas as in the existing methods. Furthermore, the toolbox uses genetic algorithms to estimate the transforms and supports multiple camera models such as the pinhole camera model and the fisheye camera model. Experiments using Velodyne VLP-16 LiDAR and Point Grey Chameleon 3 camera show robust results.

show abstract

LiDAR and Camera Data for Smart Urban Traffic Monitoring: Challenges of Automated Data Capturing and Synchronization

Vītols¹,

Bumanis

Arhipova³

et al. 2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

Online Camera LiDAR Fusion and Object Detection on Hybrid Data for Autonomous Driving

Cited by 65 publications

References 13 publications

Exploring a Multimodal Mixture-Of-YOLOs Framework for Advanced Real-Time Object Detection

Exploring a Multimodal Mixture-Of-YOLOs Framework for Advanced Real-Time Object Detection

An Interactive LiDAR to Camera Calibration

LiDAR and Camera Data for Smart Urban Traffic Monitoring: Challenges of Automated Data Capturing and Synchronization

Contact Info

Product

Resources

About