RGB-D Object Classification for Autonomous Driving Perception

Premebida, Cristiano; Melotti, Gledson; Asvadi, Alireza

doi:10.1007/978-3-030-28603-3_17

Cited by 10 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sensor technology, data-fusion, and inference algorithms as Artificial Intelligence and Machine Learning (AI/ML) applications are the enabling technologies that play a cornerstone role in autonomous driving systems. These are involved in many of the essential tasks for safe driving such as sensor-fusion, environment representation, scene understanding, semantic segmentation, tracking, object detection, and recognition [15].…”

Section: Introductionmentioning

confidence: 99%

“…Amongst sensors, the RGB (red, green, and blue) camera is acknowledged as the most commonly used and an irreplaceable one [15]. In fact, despite cameras have the known disadvantages of strong sensitivity to external illumination and limited field of view, visual recognition systems are amongst the most solid applications of autonomous driving [33].…”

Section: Introductionmentioning

confidence: 99%

“…Vehicle cameras are already exploited in many applications such as traffic sign recognition, lane detection, obstacle detection, etc. [15], [16], [17]. Additional prospective applications are being researched; for example, at intersections, knowing the location of pedestrians and bicyclists can allow the car to make sophisticated precedence decisions [16].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Secci

Ceccarelli

2023

IEEE Trans. Dependable and Secure Comput.

View full text Add to dashboard Cite

RGB cameras are one of the most relevant sensors for autonomous driving applications. It is undeniable that failures of vehicle cameras may compromise the autonomous driving task, possibly leading to unsafe behaviors when images that are subsequently processed by the driving system are altered. To support the definition of safe and robust vehicle architectures and intelligent systems, in this paper we define the failure modes of a vehicle camera, together with an analysis of effects and known mitigations. Further, we build a software library for the generation of the corresponding failed images, and we feed them to six object detectors for mono and stereo cameras and to the self-driving agent of an autonomous driving simulator. The resulting misbehaviors with respect to operating with clean images allow a better understanding of failures effects and the related safety risks in image-based applications.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Secci

Ceccarelli

2023

IEEE Trans. Dependable and Secure Comput.

View full text Add to dashboard Cite

show abstract

“…These are involved in the majority of the essential tasks for safe driving such as sensor-fusion, environment representation, scene understanding, semantic segmentation, tracking, object detection and recognition [15]. Amongst sensors, the RGB (red, green and blue) camera is acknowledged as the most commonly used and an irreplaceable one [15]. Despite cameras have the known disadvantages of strong sensitivity to external illumination and limited field of view, visual recognition systems are amongst the most solid applications of autonomous driving [33].…”

Section: Introductionmentioning

confidence: 99%

“…Sensor technology, data-fusion and inference algorithms as Artificial Intelligence and machine learning (AI/ML) applications are the enabling technologies that play a cornerstone role in autonomous driving systems. These are involved in the majority of the essential tasks for safe driving such as sensor-fusion, environment representation, scene understanding, semantic segmentation, tracking, object detection and recognition [15]. Amongst sensors, the RGB (red, green and blue) camera is acknowledged as the most commonly used and an irreplaceable one [15].…”

Section: Introductionmentioning

confidence: 99%

On failures of RGB cameras and their effects in autonomous driving applications

Secci

Ceccarelli

2020

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)

View full text Add to dashboard Cite

RGB cameras are arguably one of the most relevant sensors for autonomous driving applications. It is undeniable that failures of vehicle cameras may compromise the autonomous driving task, possibly leading to unsafe behaviors when images that are subsequently processed by the driving system are altered. To support the definition of safe and robust vehicle architectures and intelligent systems, in this paper we define the failures model of a vehicle camera, together with an analysis of effects and known mitigations. Further, we build a software library for the generation of the corresponding failed images and we feed them to the trained agent of an autonomous driving simulator: the misbehavior of the trained agent allows a better understanding of failures effects and especially of the resulting safety risk.

show abstract

Time division multiplexing based multi-spectral semantic camera for LiDAR applications

Kim,

Jeong,

Kim

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The recent progress in the development of measurement systems for autonomous recognition had a substantial impact on emerging technology in numerous fields, especially robotics and automotive applications. In particular, time-of-flight (TOF) based light detection and ranging (LiDAR) systems enable to map the surrounding environmental information over long distances and with high accuracy. The combination of advanced LiDAR with an artificial intelligence platform allows enhanced object recognition and classification, which however still suffers from limitations of inaccuracy and misidentification. Recently, multi-spectral LiDAR systems have been employed to increase the object recognition performance by additionally providing material information in the short-wave infrared (SWIR) range where the reflection spectrum characteristics are typically very sensitive to material properties. However, previous multi-spectral LiDAR systems utilized band-pass filters or complex dispersive optical systems and even required multiple photodetectors, adding complexity and cost. In this work, we propose a time-division-multiplexing (TDM) based multi-spectral LiDAR system for semantic object inference by the simultaneous acquisition of spatial and spectral information. By utilizing the TDM method, we enable the simultaneous acquisition of spatial and spectral information as well as a TOF based distance map with minimized optical loss using only a single photodetector. Our LiDAR system utilizes nanosecond pulses of five different wavelengths in the SWIR range to acquire sufficient material information in addition to 3D spatial information. To demonstrate the recognition performance, we map the multi-spectral image from a human hand, a mannequin hand, a fabric gloved hand, a nitrile gloved hand, and a printed human hand onto an RGB-color encoded image, which clearly visualizes spectral differences as RGB color depending on the material while having a similar shape. Additionally, the classification performance of the multi-spectral image is demonstrated with a convolution neural network (CNN) model using the full multi-spectral data set. Our work presents a compact novel spectroscopic LiDAR system, which provides increased recognition performance and thus a great potential to improve safety and reliability in autonomous driving.

show abstract

RGB-D Object Classification for Autonomous Driving Perception

Cited by 10 publications

References 20 publications

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

On failures of RGB cameras and their effects in autonomous driving applications

Time division multiplexing based multi-spectral semantic camera for LiDAR applications

Contact Info

Product

Resources

About