Design of Real-time Semantic Segmentation Decoder for Automated Driving

Das, Apurba; Kandan, Saranya; Yogamani, Senthil; Křížek, Pavel

doi:10.5220/0007366003930400

Cited by 9 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) Semantic Segmentation: It is the process of assigning a class label to each pixel in an image such as a pedestrian, road, or curb as shown in Figure 14 (2 nd column). CNN-based approaches have recently been very successful compared to classical computer vision-based methods on semantic segmentation employed on a pinhole front camera [82]. Although, in urban traffic scenarios, autonomous cars require a wider field-of-view to perceive what is around them, particularly at intersections.…”

Section: A Semantic Tasksmentioning

confidence: 99%

Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges

Kumar

Eising

Witt

et al. 2023

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Surround-view fisheye cameras are commonly used for near-field sensing in automated driving. Four fisheye cameras on four sides of the vehicle are sufficient to cover 360 • around the vehicle capturing the entire near-field region. Some primary use cases are automated parking, traffic jam assist, and urban driving. There are limited datasets and very little work on near-field perception tasks as the focus in automotive perception is on far-field perception. In contrast to far-field, surround-view perception poses additional challenges due to high precision object detection requirements of 10cm and partial visibility of objects. Due to the large radial distortion of fisheye cameras, standard algorithms cannot be extended easily to the surround-view use case. Thus, we are motivated to provide a self-contained reference for automotive fisheye camera perception for researchers and practitioners. Firstly, we provide a unified and taxonomic treatment of commonly used fisheye camera models. Secondly, we discuss various perception tasks and existing literature. Finally, we discuss the challenges and future direction.

show abstract

Section: A Semantic Tasksmentioning

confidence: 99%

Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges

Kumar

Eising

Witt

et al. 2023

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…Semantic Segmentation: It is the process of assigning a class label to each pixel in an image such as a pedestrian, road, or curb as shown in Figure 14 (2 nd column). CNN-based approaches have recently been very successful compared to classical computer vision based methods on semantic segmentation employed on a pinhole front camera [78]. Although, in urban traffic scenarios, autonomous cars require a wider field-of-view to perceive what is around them, particularly at intersections.…”

Section: A Semantic Tasksmentioning

confidence: 99%

Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

Kumar¹,

Eising²,

Witt³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Surround-view fisheye cameras are commonly used for near-field sensing in automated driving. Four fisheye cameras on four sides of the vehicle are sufficient to cover 360°around the vehicle capturing the entire near-field region. Some primary use cases are automated parking, traffic jam assist, and urban driving. There are limited datasets and very little work on near-field perception tasks as the main focus in automotive perception is on far-field perception. In contrast to far-field, surround-view perception poses additional challenges due to high precision object detection requirements of 10cm and partial visibility of objects. Due to the large radial distortion of fisheye cameras, standard algorithms can not be extended easily to the surround-view use case. Thus we are motivated to provide a selfcontained reference for automotive fisheye camera perception for researchers and practitioners. Firstly, we provide a unified and taxonomic treatment of commonly used fisheye camera models. Secondly, we discuss various perception tasks and existing literature. Finally, we discuss the challenges and future direction. I. INTRODUCTIONSurround-view systems use four sensors to form a network with overlap regions, sufficient to cover the near-field area around the car. Figure 1 shows the four views of a typical surround-view system, along with a representation of the typical parking use-case. Wide-angle views exceeding 180 • are used for this near-field sensing. Any perception algorithm must consider the significant fisheye distortion inherent with such camera systems. This is a significant challenge, as most work in computer vision focuses on narrow field-of-view cameras with mild radial distortion. However, as such camera systems are more widely deployed, work has been completed in this area. It is the aim of this paper to give the reader an overview of surround view cameras (e.g., image formation, configuration and rectification), to survey the existing state of the art, and to provide insights into the current challenges in the area.In theory, the field-of-view of a pinhole camera is 180 • . However, in practice, due to the practical limitations of the size of the aperture and imager, it is not easy to get over 80 • , as illustrated in Figure 2 (top). Fisheye lenses are commonly used to effectively increase the field-of-view to 180 • or more. It is interesting to note that the term fisheye is a bit of a misnomer, as illustrated in Figure 2 (bottom). Due to the bending of light rays due to refraction at the junction of water and air surface, a large field-of-view of nearly 180 • is compressed to a smaller V. Ravi Kumar, C. Witt, and S. Yogamani are with Valeo. C. Eising is with the Dept. of Electronic and Computer Engineering at the University of Limerick. V. Ravi Kumar and C. Eising are co-first authors.

show abstract

“…One of the main challenges is to build an effective dataset which covers diverse aspects [55]. CNNs are computationally intensive and efficient design techniques are critical to be incorporated [56], [57]. CNNs are well studied for rectilinear images, however, the assumption of translation invariance is broken in fisheye images which poses additional challenges [58].…”

Section: A Recognitionmentioning

confidence: 99%

Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras

Eising¹,

Horgan²,

Yogamani³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Design of Real-time Semantic Segmentation Decoder for Automated Driving

Cited by 9 publications

References 26 publications

Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges

Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges

Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras

Contact Info

Product

Resources

About