Overview and Empirical Analysis of ISP Parameter Tuning for Visual Perception in Autonomous Driving

Yahiaoui, Lucie; Horgan, Jonathan; Deegan, Brian; Yogamani, Senthil; Hughes, Ciáran; Denny, Patrick

doi:10.3390/jimaging5100078

Cited by 32 publications

(25 citation statements)

References 83 publications

(100 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, as indicated in the state-of-the-art literature, this challenge remains an open problem in the engineering field. The described work shows that optimal ISP tuning allows for performance improvements in computer vision algorithms [15] and adaptability to different image acquisition conditions [16]. In [15], the authors propose automatic ISP tuning as a challenge for the future.…”

Section: Discussionmentioning

confidence: 99%

“…Each value affects a different frequency, and the higher it is, the greater the effect. • Denoise Edge Softness Y: It is a vector of 4 decimal values between [1,15]. It serves to soften the edges and reduce the noise of the luminance channel even more.…”

Section: A Isp Parameters and Restrictionsmentioning

confidence: 99%

“…As we will see later, the process of evaluating a possible solution requires more time than other problems, so it is necessary to arrive at a good solution with the minimum number of evaluations. Another work related to the ISP tuning process, in this case focused on advanced driver assistance systems (ADAS), is the study carried out by L. Yahiaoui et al [15] from Valeo. In their experimental work, the authors demonstrate the difference of tuning the ISP for a commercial camera that seeks the image quality with respect to the subjective appearance of the human eye, compared to the tuning of an ISP whose system aims to improve the image processing algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimization of the ISP Parameters of a Camera Through Differential Evolution

Hevia¹,

Patricio

Berlanga

2020

IEEE Access

View full text Add to dashboard Cite

Within the design and development of a smartphone, an important phase arises regarding time, which is related to the tuning of the ISP (image signal processor) of the camera. The ISP is an element that allows the adjustment of the images captured by a sensor in order to achieve the best image quality. The ISP implements different image improvement algorithms such as white balancing, denoising, and demosaicing as well as other image enhancement algorithms. The purpose of the ISP tuning process is to configure the parameters of these algorithms so that the processed images are of the highest quality. This task is carried out by the camera tuning engineer, who iteratively adjusts the ISP parameters through trial and error procedures until the desired quality is achieved. The complete adjustment process can be extended to several weeks and even months. The authors present a novel solution based on differential evolution, which allows a first-adjusted approximation of the ISP in a few hours. This work presents an architecture based on an optimization through a differential evolution algorithm with which different ISP tuning tests are carried out, and the good results in quality and time are verified.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Isp Parameters and Restrictionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimization of the ISP Parameters of a Camera Through Differential Evolution

Hevia¹,

Patricio

Berlanga

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The ISP is a processing block that converts the raw digital image output from the AFE into an image that can be used for a given application. This processing is very complex and include a number of discrete processing blocks that can be arranged in a different order depending on the ISP [ 16 ]. ISP consists of Lens shading, Defective Pixel Correction (DPC), denoise, color filter array (CFA), auto white balance (AWB), auto exposure (AE), color correction matrix (CCM), Gamma correction, Chroma Resampler and so on as shown in Figure 2 .…”

Section: Methodsmentioning

confidence: 99%

Image Pre-Processing Method of Machine Learning for Edge Detection with Image Signal Processor Enhancement

2021

View full text Add to dashboard Cite

Even though computer vision has been developing, edge detection is still one of the challenges in that field. It comes from the limitations of the complementary metal oxide semiconductor (CMOS) Image sensor used to collect the image data, and then image signal processor (ISP) is additionally required to understand the information received from each pixel and performs certain processing operations for edge detection. Even with/without ISP, as an output of hardware (camera, ISP), the original image is too raw to proceed edge detection image, because it can include extreme brightness and contrast, which is the key factor of image for edge detection. To reduce the onerousness, we propose a pre-processing method to obtain optimized brightness and contrast for improved edge detection. In the pre-processing, we extract meaningful features from image information and perform machine learning such as k-nearest neighbor (KNN), multilayer perceptron (MLP) and support vector machine (SVM) to obtain enhanced model by adjusting brightness and contrast. The comparison results of F1 score on edgy detection image of non-treated, pre-processed and pre-processed with machine learned are shown. The pre-processed with machine learned F1 result shows an average of 0.822, which is 2.7 times better results than the non-treated one. Eventually, the proposed pre-processing and machine learning method is proved as the essential method of pre-processing image from ISP in order to gain better edge detection image. In addition, if we go through the pre-processing method that we proposed, it is possible to more clearly and easily determine the object required when performing auto white balance (AWB) or auto exposure (AE) in the ISP. It helps to perform faster and more efficiently through the proactive ISP.

show abstract

“…These concepts are strongly linked to image processing, car-to-infrastructure and car-to-car communication, and safety procedures. Great progress has been made in image processing techniques as an essential aspect of autonomous driving [ 13 ]. Moreover, image processing is researched effectively to improve driving and safety and detect damages (cracks) in the infrastructure, as depicted in [ 14 ].…”

Section: Related Researchmentioning

confidence: 99%

Approaching OPC UA Publish–Subscribe in the Context of UDP-Based Multi-Channel Communication and Image Transmission

Ioana

Burlacu

Korodi

2021

Sensors

View full text Add to dashboard Cite

The Open Platform Communication Unified Architecture (OPC UA) protocol is a key enabler of Industry 4.0 and Industrial Internet of Things (IIoT). OPC UA is already accepted by the industry and its presence is expected to reach more and more fields, applications, and hierarchical levels. Advances within the latest specifications are providing the opportunity to extend the capabilities and the applicability of the protocol, targeting better performances in terms of data volumes, speed, availability, footprint, and security. Continuing previous researches focusing on the publish–subscribe (pub/sub) mechanism and real-time constraints, the current study aims to consider higher data-volumes, approach the multi-channel User Datagram Protocol (UDP)-based communication, and analyze the robustness of the developed mechanism in the context of long-term data transmission. Consequently, the research proposes to extend the applicability of the OPC UA in the context of image transmission. Although highly needed, the image transmission after processing is currently beyond the reach of OPC UA or other legacy industrial protocols, being considered as a separate fraction in the industrial environment. The concept and developments are applied considering both the end-of-line industrial manufacturing process in the automotive sector and the car-to-infrastructure communication. Without special hardware constraints, the obtained results are proven to be appreciable, opening various future perspectives for image transmission using OPC UA.

show abstract

Overview and Empirical Analysis of ISP Parameter Tuning for Visual Perception in Autonomous Driving

Cited by 32 publications

References 83 publications

Optimization of the ISP Parameters of a Camera Through Differential Evolution

Optimization of the ISP Parameters of a Camera Through Differential Evolution

Image Pre-Processing Method of Machine Learning for Edge Detection with Image Signal Processor Enhancement

Approaching OPC UA Publish–Subscribe in the Context of UDP-Based Multi-Channel Communication and Image Transmission

Contact Info

Product

Resources

About