On failures of RGB cameras and their effects in autonomous driving applications

Secci, Francesco; Ceccarelli, Andrea

doi:10.1109/issre5003.2020.00011

Cited by 33 publications

(30 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Amongst the various ML-based applications that exist for Carla, in our work we prefer a self-driving agent over other agents e.g., object recognition agents. In fact, a self-driving agent allows showing the effect of persistent faults or continuous attacks applied on consequential images, rather than on individual images without a continuous context [4]. Amongst self-driving agents, we select the trained agent Learning by Cheating (LbC) developed by Chen et al [3].…”

Section: Carla Simulator and Learning By Cheating (Lbc)mentioning

confidence: 99%

See 1 more Smart Citation

Attack and Fault Injection in Self-driving Agents on the Carla Simulator – Experience Report

Piazzesi

Massimo

Ceccarelli

2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Machine Learning applications are acknowledged at the foundation of autonomous driving, because they are the enabling technology for most driving tasks. However, the inclusion of trained agents in automotive systems exposes the vehicle to novel attacks and faults, that can result in safety threats to the driving tasks. In this paper we report our experimental campaign on the injection of adversarial attacks and software faults in a self-driving agent running in a driving simulator. We show that adversarial attacks and faults injected in the trained agent can lead to erroneous decisions and severely jeopardize safety. The paper shows a feasible and easily-reproducible approach based on open source simulator and tools, and the results clearly motivate the need of both protective measures and extensive testing campaigns.

show abstract

Section: Carla Simulator and Learning By Cheating (Lbc)mentioning

confidence: 99%

“…The failure criteria is whenever the vehicle collides or the timeout expires. We include a modification of the NoCrash benchmark that halts the run whenever a collision occurs [4], because in our work we prioritize safety over travelled distance.…”

Section: Description Of the Experimental Campaignmentioning

confidence: 99%

Attack and Fault Injection in Self-driving Agents on the Carla Simulator – Experience Report

Piazzesi

Massimo

Ceccarelli

2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…This work is an extended version of [78]: we detail the five most noticeable additional contributions with respect to [78], in what we believe is increasing order of relevance. To facilitate the reader, we also explicitly mention the related Sections of this work.…”

Section: Introductionmentioning

confidence: 99%

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Secci

Ceccarelli

2023

IEEE Trans. Dependable and Secure Comput.

Self Cite

View full text Add to dashboard Cite

RGB cameras are one of the most relevant sensors for autonomous driving applications. It is undeniable that failures of vehicle cameras may compromise the autonomous driving task, possibly leading to unsafe behaviors when images that are subsequently processed by the driving system are altered. To support the definition of safe and robust vehicle architectures and intelligent systems, in this paper we define the failure modes of a vehicle camera, together with an analysis of effects and known mitigations. Further, we build a software library for the generation of the corresponding failed images, and we feed them to six object detectors for mono and stereo cameras and to the self-driving agent of an autonomous driving simulator. The resulting misbehaviors with respect to operating with clean images allow a better understanding of failures effects and the related safety risks in image-based applications.

show abstract

“…A factor that can affect this complexity is image distortion. Cameras can gather images with various distortion types, such as blur and noise [3]. The adaptive offloading scenario of Fig.…”

Section: Introductionmentioning

confidence: 99%

Early-exit deep neural networks for distorted images: providing an efficient edge offloading

Pacheco¹,

Oliveira²,

Couto³

2021

Preprint

View full text Add to dashboard Cite

Edge offloading for deep neural networks (DNNs) can be adaptive to the input's complexity by using earlyexit DNNs. These DNNs have side branches throughout their architecture, allowing the inference to end earlier in the edge. The branches estimate the accuracy for a given input. If this estimated accuracy reaches a threshold, the inference ends on the edge. Otherwise, the edge offloads the inference to the cloud to process the remaining DNN layers. However, DNNs for image classification deals with distorted images, which negatively impact the branches' estimated accuracy. Consequently, the edge offloads more inferences to the cloud. This work introduces expert side branches trained on a particular distortion type to improve robustness against image distortion. The edge detects the distortion type and selects appropriate expert branches to perform the inference. This approach increases the estimated accuracy on the edge, improving the offloading decisions. We validate our proposal in a realistic scenario, in which the edge offloads DNN inference to Amazon EC2 instances. 1

show abstract

On failures of RGB cameras and their effects in autonomous driving applications

Cited by 33 publications

References 42 publications

Attack and Fault Injection in Self-driving Agents on the Carla Simulator – Experience Report

Attack and Fault Injection in Self-driving Agents on the Carla Simulator – Experience Report

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Early-exit deep neural networks for distorted images: providing an efficient edge offloading

Contact Info

Product

Resources

About