Embedded out-of-distribution detection on an autonomous robot platform

Yuhas, Michael; Feng, Yeli; Ng, Daniel Jun Xian; Rahiminasab, Zahra; Easwaran, Arvind

doi:10.1145/3445034.3460509

Cited by 11 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under A3, we assume that either a binary classification result of 1 or an OOD detection result of 1 will trigger a hazard avoidance action. This reflects the use of OOD detectors in prior works [1], [2], [9] and leads to two possible failure modes, E = {E 0 , E 1 }, where:…”

Section: Problem Definition a Risk Minimization Over Design Parametersmentioning

confidence: 97%

“…Our goal is to design a system that minimizes risk as defined in (1), such that the average resource utilization does not exceed that of the baseline as shown in (2).…”

Section: Problem Definition a Risk Minimization Over Design Parametersmentioning

confidence: 99%

“…If the OOD detector does not yield a decision before its deadline or returns a false negative result, it provides no protection. Furthermore, when the OOD detector shares the same computational resource (like an embedded GPU) with an LEC, it interferes with the LEC's ability to meet deadlines [2]. Additionally, false positives from the OOD detector can affect the system's availability, leading to a decrease in performance [3].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems

Yuhas

Easwaran

2022

2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)

Self Cite

View full text Add to dashboard Cite

Learning enabled components (LECs), while critical for decision making in autonomous vehicles (AVs), are likely to make incorrect decisions when presented with samples outside of their training distributions. Out-of-distribution (OOD) detectors have been proposed to detect such samples, thereby acting as a safety monitor, however, both OOD detectors and LECs require heavy utilization of embedded hardware typically found in AVs. For both components, there is a tradeoff between non-functional and functional performance, and both impact a vehicle's safety. For instance, giving an OOD detector a longer response time can increase its accuracy at the expense of the LEC. We consider an LEC with binary output like an autonomous emergency braking system (AEBS) and use risk, the combination of severity and occurrence of a failure, to model the effect of both components' design parameters on each other's functional and non-functional performance, as well as their impact on system safety. We formulate a codesign methodology that uses this risk model to find the design parameters for an OOD detector and LEC that decrease risk below that of the baseline system and demonstrate it on a vision based AEBS. Using our methodology, we achieve a 42.3% risk reduction while maintaining equivalent resource utilization.

show abstract

Section: Problem Definition a Risk Minimization Over Design Parametersmentioning

confidence: 97%

“…Our goal is to design a system that minimizes risk as defined in (1), such that the average resource utilization does not exceed that of the baseline as shown in (2).…”

Section: Problem Definition a Risk Minimization Over Design Parametersmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems

Yuhas

Easwaran

2022

2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…• Digital twin models in Cyber-Physical Manufacturing Systems (CPMS) [73], [74] • Analog Twin (AT) Framework for Human and AI Supervisory Control [75] • Experimentation in a remote laboratory setting [76] • Multi-robot localization and mapping [23], [77], [78] • Learning from observation [25] • Mobile manipulator positioning for object pick-up [26] • Navigation and obstacle avoidance [35], [36], [79]- [85] • Autonomous exploration in indoor environments [30], [86]- [88] • Immersive telepresence [89] • The specific application was not mentioned [27]- [29], [32], [53], [90]-[95] Healthcare…”

Section: Researchmentioning

confidence: 99%

Emerging Trends in Realistic Robotic Simulations: A Comprehensive Systematic Literature Review

Kargar,

Yordanov,

Harvey

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Simulation plays a pivotal role in providing safely reproducible scenarios to evaluate the ever-advancing domain of computer science and robotics. It was an essential part of the pandemic when no access to physical spaces was available. The advent of AI-powered platforms in conjunction with enhanced graphics, physics and other sensory engines attracts a new breed of interdisciplinary researchers to enter the robotic field, most notably from computer science, engineering and social sciences. Integration of ROS as a uniform middleware to deploy achieved outcomes in real practice provides an opportunity to move one step closer to the sim-to-real experiences that enables researchers to test ideas beyond the close laboratory spaces. There is a lack of comprehensive evaluation of ROS-enabled simulators, and the integration of advanced AI techniques for realistic scenario replication. This paper addresses this challenge by evaluating ROSenabled simulators in the design and implementation of AI techniques through an in-depth systematic literature review (SLR). This SLR is guided by the research and commercial market demands, employing Population, Intervention, Comparison, Outcome, and Context (PICOC) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) frameworks with a major focus on Wheeled Mobile Robots (WMRs). We also highlight the increasing importance of game engines like Unity and Unreal in future of robotic simulations, especially under modelling close to real experiences. By comparing simulation platform features and capabilities, this paper offers guidance to developers and researchers, enabling them to select the most suitable platform for their projects efficiently that contradicts the commonly in use "one size fits all" approach. Finally, based on the thorough insights from the review, we identify and suggest some key future research directions in AI-enhanced realistic robotic simulations.

show abstract

“…However, their main purprose is the evaluation of different OOD detection algorithms, and not an application of OOD detection to real-world operation. Yuhas et al [20] evaluate OOD detection as an emergency breaking system for an autonomous car, though not in real-world operation, but on a custom test track.…”

Section: Related Workmentioning

confidence: 99%

Out-of-Distribution Detection for Adaptive Computer Vision

Lind

Triebel

Nardi

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

It is well known that computer vision can be unreliable when faced with previously unseen imaging conditions. This paper proposes a method to adapt camera parameters according to a normalizing flow-based out-of-distibution detector. A small-scale study is conducted which shows that adapting camera parameters according to this out-ofdistibution detector leads to an average increase of 3 to 4 percentage points in mAP, mAR and F1 performance metrics of a YOLOv4 object detector. As a secondary result, this paper also shows that it is possible to train a normalizing flow model for out-of-distribution detection on the COCO dataset, which is larger and more diverse than most benchmarks for out-of-distibution detectors.

show abstract

Embedded out-of-distribution detection on an autonomous robot platform

Cited by 11 publications

References 13 publications

Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems

Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems

Emerging Trends in Realistic Robotic Simulations: A Comprehensive Systematic Literature Review

Out-of-Distribution Detection for Adaptive Computer Vision

Contact Info

Product

Resources

About