Towards assurance evaluation of autonomous systems

Beland, Steven C.; Chang, Isaac; Chen, Alexander; Moser, Matthew; Paunicka, James; Stuart, Douglas A.; Vian, John; Westover, Christina; Yu, Huafeng

doi:10.1145/3400302.3415785

Cited by 9 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate our algorithm for extracting feature representations and the downstream analyses using two networks trained for computer vision tasks, namely TaxiNet [4,9], a regression model for center-line tracking on airport runways, and YOLOv4-Tiny [14], an object detection model trained on the nuImages [6] dataset for autonomous driving.…”

Section: Introductionmentioning

confidence: 99%

Feature-Guided Analysis of Neural Networks

Gopinath

Lungeanu²,

Mangal

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Applying standard software engineering practices to neural networks is challenging due to the lack of high-level abstractions describing a neural network’s behavior. To address this challenge, we propose to extract high-level task-specific features from the neural network internal representation, based on monitoring the neural network activations. The extracted feature representations can serve as a link to high-level requirements and can be leveraged to enable fundamental software engineering activities, such as automated testing, debugging, requirements analysis, and formal verification, leading to better engineering of neural networks. Using two case studies, we present initial empirical evidence demonstrating the feasibility of our ideas.

show abstract

Section: Introductionmentioning

confidence: 99%

Feature-Guided Analysis of Neural Networks

Gopinath

Lungeanu²,

Mangal

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Yel et al [89] provide a runtime monitoring technique using neural networks for safe motion planning. In the U.S. government sector, a Boeing team as part of the DARPA assured autonomy program, implemented runtime monitoring in a flight simulator [11]. For an overview of runtime monitoring techniques, the reader is directed to [90].…”

Section: A Runtime Monitoringmentioning

confidence: 99%

“…Lastly, investments such as the DARPA Assured Autonomy program 1 seeks to continually assure learning enabled cyber-physical systems by constructing formal methods that assure correctness at design time and perform runtime monitoring at operation time. While this program has advanced the state of the art in formal methods [10] and runtime monitoring [11], a systematic approach to identify outstanding gaps will remain unclear unless the community makes explicit and coordinated efforts to understand how such methods may be incorporated into the broader systems engineering process.…”

Section: Introductionmentioning

confidence: 99%

A Mapping of Assurance Techniques for Learning Enabled Autonomous Systems to the Systems Engineering Lifecycle

Ellis

Wigness

Fiondella

2022

2022 IEEE International Conference on Assured Autonomy (ICAA)

View full text Add to dashboard Cite

Learning enabled autonomous systems provide increased capabilities compared to traditional systems. However, the complexity of and probabilistic nature in the underlying methods enabling such capabilities present challenges for current systems engineering processes for assurance, and test, evaluation, verification, and validation (TEVV). This paper provides a preliminary attempt to map recently developed technical approaches in the assurance and TEVV of learning enabled autonomous systems (LEAS) literature to a traditional systems engineering v-model. This mapping categorizes such techniques into three main approaches: development, acquisition, and sustainment. We review the latest techniques to develop safe, reliable, and resilient learning enabled autonomous systems, without recommending radical and impractical changes to existing systems engineering processes. By performing this mapping, we seek to assist acquisition professionals by (i) informing comprehensive test and evaluation planning, and (ii) objectively communicating risk to leaders.

show abstract

“…In this work, we describe a formal analysis of a closed-loop autonomous system that addresses the above challenges. Our case study is motivated by a real-world application, namely, an experimental autonomous system for guiding airplanes on taxiways developed by Boeing [2,13].…”

Section: Introductionmentioning

confidence: 99%

“…Velocity may be provided as feedback to the controller; we ignore here for simplicity 2. Assuming the relevant state of the world is recoverable from the input image.…”

mentioning

confidence: 99%

Closed-loop Analysis of Vision-based Autonomous Systems: A Case Study

Păsăreanu¹,

Mangal²,

Gopinath³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) are increasingly used in safetycritical autonomous systems as perception components processing highdimensional image data. Formal analysis of these systems is particularly challenging due to the complexity of the perception DNNs, the sensors (cameras), and the environment conditions. We present a case study applying formal probabilistic analysis techniques to an experimental autonomous system that guides airplanes on taxiways using a perception DNN. We address the above challenges by replacing the camera and the network with a compact probabilistic abstraction built from the confusion matrices computed for the DNN on a representative image data set. We also show how to leverage local, DNN-specific analyses as run-time guards to increase the safety of the overall system. Our findings are applicable to other autonomous systems that use complex DNNs for perception.

show abstract

Towards assurance evaluation of autonomous systems

Cited by 9 publications

References 0 publications

Feature-Guided Analysis of Neural Networks

Feature-Guided Analysis of Neural Networks

A Mapping of Assurance Techniques for Learning Enabled Autonomous Systems to the Systems Engineering Lifecycle

Closed-loop Analysis of Vision-based Autonomous Systems: A Case Study

Contact Info

Product

Resources

About