Characterization of Deep Learning-Based Aerial Explosive Hazard Detection using Simulated Data

Alvey, Brendan; Anderson, Derek T.; Yang, Clare; Buck, Andrew; Keller, James M.; Yasuda, Ken; Ryan, Hollie A.

doi:10.1109/ssci50451.2021.9659899

Cited by 6 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our research conducted at the University of Missouri extensively leverages the UE across diverse areas of study. Notably, UE has played a pivotal role in our exploration of explainable AI, [12][13][14][15] procedural simulation for AI, [16][17][18] workflows enhancing computer vision, 17,[19][20][21][22][23] and multi-criteria decision making. 24,25 Additionally, UE has been instrumental in specific applications such as explosive hazard detection 14,18,26,27 and passive ranging.…”

Section: Proposed Frameworkmentioning

confidence: 99%

“…Notably, UE has played a pivotal role in our exploration of explainable AI, [12][13][14][15] procedural simulation for AI, [16][17][18] workflows enhancing computer vision, 17,[19][20][21][22][23] and multi-criteria decision making. 24,25 Additionally, UE has been instrumental in specific applications such as explosive hazard detection 14,18,26,27 and passive ranging. 28,29 While UE has served as a cornerstone in investigating these topics, the solutions developed are often tailored to specific applications and pose scalability challenges due to their reliance on local desktop computing.…”

Section: Proposed Frameworkmentioning

confidence: 99%

“…The reader could just as easily measure performance of YOLO on all vehicles in this scene if desired. Furthermore, in a related study, 14 we (Alvey et al) conducted a comprehensive analysis of performance concerning the relative viewing angles between the platform and the object. However, for the purposes of this article, delving into such granular detail was deemed unnecessary.…”

Section: Workflow 2: Multiple Agent Livesmentioning

confidence: 99%

See 2 more Smart Citations

MizSIM: a headless open-source simulation framework for training and evaluating artificial intelligence

Soloviov,

Anderson,

Kerley

et al. 2024

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications II

View full text Add to dashboard Cite

Advancements in deep learning have revolutionized the artificial intelligence (AI) landscape. However, despite considerable performance enhancements, their reliance on data and the intrinsic opacity of these models remains a challenge, hindering our ability to understand the reasons behind their failures. This paper introduces a headless open-source framework, coined MizSIM, built on the Unreal Engine (UE) to generate high volume and variety synthetic datasets for AI training and evaluation. Through the manipulation of agent and environment parameters, MizSIM can provide detailed performance analysis and failure diagnosis. Leveraging UE’s open-source distribution, cost-effective assets, and high-quality graphics, along with tools like AirSim and the Robotic Operating System (ROS), MizSIM ensures user-friendly design and seamless data extraction. In this article, we demonstrate two MizSIM workflows: one for a single-life computer vision task and the other to evaluate an object detector across hundreds of simulated lives. The overarching aim is to establish a closed-loop environment to enhance AI effectiveness and transparency.

show abstract

Section: Proposed Frameworkmentioning

confidence: 99%

Section: Proposed Frameworkmentioning

confidence: 99%

Section: Workflow 2: Multiple Agent Livesmentioning

confidence: 99%

See 1 more Smart Citation

MizSIM: a headless open-source simulation framework for training and evaluating artificial intelligence

Soloviov,

Anderson,

Kerley

et al. 2024

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications II

View full text Add to dashboard Cite

show abstract

“…f or e xplainable A I ( XAI). In, 47 we performed a camera-object relative scan to mimic a low altitude drone moving around a point of interest (potential target/object). This dataset was then subjected to a DL object detector built using real world data.…”

Section: Uc1: Object-centricmentioning

confidence: 99%

How should simulated data be collected for AI/ML and unmanned aerial vehicles?

Kerley

Anderson

Alvey

et al. 2023

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications

View full text Add to dashboard Cite

Large and diverse datasets can now be simulated with associated truth to train and evaluate AI/ML algorithms. This convergence of readily accessible simulation (SIM) tools, real-time high-performance computing, and large repositories of high-quality, free-to-inexpensive photorealistic scanned assets is a potential artificial intelligence (AI) and machine learning (ML) gamechanger. While this feat is now within our grasp, what SIM data should be generated, how should it be generated, and how can this be achieved in a controlled and scalable fashion? First, we discuss a formal procedural language for specifying scenes (LSCENE) and collecting sampled datasets (LCAP). Second, we discuss specifics regarding our production and storage of data, ground truth, and metadata. Last, two LSCENE/LCAP examples are discussed and three unmanned aerial vehicle (UAV) AI/ML use cases are provided to demonstrate the range and behavior of the proposed ideas. Overall, this article is a step towards closed-loop automated AI/ML design and evaluation.

show abstract

“…While these tools are not a perfect simulation or a replacement for the real-world, their level of realism is high and multiple works are already showing how they can be used as is to train and evaluate AI/ML algorithms [ 28 , 29 , 30 , 31 , 32 , 33 ]. In [ 34 ], we showed how this imagery can be used to systematically evaluate model performance. Through simulation, we are able to output very detailed descriptions for each image in our data set.…”

Section: Introductionmentioning

confidence: 99%

Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery

Alvey

Anderson

Keller

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Deep learning has become increasingly common in aerial imagery analysis. As its use continues to grow, it is crucial that we understand and can explain its behavior. One eXplainable AI (XAI) approach is to generate linguistic summarizations of data and/or models. However, the number of summaries can increase significantly with the number of data attributes, posing a challenge. Herein, we proposed a hierarchical approach for generating and evaluating linguistic statements of black box deep learning models. Our approach scores and ranks statements according to user-specified criteria. A systematic process was outlined for the evaluation of an object detector on a low altitude aerial drone. A deep learning model trained on real imagery was evaluated on a photorealistic simulated dataset with known ground truth across different contexts. The effectiveness and versatility of our approach was demonstrated by showing tailored linguistic summaries for different user types. Ultimately, this process is an efficient human-centric way of identifying successes, shortcomings, and biases in data and deep learning models.

show abstract

Characterization of Deep Learning-Based Aerial Explosive Hazard Detection using Simulated Data

Cited by 6 publications

References 12 publications

MizSIM: a headless open-source simulation framework for training and evaluating artificial intelligence

MizSIM: a headless open-source simulation framework for training and evaluating artificial intelligence

How should simulated data be collected for AI/ML and unmanned aerial vehicles?

Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery

Contact Info

Product

Resources

About