Procedurally generated simulated datasets for aerial explosive hazard detection

Kerley, Jeffrey; Fuller, Aaron B.; Kovaleski, Madeline; Popescu, Peter; Alvey, Brendan; Anderson, D. M.; Buck, Andrew; Keller, James M.; Scott, Grant; Yang, Clare; Yasuda, Ken; Ryan, Hollie

doi:10.1117/12.2618798

Cited by 6 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While our primary emphasis lay in a person detector, we also ventured into exploring the training, testing, and evaluation of more unique and infrequent objects like explosive hazards (EH) [ 16 , 42 , 43 , 44 ] using both single and multiple modalities (RGB and infrared). It is essential to note that the specific object category, whether it is a person, EH, car, or other, is not the central focus of this article.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…LLMs can also be used in combination with other techniques to generate novel photorealistic imagery from relatively simple, natural language text prompts [ 13 , 14 ]. Language models can be integrated with sophisticated rendering systems, such as the Unreal Engine, to create diverse photorealistic imagery with a simple set of instructions [ 15 , 16 ]. In this paper, we demonstrated an XAI system for generating textual descriptions of an object detection model.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery

Alvey

Anderson

Keller

et al. 2023

Sensors

View full text Add to dashboard Cite

Deep learning has become increasingly common in aerial imagery analysis. As its use continues to grow, it is crucial that we understand and can explain its behavior. One eXplainable AI (XAI) approach is to generate linguistic summarizations of data and/or models. However, the number of summaries can increase significantly with the number of data attributes, posing a challenge. Herein, we proposed a hierarchical approach for generating and evaluating linguistic statements of black box deep learning models. Our approach scores and ranks statements according to user-specified criteria. A systematic process was outlined for the evaluation of an object detector on a low altitude aerial drone. A deep learning model trained on real imagery was evaluated on a photorealistic simulated dataset with known ground truth across different contexts. The effectiveness and versatility of our approach was demonstrated by showing tailored linguistic summaries for different user types. Ultimately, this process is an efficient human-centric way of identifying successes, shortcomings, and biases in data and deep learning models.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery

Alvey

Anderson

Keller

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Our research conducted at the University of Missouri extensively leverages the UE across diverse areas of study. Notably, UE has played a pivotal role in our exploration of explainable AI, [12][13][14][15] procedural simulation for AI, [16][17][18] workflows enhancing computer vision, 17,[19][20][21][22][23] and multi-criteria decision making. 24,25 Additionally, UE has been instrumental in specific applications such as explosive hazard detection 14,18,26,27 and passive ranging.…”

Section: Proposed Frameworkmentioning

confidence: 99%

“…Notably, UE has played a pivotal role in our exploration of explainable AI, [12][13][14][15] procedural simulation for AI, [16][17][18] workflows enhancing computer vision, 17,[19][20][21][22][23] and multi-criteria decision making. 24,25 Additionally, UE has been instrumental in specific applications such as explosive hazard detection 14,18,26,27 and passive ranging. 28,29 While UE has served as a cornerstone in investigating these topics, the solutions developed are often tailored to specific applications and pose scalability challenges due to their reliance on local desktop computing.…”

Section: Proposed Frameworkmentioning

confidence: 99%

MizSIM: a headless open-source simulation framework for training and evaluating artificial intelligence

Soloviov,

Anderson,

Kerley

et al. 2024

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications II

View full text Add to dashboard Cite

Advancements in deep learning have revolutionized the artificial intelligence (AI) landscape. However, despite considerable performance enhancements, their reliance on data and the intrinsic opacity of these models remains a challenge, hindering our ability to understand the reasons behind their failures. This paper introduces a headless open-source framework, coined MizSIM, built on the Unreal Engine (UE) to generate high volume and variety synthetic datasets for AI training and evaluation. Through the manipulation of agent and environment parameters, MizSIM can provide detailed performance analysis and failure diagnosis. Leveraging UE’s open-source distribution, cost-effective assets, and high-quality graphics, along with tools like AirSim and the Robotic Operating System (ROS), MizSIM ensures user-friendly design and seamless data extraction. In this article, we demonstrate two MizSIM workflows: one for a single-life computer vision task and the other to evaluate an object detector across hundreds of simulated lives. The overarching aim is to establish a closed-loop environment to enhance AI effectiveness and transparency.

show abstract

“…The reader can refer to our article on "How Should Simulated Data Be Collected for AI/ML and Unmanned Aerial Vehicles" for full details about how we produce RGB, depth, object IDs, and other data layers. 25…”

Section: Simulation Environmentmentioning

confidence: 99%

Simulated gold-standard for quantitative evaluation of monocular vision algorithms

Akers

Buck

Camaioni

et al. 2023

Geospatial Informatics XIII

View full text Add to dashboard Cite

In the physical universe, truth for computer vision (CV) is impractical if not impossible to obtain. As a result, the CV community has resorted to qualitative practices and sub-optimal quantitative measures. This is problematic because it limits our ability to train, evaluate, and ultimately understand algorithms such as single image depth estimation (SIDE) and structure from motion (SfM). How good are these algorithms, individually and relatively, and where do they break? Herein, we discuss that while truth evades both the real and simulated (SIM) universes, a SIM CV gold-standard can be achieved. We outline an extensible SIM framework and data collection workflow using unreal engine with the Robot Operating System (ROS) for three-dimensional mapping on low altitude aerial vehicles. Furthermore, voxel-based mapping measures from algorithm output to a SIM gold-standard are discussed. The proposed metrics are demonstrated by analyzing performance across changes in platform context. Ultimately, the current article is a step towards an improved process for comparing algorithms, evaluating their strengths and weaknesses, and automating algorithm design.

show abstract

Procedurally generated simulated datasets for aerial explosive hazard detection

Cited by 6 publications

References 0 publications

Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery

Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery

MizSIM: a headless open-source simulation framework for training and evaluating artificial intelligence

Simulated gold-standard for quantitative evaluation of monocular vision algorithms

Contact Info

Product

Resources

About