Supporting robust system analysis with the test matrix tool framework

Abstract

We present the Test Matrix Tool (TMT) framework, a simulationagnostic framework providing end-to-end support for robust analysis of complex systems. The need to execute a large number of simulations is common to many problem environments, even those already reduced by Design of Experiments or similar methodologies. TMT addresses key end-user needs in easing the specification, execution and analysis of simulation workloads in ways that are consistent between specific applications of the framework. The TMT design contributes modular specifications for key data communicated between and within the specification, execution and analysis components. Our TMT implementation is an instantiation of those formats freely available for general use. TMT's data analysis component provides a variety of featuresdata filtering, comparison, transformation and visualization-for analytic tasks on any TMT-embedded model. We provide a brief case study as an example of its use in a real-world application.

Estimating field-of-view loss in bathymetric lidar: application to large-scale simulations

“…Because variance must be estimated separately at each path length, the required number of simulations is well over 1000. To analyze the results of such large-scale simulations we employ a specialized software-testing environment, the Test Matrix Tool (TMT), to organize the computations [12]. TMT generates the input files, runs the waveform simulator, processes the waveforms, and writes the ranging estimates for each case.…”

Lytic

“…Lytic is based on the data analysis component of a modeling and simulation management framework [11], and thus its origins are in analyzing large, high-dimensional simulation datasets for scientific and engineering applications. As a result, we think of it as a sort of scientific intelligence tool, complementing the existing business intelligence tools.…”