Performance Assessment of Intelligence, Surveillance, and Reconnaissance (ISR) Enterprise Workflows

Abstract

One of the challenges of developing complex systems is the question of how to measure system performance. Good engineering practice is to define a single compound measure of system performance, called the figure of merit (FOM), and to select design options by their effects upon the figure of merit. The Joint Services of the Department of Defense (DoD) are in the process of transitioning legacy computing applications for intelligence, surveillance, and reconnaissance (ISR) tasks to standards-based information enterprises. This process is occurring with very little theoretical or practical understanding of the appropriate FOMs needed to assess the performance of these large information systems. The danger of developing systems without clearly stated and understood FOMs is that important but difficult to measure system characteristics are overlooked in favor of less important but easier to measure system characteristics. As a result, operational users, system developers, and program managers who rely on accurate performance information are left with considerable uncertainty regarding how to best use, improve, and deploy these systems.This report presents an end-to-end assessment framework for C4ISR enterprises that identifies potential FOMs and thereby eliminates a significant fraction of this uncertainty. The framework delineates how measurements from laboratory tests, operational experiments, and field deployments can be analyzed to produce accurate, quantitative descriptions of system performance. These FOMs not only inform the development and acquisition processes but also allow military users to ensure that they are optimizing the information enterprise to achieve improved operational capability. In fact, active participation by the military user community in defining appropriate FOMs is essential to providing users with systems that meet their operational needs. An analysis of a simple ISR enterprise is conducted to show how FOMs might be calculated and used to assess the relative merit of different communications architectures for the construction of a common operational picture (COP) among a collection of distributed sensors.111.. TABLE OF CONTENTS Abstract iiiList of Illustrations vii