Achieving Fairness" in Data Fusion Performance Evaluation Development"

Abstract

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of informatiol including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jeffersoi Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing 1 comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM-YYYY)2.REPORT TYPE DATES COVERED (From -To SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S)USAF SUPPLEMENTARY NOTES ABSTRACTThe key issue for and evaluation (T&E) organization is how to affordably achieve fairness in the application of its PE systems.Our PE framework provides a methodology to accomplish this; viz., the DNN Data Fusion & Resource Management (DF&RM) framework provides the hierarchical PE components for PE solution space and a methodology for mapping PE solution space int various PE problem spaces. The scope of this fairness study for performance evaluation of data fusion (DF) systems is to define philosophy of fairness that is defendable as a basis for developing a PE system. Sample PE system MoEs need to be defined to understand the PE problem space, PE solution space and the PE problem-to-solution space mapping (i.e., the 'rules' to map the alternative PE system design solutions to the needed "Fir" PE capability). Implicitly, we are seeking design guidelines for a "best" PE that balances affordability with fairness as defined above.The first technical contribution of this report is in the reusabl and extendable PE solution framework within which all applications-layer approaches to PE known to the authors can be expressed. As such, this PE framework exposes PE system design alternatives to the PE system developer and provides a common framework within which alternative PE systems can be comp~ared. SUBJECT TERMS SECURITY IntroductionIn our previous work (Refs 1, 2, 3) we have been studying the definition, design, and prototyping of formal methods for the performance evaluation (PE) of Data Fusion systems. That work shows that there are in any given case alternative PE strategies that could be employed. One rationale for the down-selection of a preferred approach among such PE candidates would be to choose one which is most "fair" or equitable in comparison to the others. Since many PE cases involve comparisons among competing approaches, an equitable approach has an inherent appeal in general. Fair is defined as something "marked by impa...