No abstract

No abstract

Current needs in tactical situational awareness require a new type of infrastructure to encode, transmit, store, fuse, and display vastly heterogeneous data that may include "hard" sensor types including video, radar, multispectral, acoustic sensor array, 3D flash LIDAR, and "soft" sensor inputs such as textual reports from trained and untrained personnel, unsolicited and solicited open source web information, and hybrid "hard/soft" data such as human-annotated image or video data -which can be highly useful, but difficult to categorize and exploit.While the demand for scalability, rapid deployment, and decentralized access to data and services grows, the need for data security and integrity is as critical as ever. Methods for handling the conflicting needs between access and security are addressed. Furthermore, the evolving role of humans in data fusion systems must be addressed by the infrastructure. In addition to systems enhancing human data analysis capabilities through advanced visualization and sonification techniques, the data itself is more likely to contain information about humans -which is not always a task well suited to conventional data storage and retrieval methods.This paper describes a multi-agent approach to designing a secure, distributed, service-oriented infrastructure to support human-centric hard and soft information fusion.

No abstract

Although the capability of computer-based artificial intelligence techniques for decision-making and situational awareness has seen notable improvement over the last several decades, the current state-of-the-art still falls short of creating computer systems capable of autonomously making complex decisions and judgments in many domains where data is nuanced and accountability is high. However, there is a great deal of potential for hybrid systems in which software applications augment human capabilities by focusing the analyst's attention to relevant information elements based on both a priori knowledge of the analyst's goals and the processing/correlation of a series of data streams too numerous and heterogeneous for the analyst to digest without assistance.Researchers at Penn State University are exploring ways in which an information framework influenced by Klein's (Recognition Primed Decision) RPD model, Endsley's model of situational awareness, and the Joint Directors of Laboratories (JDL) data fusion process model can be implemented through a novel combination of Complex Event Processing (CEP) and Multi-Agent Software (MAS). Though originally designed for stock market and financial applications, the high performance data-driven nature of CEP techniques provide a natural compliment to the proven capabilities of MAS systems for modeling naturalistic decision-making, performing process adjudication, and optimizing networked processing and cognition via the use of "mobile agents." This paper addresses the challenges and opportunities of such a framework for augmenting human observational capability as well as enabling the ability to perform collaborative context-aware reasoning in both human teams and hybrid human / software agent teams.