Conserving analyst attention units: use of multi-agent software and CEP methods to assist information analysis

Abstract

Although the capability of computer-based artificial intelligence techniques for decision-making and situational awareness has seen notable improvement over the last several decades, the current state-of-the-art still falls short of creating computer systems capable of autonomously making complex decisions and judgments in many domains where data is nuanced and accountability is high. However, there is a great deal of potential for hybrid systems in which software applications augment human capabilities by focusing the analyst's attention to relevant information elements based on both a priori knowledge of the analyst's goals and the processing/correlation of a series of data streams too numerous and heterogeneous for the analyst to digest without assistance.Researchers at Penn State University are exploring ways in which an information framework influenced by Klein's (Recognition Primed Decision) RPD model, Endsley's model of situational awareness, and the Joint Directors of Laboratories (JDL) data fusion process model can be implemented through a novel combination of Complex Event Processing (CEP) and Multi-Agent Software (MAS). Though originally designed for stock market and financial applications, the high performance data-driven nature of CEP techniques provide a natural compliment to the proven capabilities of MAS systems for modeling naturalistic decision-making, performing process adjudication, and optimizing networked processing and cognition via the use of "mobile agents." This paper addresses the challenges and opportunities of such a framework for augmenting human observational capability as well as enabling the ability to perform collaborative context-aware reasoning in both human teams and hybrid human / software agent teams.

A situation refinement model for complex event processing

“…As the CEP technology becomes a core component in many application areas nowadays, several related studies have focused on the adaptation of the JDL data fusion model into CEP [38][39][40][41]. Despite partial success in this direction, though, the JDL model's applicability to the CEP paradigm faces two major challenges: First, the process of enriching CEP detection rules with external domain knowledge requires an efficient information-fusion mechanism capable of processing a vast amount of information in near real-time [10,27], and second, the information fusion process must take into account the limited time available to process information from knowledge sources that are heterogeneous by nature, such as ontologies, linked data, etc.…”

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

“…Other revisions focused on the fusion, impact, and communication between hard data (machine or environmental data) and soft data (human data). Changes in specific levels such as Levels 1 and 2 incorporating the human in the fusion and preprocessing levels [54], subsequently in Levels 4 and 5 through hybrid-sensing and hybrid-cognition [55], or even the incorporating a management network in parallel to the JDL model [56,57] were proposed.…”