The Impact of Meta-Information on Decision-Making in Intelligence Operations

Abstract

Decision-making in complex, dynamic, high-risk environments is clearly challenging. Part of this challenge is due to the presence of qualifiers of information, or meta-information (e.g., staleness, uncertainty, source), that alter a person's information processing, situational awareness, and decision-making. We investigated the influence of meta-information on decision-making in a Military Intelligence Operations (IO) environment using Cognitive Task Analysis (CTA) techniques. We performed a CTA on IO tasks surrounding the use of smart sensor webs, a relatively new technology that can be used for a variety of IO purposes. Our analysis addressed information management tasks and tactical decision-making tasks using sensor webs. We discovered that a variety of types of meta-information significantly impacted decision-making, and that the influence of meta-information was both context- and task- sensitive. In this paper, we present the results of the CTA and discuss the implications for the development of decision-aiding systems, including the design of constituent displays, interfaces, and automated systems.

Framing and Contextualizing Information Requests: Problem Formulation as Part of the Intelligence Analysis Process

“…Metainformation refers to qualifiers of information, such as the uncertainty associated with the information, the age of the information, and the source of the information (Bisantz et al, 2009;Pfautz et al, 2005). Study 1 indicated that reasoning about metainformation is a key cognitive challenge in intelligence analysis, as analysts must consider not only how to collect information to satisfy an information request but also how to collect credible/reliable-enough information and maximally relevant information, all within the time constraints of the original request.…”

On reliability and trustworthiness of high-level fusion-based decision support systems: basic concepts and possible formal methodologies

“…There are many applications, of belief functions and possibility measures in particular, in which it is misleading to regard a set of probability distributions as a set of hypotheses about the "correct" probabilities, because it is meaningless to talk of "correct" probabilities. 6. On the relationship between Bayesian inference and evidence theory Although many papers discuss the abstract relationships between different approaches to uncertainty management, such papers are less abundant that compare the precision and robustness of different approaches in specific application examples.…”

“…[5,6], few studies discuss basic reliability concepts in high-level fusion-based decision support systems (HLFB-DSS) and applications. In such systems, generic conceptual architectures such as Bayesian networks [7], decision trees, influence diagrams [8], and Bayesian games [9] are frequently adopted, while different approaches to uncertainty management are often combined [2,3,10].…”

Cognitive Complexities Impacting Army Intelligence Analysis

“…We therefore differentiate our research from a variety of other work addressing intelligence analysis more generally (e.g., (Bodnar, 2005;Pirolli & Card, 2005;Elm et al, 2005)). This general research has been used to guide the development of computational tools to support the Intelligence community, ranging from automated knowledge extraction tools (Boschee, Weischedel, & Zamanian, 2005) to high-level decision support methods (Sticha, Buede, & Rees, 2005;Pfautz et al, 2005a) to numerous data fusion techniques (Gardner & Uhlmann, 2005;Waltz, 1986) to particular display and interface designs (Horn, 2005;Pfautz et al, 2005b). This research has also led to the development of methodologies and metrics for evaluating human performance both with and without those tools (Morse, Potts Steves, & Scholtz, 2005).…”