Most modern maximum likelihood multiple target tracking systems (e.g., Multiple Hypothesis Tracking (MHT) and Numerica's Multiple Frame Assignment (MFA)) need to determine how to separate their input measurements into subsets corresponding to the observations of individual targets. These observation sets form the tracks of the system, and the process of determining these sets is known as data association. Real-time constraints frequently force the use of only the maximum likelihood choice for data association (over some time window), although alternative data association choices may have been considered in the process of choosing the most likely. This paper presents a Tracker Adjunct Processing (TAP) system that captures and manages the uncertainty encountered in making data association decisions. The TAP combines input observation data and the data association alternatives considered by the tracker into a dynamic Bayesian network (DBN). The network efficiently represents the combined alternative tracking hypotheses. Bayesian network evidence propagation methods are used to update the network in light of new evidence, which may consist of new observations, new alternative data associations, newly received late observations, hypothetical connections, or other flexible queries. The maximum likelihood tracking hypothesis can then be redetermined, which may result in changes to the best tracking hypothesis. The recommended changes can then be communicated back to the associated tracking system, which can then update its tracks. In this manner, the TAP's interpretation makes the firm, fixed (formerly maximum likelihood) decisions of the tracker "softer," i.e., less absolute. The TAP can also assess (and reassess) track purity regions by ambiguity level.We illustrate the working of the TAP with several examples, one in particular showing the incorporation of critical, late or infrequent data. These data are critical in the sense that they are very valuable in resolving ambiguities in tracking and combat identification; thus, the motivation to use these data is high even though there are complexities in applying it. Some data may be late because of significant network delays, while other data may be infrequently reported because they come from "specialized" sensors that provide updates only every once in a while.

A common problem in video-based tracking of urban targets is occlusion due to buildings and vehicles. Fortunately, when multiple video sensors are present with enough geometric diversity, track breaks due to temporary occlusion can be substantially reduced by correlating and fusing source-level track data into system-level tracks. Furthermore, when operating in a communication-constrained environment, it is preferable to transmit track data rather than either raw video data or detection measurements. To avoid statistical correlation due to common prior information, tracklets can be formed from the source tracks prior to transmission to a central command node, which is then responsible for system track maintenance via correlation and fusion. To maximize the operational benefit of the system-level track picture, it should be distributed in an efficient manner to all platforms, especially the local trackers at the sensors. In this paper, we describe a centralized architecture for multi-sensor video tracking that uses tracklet-based feedback to maintain an accurate and complete track picture at all platforms. We will also use challenging synthetic video data to demonstrate that our architecture improves track completeness, enhances track continuity (in the presence of occlusions), and reduces track initiation time at the local trackers.

No abstract