A joint manifold leaning-based framework for heterogeneous upstream data fusion

Shen, Dan; Blasch, Erik; Zulch, Peter; DiStasio, Marcello; Niu, Ruixin; Lu, Jingyang; Wang, Zhonghai; Chen, Genshe

doi:10.1177/1748301818791507

Cited by 37 publications

(9 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of an autoencoder-based dynamic deep directional unit network [ 24 ] was capable of learning compact and abstract feature representations from high-dimensional spatiotemporal data of full motion video and I/Q data for the purposes of event behavior characterization. Other research into achieving EO/RF fusion for vehicle tracking and detection using Full Motion Video and P-RF includes joint manifold learning [ 25 ], a sheaf-based approach with its data [ 26 ], and SVM classifier [ 23 ]. In [ 25 , 26 ], simulation data were used as the primary method of training and testing, while in [ 23 ] real data were used.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Other research into achieving EO/RF fusion for vehicle tracking and detection using Full Motion Video and P-RF includes joint manifold learning [ 25 ], a sheaf-based approach with its data [ 26 ], and SVM classifier [ 23 ]. In [ 25 , 26 ], simulation data were used as the primary method of training and testing, while in [ 23 ] real data were used. In [ 25 ], a joint manifold learning fusion approach was used for mixed simulation data using DIRSIG-generated data.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Vakil

Blasch

Ewing

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate the utilization of a Long Short-Term Memory (LSTM-CCA) model for the fusion of Passive RF (P-RF) and Electro-Optical (EO) data in order to gain insights into how P-RF data are utilized. The P-RF data are constructed from the in-phase and quadrature component (I/Q) data processed via histograms, and are combined with enhanced EO data via dense optical flow (DOF). The preprocessed data are then used as training data with an LSTM-CCA model in order to achieve object detection and tracking. In order to determine the impact of the different data inputs, a greedy algorithm (explainX.ai) is implemented to determine the weight and impact of the canonical variates provided to the fusion model on a scenario-by-scenario basis. This research introduces an explainable LSTM-CCA framework for P-RF and EO sensor fusion, providing novel insights into the sensor fusion process that can assist in the detection and differentiation of targets and help decision-makers to determine the weights for each input.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Vakil

Blasch

Ewing

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…G zn . Any number of distance-graph-based upstream fusion techniques (e.g, similarity network fusion [25], [23] or joint manifold learning [7], [21]) could then be used to produce a fused weighed graph G f . The final step of LESS could then be applied to produce the fused event sequence.…”

Section: Towards Less As a Fusion Techniquementioning

confidence: 99%

Graph Spectral Embedding for Parsimonious Transmission of Multivariate Time Series

Yao

Bendich

2019

Preprint

View full text Add to dashboard Cite

We propose a graph spectral representation of time series data that 1) is parsimoniously encoded to user-demanded resolution; 2) is unsupervised and performant in data-constrained scenarios; 3) captures event and event-transition structure within the time series; and 4) has near-linear computational complexity in both signal length and ambient dimension. This representation, which we call Laplacian Events Signal Segmentation (LESS), can be computed on time series of arbitrary dimension and originating from sensors of arbitrary type. Hence, time series originating from sensors of heterogeneous type can be compressed to levels demanded by constrained-communication environments, before being fused at a common center.Temporal dynamics of the data is summarized without explicit partitioning or probabilistic modeling. As a proof-of-principle, we apply this technique on high dimensional wavelet coefficients computed from the Free Spoken Digit Dataset to generate a memory efficient representation that is interpretable. Due to its unsupervised and non-parametric nature, LESS representations remain performant in the digit classification task despite the absence of labels and limited data.

show abstract

“…7 Combing EO with radar signatures allows for machine processing of multiresolution data with sparsity and complexity. 8 These data fusion methods afford interpretability of data for task success. Examples include interpretability over compressed imagery data, 9 3D volumetric lidar data, 10 and classifier assessment.…”

Section: Introductionmentioning

confidence: 99%

Targeted adversarial discriminative domain adaptation

Chen

Savakis

Diehl

et al. 2021

J. Appl. Rem. Sens.

Self Cite

View full text Add to dashboard Cite

Domain adaptation is a technology enabling aided target recognition and other algorithms for environments and targets with data or labeled data that is scarce. Recent advances in unsupervised domain adaptation have demonstrated excellent performance but only when the domain shift is relatively small. We proposed targeted adversarial discriminative domain adaptation (T-ADDA), a semi-supervised domain adaptation method that extends the ADDA framework. By providing at least one labeled target image per class, used as a cue to guide the adaption, T-ADDA significantly boosts the performance of ADDA and is applicable to the challenging scenario in which the sets of targets in the source and target domains are not the same. The efficacy of T-ADDA is demonstrated by cross-domain, cross-sensor, and cross-target experiments using the common digits datasets and several aerial image datasets. Results demonstrate an average increase of 15% improvement with T-ADDA over ADDA using just a few labeled images when adapting to a small domain shift and afforded a 60% improvement when adapting to large domain shifts. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

show abstract

A joint manifold leaning-based framework for heterogeneous upstream data fusion

Cited by 37 publications

References 41 publications

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Graph Spectral Embedding for Parsimonious Transmission of Multivariate Time Series

Targeted adversarial discriminative domain adaptation

Contact Info

Product

Resources

About