EnsembleLens: Ensemble-based Visual Exploration of Anomaly Detection Algorithms with Multidimensional Data

Abstract

The results of anomaly detection are sensitive to the choice of detection algorithms as they are specialized for different properties of data, especially for multidimensional data. Thus, it is vital to select the algorithm appropriately. To systematically select the algorithms, ensemble analysis techniques have been developed to support the assembly and comparison of heterogeneous algorithms. However, challenges remain due to the absence of the ground truth, interpretation, or evaluation of these anomaly detectors. In this paper, we present a visual analytics system named EnsembleLens that evaluates anomaly detection algorithms based on the ensemble analysis process. The system visualizes the ensemble processes and results by a set of novel visual designs and multiple coordinated contextual views to meet the requirements of correlation analysis, assessment and reasoning of anomaly detection algorithms. We also introduce an interactive analysis workflow that dynamically produces contextualized and interpretable data summaries that allow further refinements of exploration results based on user feedback. We demonstrate the effectiveness of EnsembleLens through a quantitative evaluation, three case studies with real-world data and interviews with two domain experts.

A Survey of Human‐Centered Evaluations in Human‐Centered Machine Learning

“…While papers describe interactions at varying levels of detail, a few common interaction types emerged in our analysis, including selection (e.g., [KBJ∗20; WBL∗20]), filtering (e.g., [LGM∗20; PNKC20]), zooming (e.g., [LPH∗20; LSC∗18]), tuning weights/hyperparameters (e.g., [DSKE20; SLC∗20]), and annotation/labeling (e.g., [SSKE19; XXM∗19]). Multiple of these interaction types tend to be combined in a single system, for example when users first select a set of data instances before labeling them (e.g., [BHZ∗18; ZWLC19]).…”

Outliagnostics: Visualizing Temporal Discrepancy in Outlying Signatures of Data Entries

“…There are several works using visual interfaces and interactions to explore and validate the outlying data. To name a few, En-sembLens [40] applies ensemble analysis; Viola [10] is based on Canonical Polyadic (CP) decomposition methods with tensor-based anomaly analysis algorithm; TargetVue [11] uses TLOF [8]; Rclens adopts active learning algorithm to identify rare category; CVExplorer [33], MTDES [35], and TimeMatrix [14] use visual overview with supported interactions for discovery and exploratory of data patterns. Zhang et al [42] also provided a good survey of visualization for network anomalies.…”

The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

“…As a result, many papers are published in this area [AEM11, BLBC12, JHB*17, WLS19], making this topic class with 30 papers one of the most prominent in our analysis. Topic 8 – models’ predictions & design prototyping. Another generic topic class with 18 related papers contains, among others, the subject of ML models’ predictions [SJS*17, XXM*19] that has already been seen in Topic 5. The difference between this class and Topic 5 is the focus of its related papers, which is on the instantiation of visualization prototypes with different design choices that should be carefully considered based on previous InfoVis research.…”