Interactive Correction of Mislabeled Training Data

Abstract

Figure 1: DataDebugger: (a) the navigation stack records the explored hierarchical levels; (b) the tSNE-based visualization shows the item distribution and the confusion between different classes; (c)(d) the selected item view and the trusted item view display the images of selected items and trusted items, respectively; (e) The action trail records the correction history. ABSTRACTIn this paper, we develop a visual analysis method for interactively improving the quality of labeled data, which is essential to the success of supervised and semi-supervised learning. The quality improvement is achieved through the use of user-selected trusted items. We employ a bi-level optimization model to accurately match the labels of the trusted items and to minimize the training loss. Based on this model, a scalable data correction algorithm is developed to handle tens of thousands of labeled data efficiently. The selection of the trusted items is facilitated by an incremental tSNE with improved computational efficiency and layout stability to ensure a smooth transition between different levels. To prioritize the display of mislabeled data, we have taken specific consideration of outliers, items whose labels are different from those of its neighboring items, in the sampling process. We evaluated our method on real-world datasets through quantitative evaluation and case studies, and the results were generally favorable.

Interactive Graph Construction for Graph-Based Semi-Supervised Learning

“…To handle data quality problems not restricted to crowdsourcing settings, Xiang et al [18] developed a scalable data correction algorithm to propagate the labels of trusted items to other unverified items. A hierarchical visualization was proposed to facilitate the exploration and identification of trusted items.…”

“…To allow efficient exploration of a large number of samples, we organize the samples in a hierarchy. An uncertainty-biased sampling method [18], [59] is used to build the hierarchy. This sampling method takes both region density and classification uncertainty into consideration and increases the sampling ratio from dense regions with low uncertainty to sparse regions with high uncertainty.…”

Pyramid-based Scatterplots Sampling for Progressive and Streaming Data Visualization

“…Methods. We compare our method with seven sampling methods: random sampling (RS), blue noise sampling (BNS) [11], density-based sampling (DBS) [32], non-uniform sampling (NUS) [5], outlier biased density-based sampling (OBDBS) [46], multi-view z-order sampling (MVZS) [24], and KD-tree-based sampling (KBS) [10]. Though KBS and MVZS are designed for multi-class sampling, we use them on single-class data without multi-class constraints.…”

A survey of visual analytics techniques for machine learning

“…For example, the ImageNet dataset [277] only contains the labels cleaned by automatic noise removal methods. To tackle these datasets, Xiang et al [43] developed DataDebugger to interactively improve data quality by utilizing user-selected trusted items. Hierarchical visualization combined with an incremental projection method and an outlier biased sampling method facilitates the exploration and identification of trusted items.…”