Semi-supervised learning using randomized mincuts

Abstract

In many application domains there is a large amount of unlabeled data but only a very limited amount of labeled training data. One general approach that has been explored for utilizing this unlabeled data is to construct a graph on all the data points based on distance relationships among examples, and then to use the known labels to perform some type of graph partitioning. One natural partitioning to use is the minimum cut that agrees with the labeled data (Blum & Chawla, 2001), which can be thought of as giving the most probable label assignment if one views labels as generated according to a Markov Random Field on the graph. Zhu et al. (2003) propose a cut based on a relaxation of this field, and Joachims (2003) gives an algorithm based on finding an approximate min-ratio cut.In this paper, we extend the mincut approach by adding randomness to the graph structure. The resulting algorithm addresses several shortcomings of the basic mincut approach, and can be given theoretical justification from both a Markov random field perspective and from sample complexity considerations. In cases where the graph does not have small cuts for a given classification problem, randomization may not help. However, our experiments on several datasets show that when the structure of the graph supports small cuts, this can result in highly accurate classifiers with good accuracy/coverage tradeoffs. In addition, we are able to achieve good performance with a very simple graph-construction procedure.

Robust and Scalable Graph-Based Semisupervised Learning

“…The small portion of vertices carrying seed labels are then harnessed via graph partition or information propagation to predict the labels for the unlabeled vertices. For instance, the graph mincuts approach formulated GSSL as a graph cut problem [3], [4]. Other GSSL methods such as graph transduction formulated GSSL as a regularized function estimation over the graph.…”

“…The weighted graph, producing the optimal label prediction function, essentially propagates the initial label information from the labeled samples to the vast amount of unlabeled ones. Many popular GSSL algorithms including graph cuts [3], [4], [25], [31], graph-based random walks [1], [45], manifold regularization [2], [42], and graph transduction [61], [64] have been proposed. Comprehensive surveys of these methods can be found in [6] and [63].…”

Semi-Supervised Sequence Labeling with Self-Learned Features

“…Graph-based semi-supervised approaches make use of dependencies introduced between the labels of nearby examples on a constructed graph [22], [29]. These models train to encourage nearby data points to have the same class labels.…”

Learning Unknown Graphs

“…This approach guarantees that the number of mistakes is bounded by a quantity that depends linearly on the cutsize Φ G (y). Further results involving the prediction of node labels in graphs with known structure include [5,6,7,8,9,10,11,12,13,14]. Since all these papers assume knowledge of the entire graph in advance, the techniques proposed for transductive binary prediction do not have any mechanism for guiding the exploration of the graph, hence they do not work well on the exploration/prediction problem studied in this work.…”