FLAG: Faster Learning on Anchor Graph with Label Predictor Optimization

Abstract

Abstract-Knowledge graphs have received intensive research interests. When the labels of most nodes or datapoints are missing, anchor graph and hierarchical anchor graph models can be employed. With an anchor graph or hierarchical anchor graph, we only need to optimize the labels of the coarsest anchors, and the labels of datapoints can be inferred from these anchors in a coarse-tofine manner. The complexity of optimization is therefore reduced to a cubic cost with respect to the number of the coarsest anchors. However, to obtain a high accuracy when a data distribution is complex, the scale of this anchor set still needs to be large, which thus inevitably incurs an expensive computational burden. As such, a challenge in scaling up these models is how to efficiently estimate the labels of these anchors while keeping classification performance. To address this problem, we propose a novel approach that adds an anchor label predictor in the conventional anchor graph and hierarchical anchor graph models. In the proposed approach, the labels of the coarsest anchors are not directly optimized, and instead, we learn a label predictor which estimates the labels of these anchors with their spectral representations. The predictor is optimized with a regularization on all datapoints based on a hierarchical anchor graph, and we show that its solution only involves the inversion of a small-size matrix. Built upon the anchor hierarchy, we design a sparse intra-layer adjacency matrix over these anchors, which can simultaneously accelerate spectral embedding and enhance effectiveness. Our approach is named Faster Learning on Anchor Graph (FLAG) as it improves conventional anchor-graph-based methods in terms of efficiency. Experiments on a variety of publicly available datasets with sizes varying from thousands to millions of samples demonstrate the effectiveness of our approach.

A Survey on Large-Scale Machine Learning

“…Graph-based Models label propagation on sparse graph approximate search [110], [227], [232], division and conquer [45], [197]. optimization with anchor graph single layer [132], [201], [228], hierarchical layers [77], [200].…”

“…After that, hierarchical anchor graphs propose to retain sparse similarities over all instances while keeping a small number of anchors for label inference [200]. In case that the smallest set of anchors still needs to be large and brings considerable computations, FLAG developes label optimizers for further acceleration [77]. Besides, EAGR proposes to perform label smoothness over anchors with pruned adjacency [201].…”

“…The reason is that, once a set of anchors can be stored in the memory, anchor graphs can be efficiently constructed with the memory cost of O(md+nk) rather than O(nd). For example, with hierarchical anchor graphs, the classification on 8 million instances could be implemented on a personal computer within 2 mins [77], [200].…”

Sub-Graph Regularization on Kernel Regression for Robust Semi-Supervised Dimensionality Reduction

“…As a result, the computational complexity can be greatly reduced. While there are different ways to build the adjacency matrix S in AGR [24][25][26], we argue that most of them are developed intuitively and lack a probability explanation. In addition, AGR cannot directly infer the class labels of incoming data.…”