Label Efficient Semi-Supervised Learning via Graph Filtering

Abstract

Graph-based methods have been demonstrated as one of the most effective approaches for semi-supervised learning, as they can exploit the connectivity patterns between labeled and unlabeled data samples to improve learning performance. However, existing graph-based methods either are limited in their ability to jointly model graph structures and data features, such as the classical label propagation methods, or require a considerable amount of labeled data for training and validation due to high model complexity, such as the recent neural-network-based methods. In this paper, we address label efficient semi-supervised learning from a graph filtering perspective. Specifically, we propose a graph filtering framework that injects graph similarity into data features by taking them as signals on the graph and applying a low-pass graph filter to extract useful data representations for classification, where label efficiency can be achieved by conveniently adjusting the strength of the graph filter. Interestingly, this framework unifies two seemingly very different methods -label propagation and graph convolutional networks. Revisiting them under the graph filtering framework leads to new insights that improve their modeling capabilities and reduce model complexity. Experiments on various semi-supervised classification tasks on four citation networks and one knowledge graph and one semi-supervised regression task for zero-shot image recognition validate our findings and proposals.

Adaptive Graph Convolution Using Heat Kernel for Attributed Graph Clustering

“…Equation 7 [10] assigns higher importance to high-frequency basis signals and discounts graph smoothness. GCN [3] defines graph convolution based on the frequency function g λ q = 1 − λ q [12], which cannot perform better low-pass characteristics as g λ q is negative for 1 < λ q ≤ 2. The frequency response function in AGC [7] is g λ q = 1 − 1 2 λ q , which exploits a linear low-pass filter to suppress the high-frequency signal; however, it might not highlight low-frequency signals adequately and cannot determine the appropriate neighborhood that reflects the relevant information of smoothness.…”

Knowledge and Cross-Pair Pattern Guided Semantic Matching for Question Answering

“…where Y P is the indices of labeled Q-A pairs, Y is the label indicator matrix, and F is the dimension of the output, which is equal to the number of classes. In order to increase flexibility and label efficiency, we apply Auto-Regressive (AR) filter to get an improved GCN (IGCN) (Li et al 2019), which replacesà withà :…”

“…and k = 4α is enough according to (Li et al 2019). IGCN can achieve label efficiency by using the exponent k to conveniently adjust the filter strength.…”

“…A graph is built on a whole QA corpus (under-sampled on TrecQA) to capture cross-pair patterns (summarized in Table 1) with labels of the validation and testing sets masked following (Yao, Mao, and Luo 2019). Since negative answers are less useful for learning explicit Q-A patterns, and may introduce noise during propagation (if not randomly selected, typical wrong Q-A patterns may also help), we also apply sample masks on training data to tune the negative sampling rate r. We set the filter parameter α = 10 for WikiQA and α = 1 for TrecQA according to label rates (Li et al 2019). GCN applies the first-order convolutional filter to integrate graph and feature information.…”

“…For stacked GCN, the hidden layer size is set to 64. Adam (Kingma and Ba 2014) is adopted for training and the model with the lowest training loss in 400 steps is selected (Li et al 2019). Other hyperparameters are shown in Table 2.…”