Neural Graph Learning

Abstract

Label propagation is a powerful and flexible semi-supervised learning technique on graphs. Neural networks, on the other hand, have proven track records in many supervised learning tasks. In this work, we propose a training framework with a graph-regularised objective, namely Neural Graph Machines, that can combine the power of neural networks and label propagation. This work generalises previous literature on graph-augmented training of neural networks, enabling it to be applied to multiple neural architectures (Feed-forward NNs, CNNs and LSTM RNNs) and a wide range of graphs. The new objective allows the neural networks to harness both labeled and unlabeled data by: (a) allowing the network to train using labeled data as in the supervised setting, (b) biasing the network to learn similar hidden representations for neighboring nodes on a graph, in the same vein as label propagation. Such architectures with the proposed objective can be trained efficiently using stochastic gradient descent and scaled to large graphs, with a runtime that is linear in the number of edges. The proposed joint training approach convincingly outperforms many existing methods on a wide range of tasks (multi-label classification on social graphs, news categorization, document classification and semantic intent classification), with multiple forms of graph inputs (including graphs with and without node-level features) and using different types of neural networks. CCS CONCEPTS • Computing methodologies → Neural networks; Semi-supervised learning settings;

Adversarial Graph Learning and Deep Learning Techniques for improving diagnosis within CT and Ultrasound images

“…By incorporating adversarial examples into the training procedure of the neural network, the robustness of the model was proven to be increased. Bui et al [24] propose a training framework containing a graph-regularized objective, which achieved state-of-the art performance for multi-label classification on social graphs, news categorization, semantic intent classification, and document classification. Da-Cheng et al [37] show a graph-learning framework that allows to obtain embeddings capable of discriminating between 40 million semantic labels.…”

“…Neural Structured Learning (NSL) leverages structured signals along with feature input for training neural networks. NSL generalizes to Adversarial Learning when neighbors are generated by adversarial perturbation [23], or to Neural Graph Learning when the neighbors are explicitly represented with a graph [24].…”

Human Activity Recognition Using Wearable Sensors, Discriminant Analysis, and Long Short-Term Memory-based Neural Structured Learning

“…It can be utilized for training taking the advantage of structured signals related to the feature inputs. NSL is a neural graph learning approach to train neural networks depending on graphs and structured data [30]. NSL also generalizes basic adversarial learning [31] utilizing the structured data with good relational information among the samples.…”

“…The first one is by using neural graph learning where neighbors are connected by a graph. The second one is by utilizing adversarial learning where the neighbors are induced by the adversarial perturbation [30].…”

HyperKvasir, A Comprehensive Multi-Class Image and Video Dataset for Gastrointestinal Endoscopy

“…This technique is known as semi-supervised learning and has lately been successfully applied in different medical image analyses 42 . Examples of semi-supervised learning are self-learning 43,44 and neural graph learning 45 , which both make use of unlabeled data in addition to a small number of labeled data to extract additional information 43,44,46 . We believe these new algorithms might be the development needed to make AI even more useful for medical applications.…”