Generalization and Representational Limits of Graph Neural Networks

Garg, Vikas K.; Jegelka, Stefanie; Jaakkola, Tommi

doi:10.48550/arxiv.2002.06157

Cited by 6 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, various methods were proposed to focus on some graph characteristics, which could not be captured by GCNNs (e.g. longest circle in [12]).…”

Section: Graph Neural Networkmentioning

confidence: 99%

Deep Learning Techniques for In-Crop Weed Identification: A Review

Hu,

Wang,

Coleman

et al. 2021

Preprint

View full text Add to dashboard Cite

Weeds are a significant threat to the agricultural productivity and the environment. The increasing demand for sustainable agriculture has driven innovations in accurate weed control technologies aimed at reducing the reliance on herbicides. With the great success of deep learning in various vision tasks, many promising image-based weed detection algorithms have been developed. This paper reviews recent developments of deep learning techniques in the field of image-based weed detection. The review begins with an introduction to the fundamentals of deep learning related to weed detection. Next, recent progresses on deep weed detection are reviewed with the discussion of the research materials including public weed datasets. Finally, the challenges of developing practically deployable weed detection methods are summarized, together with the discussions of the opportunities for future research. We hope that this review will provide a timely survey of the field and attract more researchers to address this inter-disciplinary research problem.

show abstract

“…Recently, various methods were proposed to focus on some graph characteristics, which could not be captured by GCNNs (e.g. longest circle in [12]).…”

Section: Graph Neural Networkmentioning

confidence: 99%

Deep Learning Techniques for In-Crop Weed Identification: A Review

Hu,

Wang,

Coleman

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In theory (Zaheer et al 2017, Theorem 2), if COMB 1,2 are given 'sufficient' hidden units, this set representation is universal. In practice, however, commutative-associative aggregation suffers from limited expressiveness (Pabbaraju and Jain 2019;Wagstaff et al 2019;Garg, Jegelka, and Jaakkola 2020;Cohen-Karlik, David, and Globerson 2020), which degrades the quality of x u and s(•, •), as described below. Specifically, their expressiveness is constrained from two perspectives.…”

Section: Gnns and Their Limitationsmentioning

confidence: 99%

Adversarial Permutation Guided Node Representations for Link Prediction

Roy

Chakrabarti

2021

AAAI

View full text Add to dashboard Cite

After observing a snapshot of a social network, a link prediction (LP) algorithm identifies node pairs between which new edges will likely materialize in future. Most LP algorithms estimate a score for currently non-neighboring node pairs, and rank them by this score. Recent LP systems compute this score by comparing dense, low dimensional vector representations of nodes. Graph neural networks (GNNs), in particular graph convolutional networks (GCNs), are popular examples. For two nodes to be meaningfully compared, their embeddings should be indifferent to reordering of their neighbors. GNNs typically use simple, symmetric set aggregators to ensure this property, but this design decision has been shown to produce representations with limited expressive power. Sequence encoders are more expressive, but are permutation sensitive by design. Recent efforts to overcome this dilemma turn out to be unsatisfactory for LP tasks. In response, we propose PermGNN, which aggregates neighbor features using a recurrent, order-sensitive aggregator and directly minimizes an LP loss while it is `attacked' by adversarial generator of neighbor permutations. PermGNN has superior expressive power compared to earlier GNNs. Next, we devise an optimization framework to map PermGNN's node embeddings to a suitable locality-sensitive hash, which speeds up reporting the top-K most likely edges for the LP task. Our experiments on diverse datasets show that PermGNN outperforms several state-of-the-art link predictors by a significant margin, and can predict the most likely edges fast.

show abstract

“…We use similar arguments to show that GNNs have enough expressive power to solve a task on a set of small graphs and to fail on it on a set of large graphs. Several works studied generalization bounds for certain classes of GNNs [Garg et al, 2020, Puny et al, 2020, Verma and Zhang, 2019], but did not discuss size-generalization. Sinha et al [2020] proposed a benchmark for assessing the logical generalization abilities of GNNs.…”

Section: Related Workmentioning

confidence: 99%

From Local Structures to Size Generalization in Graph Neural Networks

Yehudai¹,

Fetaya²,

Meirom³

et al. 2020

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) can process graphs of different sizes but their capacity to generalize across sizes is still not well understood. Size generalization is key to numerous GNN applications, from solving combinatorial optimization problems to learning in molecular biology. In such problems, obtaining labels and training on large graphs can be prohibitively expensive, but training on smaller graphs is possible.This paper puts forward the size-generalization question and characterizes important aspects of that problem theoretically and empirically. We show that even for very simple tasks, GNNs do not naturally generalize to graphs of larger size. Instead, their generalization performance is closely related to the distribution of patterns of connectivity and features and how that distribution changes from small to large graphs. Specifically, we show that in many cases, there are GNNs that can perfectly solve a task on small graphs but generalize poorly to large graphs and that these GNNs are encountered in practice. We then formalize size generalization as a domain-adaption problem and describe two learning setups where size generalization can be improved. First, as a self-supervised learning problem (SSL) over the target domain of large graphs. Second, as a semi-supervised learning problem when few samples are available in the target domain. We demonstrate the efficacy of these solutions on a diverse set of benchmark graph datasets.

show abstract

Generalization and Representational Limits of Graph Neural Networks

Cited by 6 publications

References 13 publications

Deep Learning Techniques for In-Crop Weed Identification: A Review

Deep Learning Techniques for In-Crop Weed Identification: A Review

Adversarial Permutation Guided Node Representations for Link Prediction

From Local Structures to Size Generalization in Graph Neural Networks

Contact Info

Product

Resources

About