Goal-directed graph construction using reinforcement learning

Abstract

Graphs can be used to represent and reason about systems and a variety of metrics have been devised to quantify their global characteristics. However, little is currently known about how to construct a graph or improve an existing one given a target objective. In this work, we formulate the construction of a graph as a decision-making process in which a central agent creates topologies by trial and error and receives rewards proportional to the value of the target objective. By means of this conceptual framework, we propose an algorithm based on reinforcement learning and graph neural networks to learn graph construction and improvement strategies. Our core case study focuses on robustness to failures and attacks, a property relevant for the infrastructure and communication networks that power modern society. Experiments on synthetic and real-world graphs show that this approach can outperform existing methods while being cheaper to evaluate. It also allows generalization to out-of-sample graphs, as well as to larger out-of-distribution graphs in some cases. The approach is applicable to the optimization of other global structural properties of graphs.

Planning spatial networks with Monte Carlo tree search

“…Namely, Darvariu et al . [12] reports a wall clock time that is equivalent to 56 h of a single core of a comparable CPU to train a model on graphs of size N=20 and, due to the complexity of the problem, does not train models directly beyond graphs with N=50. By contrast, on similar computational infrastructure, our proposed SG-UCT requires 11 h on average to optimize a much larger graph with N=200 nodes.…”

“…In particular, we use the definition in [15], which considers the size of the LCC as nodes are removed from the network. We consider only the targeted attack case as previous work has found it is more challenging [12,37]. We define the robustness measure as scriptFRfalse(Gfalse)=double-struckEξfalse[false(1/Nfalse)∑i=1Ns(G,ξ,i)false], where sfalse(G,ξ,ifalse) denotes the fraction of nodes in the LCC of G after the removal of the first i nodes in the permutation ξ (in which nodes appear in descending order of their degrees).…”

“…In the machine learning community, model-free RL techniques have been applied for deriving adversarial examples for graph-based classifiers by changing the network structure [13] and the goal-directed generation of molecular graphs [10]. [12] formulated the goal-directed construction of a graph as an MDP and proposed a method based on RL and graph neural networks, showing some advantages over prior methods in the network science literature in terms of its ability to optimize the objective and the fast runtime of the policy once it has been trained.…”

“…In particular, we use the definition in [15], which considers the size of the LCC as nodes are removed from the network. We consider only the targeted attack case as previous work has found it is more challenging [12,37]. We define the robustness measure as…”

SCGG: A deep structure-conditioned graph generative model

“…In addition to the methods mentioned above that are more related to our proposed approach, there are other categories of modern graph generation approaches, the most noteworthy of which are autoencoder-based methods [ 18 , 38 – 42 ], RL-based approaches [ 43 – 45 ], GAN-based generating strategies [ 15 , 19 , 46 ], and flow-based models [ 47 , 48 ].…”