OpenGraphGym: A Parallel Reinforcement Learning Framework for Graph Optimization Problems

Abstract

This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new opensource RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.

Combinatorial optimization and reasoning with graph neural networks

“…Conversely, libraries to simplify the usage of machine learning in CO have also been developed. OR-Gym [60] and OpenGraphGym [152] are libraries designed to facilitate the learning of heuristics for CO problems in a similar interface to the popular OpenAI Gym library [14]. In contrast, MIPLearn [142] is a library that facilitates the learning of con guration parameters for CO solvers.…”

graphenv: a Python library for reinforcement learning on graph search spaces

“…Related software efforts have addressed parts of the above need. OpenGraphGym (Zheng et al, 2020) implements RL-based stragies for common graph optimization challenges such as minimum vertex cover or maximum cut, but does not interface with external RL libraries and has minimal documentation. Ecole (Prouvost et al, 2020) provides an OpenAI-like gym environment for combinatorial optimization, but intends to operate in concert with traditional mixed integer solvers rather than directly exposing the environment to an RL agent.…”

Graph-based reinforcement learning for discrete cross-section optimization of planar steel frames

“…There are a number of methods that combine RL and GE [34][35][36][37][38]; however, they are limited to the problem of selecting nodes.…”