XLVIN: eXecuted Latent Value Iteration Nets

Deac, Andreea; Veličković, Petar; Milinković, Ognjen; Bacon, Pierre-Luc; Tang, J.; Nikolić, Mladen

doi:10.48550/arxiv.2010.13146

Cited by 3 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As in other areas of machine learning, RL has seen increasing interest in forgoing the use of explicit models, instead structuring the policy to include a planning inductive bias such that an agent can perform implicit planning (Tamar et al, 2016;Deac et al, 2020;Amos et al, 2018;Jin et al, 2020). A classic example is value iteration networks (Tamar et al, 2016), which replace the explicit value iteration algorithm with an inductive bias in the form of a convolutional neural network (Fukushima, 1988;LeCun et al, 1989).…”

Section: End-to-end Sysid and Controlmentioning

confidence: 99%

Myriad: a real-world testbed to bridge trajectory optimization and deep learning

Howe¹,

Dufort-Labbé²,

Rajkumar³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present Myriad, a testbed written in JAX for learning and planning in real-world continuous environments. The primary contributions of Myriad are threefold. First, Myriad provides machine learning practitioners access to trajectory optimization techniques for application within a typical automatic differentiation workflow. Second, Myriad presents many real-world optimal control problems, ranging from biology to medicine to engineering, for use by the machine learning community. Formulated in continuous space and time, these environments retain some of the complexity of real-world systems often abstracted away by standard benchmarks. As such, Myriad strives to serve as a stepping stone towards application of modern machine learning techniques for impactful real-world tasks. Finally, we use the Myriad repository to showcase a novel approach for learning and control tasks. Trained in a fully end-to-end fashion, our model leverages an implicit planning module over neural ordinary differential equations, enabling simultaneous learning and planning with complex environment dynamics.

show abstract

Section: End-to-end Sysid and Controlmentioning

confidence: 99%

Myriad: a real-world testbed to bridge trajectory optimization and deep learning

Howe¹,

Dufort-Labbé²,

Rajkumar³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Through the lens of algorithmic alignment (Xu et al, 2019), GNNs can be constructed that closely mimic iterative computation (Veličković et al, 2019;, linearithmic sequence processing (Freivalds et al, 2019), and pointer-based data structures . Also such approaches are capable of strongly generalising (Yan et al, 2020) and data-efficient planning (Deac et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Persistent Message Passing

Strathmann,

Barekatain,

Blundell

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Graph neural networks (GNNs) are a powerful inductive bias for modelling algorithmic reasoning procedures and data structures. Their prowess was mainly demonstrated on tasks featuring Markovian dynamics, where querying any associated data structure depends only on its latest state. For many tasks of interest, however, it may be highly beneficial to support efficient data structure queries dependent on previous states. This requires tracking the data structure's evolution through time, placing significant pressure on the GNN's latent representations. We introduce Persistent Message Passing (PMP), a mechanism which endows GNNs with capability of querying past state by explicitly persisting it: rather than overwriting node representations, it creates new nodes whenever required. PMP generalises out-of-distribution to more than 2× larger test inputs on dynamic temporal range queries, significantly outperforming GNNs which overwrite states.

show abstract

“…Specifically, the XLVIN architecture [Deac et al, 2020] is an exact instance of our blueprint for the VI algorithm. Besides improved data efficiency over more traditional approaches to RL, it also compared favourably against ATreeC [Farquhar et al, 2017], which attempts to directly apply VI in a neural pipeline, thus encountering the algorithmic bottleneck problem in low-data regimes.…”

mentioning

confidence: 99%

Neural Algorithmic Reasoning

Veličković,

Blundell

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another. We argue that algorithms possess fundamentally different qualities to deep learning methods, and this strongly suggests that, were deep learning methods better able to mimic algorithms, generalisation of the sort seen with algorithms would become possible with deep learning-something far out of the reach of current machine learning methods. Furthermore, by representing elements in a continuous space of learnt algorithms, neural networks are able to adapt known algorithms more closely to real-world problems, potentially finding more efficient and pragmatic solutions than those proposed by human computer scientists.Here we present neural algorithmic reasoning-the art of building neural networks that are able to execute algorithmic computation-and provide our opinion on its transformative potential for running classical algorithms on inputs previously considered inaccessible to them.Algorithms and Deep Learning Algorithms are pervasive in modern societyfrom elevators, microwave ovens and other household equipment to procedures for electing government officials. Algorithms allow us to automate and engineer systems that reason. Remarkably, algorithms applied in one domain-such as a microwave oven-may be slightly adjusted and deployed in a completely different domain-such as a heart pacemaker (e.g., a control algorithm such as PID). That is not to say that you would expect to be able to safely run a microwave oven using a pacemaker (or vice versa) without modification, but the same recipe underlies both constructions.An undergraduate textbook on algorithms [Cormen et al., 2009] will cover fewer than 60 distinct algorithms. A subset of these will serve as the useful basis for someone's life-long career in software engineering in almost any domain. Part of the skill of a software engineer lies in choosing which algorithm to use, when, and in combination with what else. Only rarely will an entirely novel algorithm be warranted.This same algorithmic basis could also help us solve one of the hardest problems in deep learning: generalisation. Deep learning methods learn from data and are then deployed to make predictions or decisions. The core generalisation

show abstract

XLVIN: eXecuted Latent Value Iteration Nets

Cited by 3 publications

References 20 publications

Myriad: a real-world testbed to bridge trajectory optimization and deep learning

Myriad: a real-world testbed to bridge trajectory optimization and deep learning

Persistent Message Passing

Neural Algorithmic Reasoning

Contact Info

Product

Resources

About