Propagation Networks for Model-Based Control Under Partial Observation

Abstract

There has been an increasing interest in learning dynamics simulators for model-based control. Compared with off-the-shelf physics engines, a learnable simulator can quickly adapt to unseen objects, scenes, and tasks. However, existing models like interaction networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNet), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. Experiments show that our propagation networks not only outperform current learnable physics engines in forward simulation, but also achieve superior performance on various control tasks. Compared with existing model-free deep reinforcement learning algorithms, model-based control with propagation networks is more accurate, efficient, and generalizable to new, partially observable scenes and tasks. k∈Ni e k,t ), i = 1 . . . |O|,where o i,t = x i,t , a o i , p i,t denotes object i at time t, u k and v k are the receiver and sender of relation r k , and N i denotes the relations where object i is the receiver. B. Propagation NetworksIN defines a flexible and efficient model for explicit reasoning of objects and their relations in a complex system. It can handle a variable number of objects and relations and has performed well in domains like n-body systems, bouncing

Combining Self-Organizing and Graph Neural Networks for Modeling Deformable Objects in Robotic Manipulation

“…With the objective to support the requirements of path planning and dynamic interaction of a robotic hand with a deformable object, Graph Neural Network (GNN) based models are also adapted in our framework to predict the future object state using the information of the current object state and the manipulation actions of the robotic hand. Specifically, we use the Interaction Network (IN) framework (Battaglia et al, 2016 ) along with its extension known as PropNet (Li et al, 2019b ) for supervised learning on graph structures. Unlike standard GNNs, the IN is specifically designed to learn the dynamics of physical interactive systems.…”

“…Therefore, several iterations of the algorithm are needed in order to propagate the information on the graph, and thus reach remote nodes. As an extension to IN, the PropNet (Li et al, 2019b ) formulation (Algorithm 4) proposes the inclusion of a multi-step propagation phase, which consists of computing the edge and node effects using an additional iterative process, where l corresponds to the current propagation step parameter, and is set to a value within the range of 1 ≤ l ≤ L . Also, the update functions, , , are used to encode the input edge and node features, respectively.…”

Learning to see the physical world

“…Modeling dynamics directly in the pixel space is universal but challenging due to the intricate interplay between physics and graphics; an alternative is to separate perception from dynamics modeling, and learn dynamics from object states. Our recent work along this line has shown that a model that learns to approximate object dynamics can be useful for planning [Janner et al, 20191, generalize to scenarios where only partial observations are available [Li et al, 2019c], and discover physical object properties without supervision [Zheng et al, 2018]. We have further extended our model to particle-based representations, so that it can characterize the dynamics of soft robots [Hu et al, 2019] (Figure 1-3C) and of scenes with complex interactions among rigid bodies, deformable shapes, and fluids (Figure 1-3D).…”

Data_Sheet_1.PDF

“…With the objective to support the requirements of path planning and dynamic interaction of a robotic hand with a deformable object, Graph Neural Network (GNN) based models are also adapted in our framework to predict the future object state using the information of the current object state and the manipulation actions of the robotic hand. Specifically, we use the Interaction Network (IN) framework (Battaglia et al, 2016) along with its extension known as PropNet (Li et al, 2019b) for supervised learning on graph structures. Unlike standard GNNs, the IN is specifically designed to learn the dynamics of physical interactive systems.…”

“…Therefore, several iterations of the algorithm are needed in order to propagate the information on the graph, and thus reach remote nodes. As an extension to IN, the PropNet (Li et al, 2019b) formulation (Algorithm 4) proposes the inclusion of a multi-step propagation phase, which consists of computing the edge and node effects using an additional iterative process, where l corresponds to the current propagation step parameter, and is set to a value within the range of 1 ≤ l ≤ L. Also, the update functions, φ enc R , φ enc O , are used to encode the input edge and node features, respectively. While the function, φ dec O , is used to decode the output node feature.…”