A probabilistic data-driven model for planar pushing

Bauzá, Maria; Rodríguez, Alberto

doi:10.1109/icra.2017.7989345

Cited by 89 publications

(105 citation statements)

References 17 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…Since we presented our earlier dataset on planar pushing [1], it has been directly used for: 1) Stochastic modeling: [23,10,17] 2) Modeling from rendered images: [16] 3) Model identification: [24] 4) Learning models for control: [25,26] 5) Filtering: [27] 6) Meta-learning: [28] With this new dataset we hope to further facilitate research in learning models and control.…”

Section: Related Workmentioning

confidence: 99%

Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video

Bauzá

Alet

Lin

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Pushing is a fundamental robotic skill. Existing work has shown how to exploit models of pushing to achieve a variety of tasks, including grasping under uncertainty, in-hand manipulation and clearing clutter. Such models, however, are approximate, which limits their applicability.Learning-based methods can reason directly from raw sensory data with accuracy, and have the potential to generalize to a wider diversity of scenarios. However, developing and testing such methods requires rich-enough datasets. In this paper we introduce Omnipush, a dataset with high variety of planar pushing behavior.In particular, we provide 250 pushes for each of 250 objects, all recorded with RGB-D and a high precision tracking system. The objects are constructed so as to systematically explore key factors that affect pushing -the shape of the object and its mass distribution-which have not been broadly explored in previous datasets, and allow to study generalization in model learning.Omnipush includes a benchmark for meta-learning dynamic models, which requires algorithms that make good predictions and estimate their own uncertainty. We also provide an RGB video prediction benchmark and propose other relevant tasks that can be suited with this dataset. Data and code are available at https://web.mit.edu/mcube/omnipush-dataset/.

show abstract

Section: Related Workmentioning

confidence: 99%

Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video

Bauzá

Alet

Lin

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…They provide an insightful table that lists the assumptions and approximations made in much of the work cited in this section. Finally, Bauzá and Rodriguez (2017) used a data-driven approach to model planar pushing interaction to predict both the most likely outcome of a push and, as a novelty, its expected variability. The learned models (also trained on the dataset by Yu et al (2016)) rely on a variation of Gaussian processes whilst avoiding and evaluating the quasi-static assumption.…”

Section: Complementing Analytical Approaches With Data-driven Methodsmentioning

confidence: 99%

Let's Push Things Forward: A Survey on Robot Pushing

2020

View full text Add to dashboard Cite

As robot make their way out of factories into human environments, outer space, and beyond, they require the skill to manipulate their environment in multifarious, unforeseeable circumstances. With this regard, pushing is an essential motion primitive that dramatically extends a robot's manipulation repertoire. In this work, we review the robotic pushing literature. While focusing on work concerned with predicting the motion of pushed objects, we also cover relevant applications of pushing for planning and control. Beginning with analytical approaches, under which we also subsume physics engines, we then proceed to discuss work on learning models from data. In doing so, we dedicate a separate section to deep learning approaches which have seen a recent upsurge in the literature. Concluding remarks and further research perspectives are given at the end of the paper. Stüber et al. Let's Push Things Forward PROBLEM STATEMENTEven in ideal conditions, such as structured environments where an agent has a complete model of the environment and perfect sensing abilities, the problems of robotic grasping and manipulation are not trivial. By complete model of the environment we mean that physical and geometric properties of the world, such as pose, shape, friction parameters and the mass of the object we wish to manipulate, are exactly known. In fact, the object to be manipulated is indirectly controlled by contacts with a robot manipulator (e.g. pushing by a contacting finger part), and an inverse model (IM), which computes an action to produce the desired motion or set of forces on the object, may not be known. Sometimes forward models (FM) may be fully or partially known, even where IMs are not available. In such cases, an FM can be used to estimate the next state of a system, given the current state and a set of executable actions. This enables planning to be 4. Physics engines. It employs a physics engine as a "black box" to make predictions about the interactions.5. Data-driven. It learns how to predict physical interaction from examples. 6. Deep learning. As the data-driven approaches, it learns how to construct an FM from examples. The key insight is that the deep learning approaches are based on feature extraction.The features highlighted for each approach are as follows.• The assumptions made by the authors on their approach. We highlight i) the quasi-static assumption in the model, ii) if it is a seminal work on 2D shapes, and iii) if the method required a known model of the object to be manipulated.• The type of motion analysed in the paper, such as 1D, planar (2D translation and 1D rotation around the x−axis), or full 3D (3D translation and 3D rotation).• The aim of the paper. We distinguish between predicting the motion of the object, estimating physical parameters, planning pushes, and analysing a push to reach a stable grasp.

show abstract

“…The literature in this area is vast, emerging early from classical solutions that explicitly model the dynamics of pushing with frictional forces [1], [2]. While inspiring, many of these methods rely on modeling assumptions that do not hold in practice [16], [4]. For example, non-uniform friction distributions across object surfaces and the variability of friction are only some of the factors that can lead to erroneous predictions of friction-modeling pushing solutions in real-world settings.…”

Section: Related Workmentioning

confidence: 99%

Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning

Zeng

Song

Welker

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

541

466

View full text Add to dashboard Cite

Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn these synergies from scratch through model-free deep reinforcement learning. Our method involves training two fully convolutional networks that map from visual observations to actions: one infers the utility of pushes for a dense pixel-wise sampling of end effector orientations and locations, while the other does the same for grasping. Both networks are trained jointly in a Q-learning framework and are entirely self-supervised by trial and error, where rewards are provided from successful grasps. In this way, our policy learns pushing motions that enable future grasps, while learning grasps that can leverage past pushes. During picking experiments in both simulation and real-world scenarios, we find that our system quickly learns complex behaviors amid challenging cases of clutter, and achieves better grasping success rates and picking efficiencies than baseline alternatives after only a few hours of training. We further demonstrate that our method is capable of generalizing to novel objects. Qualitative results (videos), code, pre-trained models, and simulation environments are available at

show abstract

A probabilistic data-driven model for planar pushing

Cited by 89 publications

References 17 publications

Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video

Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video

Let's Push Things Forward: A Survey on Robot Pushing

Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning

Contact Info

Product

Resources

About