Robust Task and Motion Planning for Long-Horizon Architectural Construction Planning

Hartmann, Valentin; Oguz, Ozgur S.; Driess, Danny; Toussaint, Marc; Menges, Achim

doi:10.1109/iros45743.2020.9341502

Cited by 28 publications

(24 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, scaling to significantly longer action sequence lengths, it becomes clear that it is neither necessary nor feasible to optimize trajectories with complete global consistency. Introducing further hierarchies (Kaelbling and Lozano-Pe ´rez, 2011) or breaking the manipulation down into (less-dependent) subgoals (Driess et al, 2019a;Hartmann et al, 2020Hartmann et al, , 2021 becomes necessary.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Learning to solve sequential physical reasoning problems from a scene image

Driess

Toussaint

2021

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

In this article, we propose deep visual reasoning, which is a convolutional recurrent neural network that predicts discrete action sequences from an initial scene image for sequential manipulation problems that arise, for example, in task and motion planning (TAMP). Typical TAMP problems are formalized by combining reasoning on a symbolic, discrete level (e.g., first-order logic) with continuous motion planning such as nonlinear trajectory optimization. The action sequences represent the discrete decisions on a symbolic level, which, in turn, parameterize a nonlinear trajectory optimization problem. Owing to the great combinatorial complexity of possible discrete action sequences, a large number of optimization/motion planning problems have to be solved to find a solution, which limits the scalability of these approaches. To circumvent this combinatorial complexity, we introduce deep visual reasoning: based on a segmented initial image of the scene, a neural network directly predicts promising discrete action sequences such that ideally only one motion planning problem has to be solved to find a solution to the overall TAMP problem. Our method generalizes to scenes with many and varying numbers of objects, although being trained on only two objects at a time. This is possible by encoding the objects of the scene and the goal in (segmented) images as input to the neural network, instead of a fixed feature vector. We show that the framework can not only handle kinematic problems such as pick-and-place (as typical in TAMP), but also tool-use scenarios for planar pushing under quasi-static dynamic models. Here, the image-based representation enables generalization to other shapes than during training. Results show runtime improvements of several orders of magnitudes by, in many cases, removing the need to search over the discrete action sequences.

show abstract

Section: Discussionmentioning

confidence: 99%

“…LGP will be the underlying framework of the present work. For large-scale problems, however, LGP also suffers from the exponentially increasing number of possible symbolic action sequences (Hartmann et al, 2020). Solving this issue is one of the main motivations for our work.…”

Section: Learning Heuristics For Tamp and Mip In Roboticsmentioning

confidence: 99%

Learning to solve sequential physical reasoning problems from a scene image

Driess

Toussaint

2021

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…The next best planner is the OMPL planner BiRLRT (15) [56] with 1.52s, QRRT with 2.01s and RRTConnect (6) with 1.70s. We note that also the planner PDST (35) [50], RLRT ( 14) [56] and KPIECE1 (36) [90] perform competively with 3.25s, 3.68s and 6.27s, respectively. The planner QRRT* does not perform well on this problem instance with 25.35s, due to similar problems as on the Bugtrap scenario.…”

Section: E 37-dof Pre-graspmentioning

confidence: 99%

“…However, sampling-based algorithms do not perform well when the state space of the robot contains narrow passages [58,92,38,78], which are low-measure regions which have to be traversed to reach a goal. Narrow passages are often occurring in tasks which are particularly important in robotic applications, like grasping, peg-in-hole, egress/ingress or long-horizon planning problems [24,35].…”

Section: Introductionmentioning

confidence: 99%

Section Patterns: Efficiently Solving Narrow Passage Problems in Multilevel Motion Planning

Orthey¹,

Toussaint²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Sampling-based planning methods often become inefficient due to narrow passages. Narrow passages induce a higher runtime, because the chance to sample them becomes vanishingly small. In recent work, we showed that narrow passages can be approached by relaxing the problem using admissible lowerdimensional projections of the state space. Those relaxations often increase the volume of narrow passages under projection. Solving the relaxed problem is often efficient and produces an admissible heuristic we can exploit. However, given a base path, i.e. a solution to a relaxed problem, there are currently no tailored methods to efficiently exploit the base path. To efficiently exploit the base path and thereby its admissible heuristic, we develop section patterns, which are solution strategies to efficiently exploit base paths in particular around narrow passages. To coordinate section patterns, we develop the pattern dance algorithm, which efficiently coordinates section patterns to reactively traverse narrow passages. We combine the pattern dance algorithm with previously developed multilevel planning algorithms and benchmark them on challenging planning problems like the Bugtrap, the double L-shape, an egress problem and on four pregrasp scenarios for a 37 degrees of freedom shadow hand mounted on a KUKA LWR robot. Our results confirm that section patterns are useful to efficiently solve high-dimensional narrow passage motion planning problems.

show abstract

“…Our method can be directly integrated as an inner module in TAMP approaches, enabling them to solve more challenging and constrained problems. In fact, in complex and cluttered scenarios like a construction site, generating the modeswitches becomes one of the computational bottlenecks of the whole TAMP pipeline [37].…”

Section: Robotic Sequential Manipulationmentioning

confidence: 99%

Learning Efficient Constraint Graph Sampling for Robotic Sequential Manipulation

Ortiz-Haro¹,

Hartmann²,

Oguz³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Efficient sampling from constraint manifolds, and thereby generating a diverse set of solutions of feasibility problems, is a fundamental challenge. We consider the case where a problem is factored, that is, the underlying nonlinear mathematical program is decomposed into differentiable equality and inequality constraints, each of which depends only on some variables. Such problems are at the core of efficient and robust sequential robot manipulation planning. Naive sequential conditional sampling of individual variables, as well as fully joint sampling of all variables at once (e.g., leveraging optimization methods), can be highly inefficient and non-robust. We propose a novel framework to learn how to break the overall problem into smaller sequential sampling problems. Specifically, we leverage Monte-Carlo Tree Search to learn which variable subsets should be assigned in which sequential order, in order to minimize the computation time to generate full samples. This strategy allows us to efficiently compute a set of diverse valid robot configurations for modeswitches within sequential manipulation tasks, which are waypoints for subsequent trajectory optimization or sampling-based motion planning algorithms. We show that the learning method quickly converges to the best sampling strategy for a given problem, and outperforms user-defined orderings and joint optimization, while also providing a higher sample diversity. Video: https://youtu.be/xWAjBGACZhs

show abstract

Robust Task and Motion Planning for Long-Horizon Architectural Construction Planning

Cited by 28 publications

References 12 publications

Learning to solve sequential physical reasoning problems from a scene image

Learning to solve sequential physical reasoning problems from a scene image

Section Patterns: Efficiently Solving Narrow Passage Problems in Multilevel Motion Planning

Learning Efficient Constraint Graph Sampling for Robotic Sequential Manipulation

Contact Info

Product

Resources

About