Robust trajectory selection for rearrangement planning as a multi-armed bandit problem

Abstract

Abstract-We present an algorithm for generating openloop trajectories that solve the problem of rearrangement planning under uncertainty. We frame this as a selection problem where the goal is to choose the most robust trajectory from a finite set of candidates. We generate each candidate using a kinodynamic state space planner and evaluate it using noisy rollouts.Our key insight is we can formalize the selection problem as the "best arm" variant of the multi-armed bandit problem. We use the successive rejects algorithm to efficiently allocate rollouts between candidate trajectories given a rollout budget. We show that the successive rejects algorithm identifies the best candidate using fewer rollouts than a baseline algorithm in simulation. We also show that selecting a good candidate increases the likelihood of successful execution on a real robot. I. IntroductionWe explore the rearrangement planning problem [11] where a robot must rearrange several objects in a cluttered environment to achieve a goal ( Fig.1-Top). Recent work has used simple physics models, like quasistatic pushing [8,9,22,32,39,43], to quickly produce efficient and intricate plans. However, the intricacy of these plans makes them particularly sensitive to uncertainty in object pose, physics parameters, and trajectory execution.We address this problem by generating open-loop trajectories that are robust to uncertainty. However, this is particularly hard for rearrangement planning.First, the problem is set in a high-dimensional space with continuous actions. Second, contact causes physics to evolve in complex, non-linear ways and quickly leads to multi-modal and non-smooth distributions [25,26,34]. Third, finding good trajectories is inherently hard: most trajectories achieve success with zero probability.As a consequence, this problem lies outside the domains of standard conformant planning [19,37], POMDP [20] and policy search [7] algorithms.In response to these challenges, we propose a domain-agnostic algorithm that only requires: (1) a stochastic method of generating trajectories, (2) the ability to forward-simulate the system's dynamics, and (3) the capability of testing whether an execution is successful.Exploiting the fact that we can quickly generate feasible state space trajectories [22], we formulate rearrangement planning under uncertainty (sections III and IV)

Learning image-based Receding Horizon Planning for manipulation in clutter

“…We use a similar kinodynamic RRT planner to generate demonstrations in different task instances. There are planners which also take uncertainty into account before the generation of the motion trajectory [13,34,25,43], but these planners typically rely on uncertainty reducing actions, which generate a conservative sequence of actions, limiting the robot from using the complete dynamics of the domain. Alternatively, to avoid the uncertainty associated with multiple objects interacting in a cluttered environment, planning approaches have been developed to avoid contact altogether with environment obstacles.…”

Efficient and High-quality Prehensile Rearrangement in Cluttered and Confined Spaces

“…Non-prehensile actions have also been explored as they can simultaneously move multiple objects [40]- [42], quickly declutter a scene and help minimize uncertainty [43] under a conformant probabilistic planning formulation [44], [45]. Though predicting the effect of pushing actions has been studied [46], [47], they are not as predictable as prehensile ones.…”

Modelling reversible execution of robotic assembly

“…7 Koval et al demonstrated a similar passive approach. 8 The approach presented in this paper is different in that it does not explicitly model the uncertainties of the assembly task, but relies on probabilistic effects to eventually handle errors rather than trying to avoid them.…”