POMDP-Based Candy Server:Lessons Learned from a Seven Day Demo

Abstract

An autonomous robot must decide a good strategy to achieve its long term goal, despite various types of uncertainty. The Partially Observable Markov Decision Processes (POMDPs) is a principled framework to address such a decision making problem. Despite the computational intractability of solving POMDPs, the past decade has seen substantial advancement in POMDP solvers. This paper presents our experience in enabling on-line POMDP solving to become the sole motion planner for a robot manipulation demo at IEEE SIMPAR and ICRA 2018. The demo scenario is a candy-serving robot: A 6-DOFs robot arm must pick-up a cup placed on a table by a user, use the cup to scoop candies from a box, and put the cup of candies back on the table. The average perception error is ∼3cm (≈ the radius of the cup), affecting the position of the cup and the surface level of the candies. This paper presents a strategy to alleviate the curse of history issue plaguing this scenario, the perception system and its integration with the planner, and lessons learned in enabling an online POMDP solver to become the sole motion planner of this entire task. The POMDP-based system were tested through a 7 days live demo at the two conferences. In this demo, 150 runs were attempted and 98% of them were successful. We also conducted further experiments to test the capability of our POMDP-based system when the environment is relatively cluttered by obstacles and when the user moves the cup while the robot tries to pick it up. In both cases, our POMDP-based system reaches a success rate of 90% and above.

Multilevel Monte Carlo for solving POMDPs on-line

“…Although solving a POMDP exactly is computationally intractable (Papadimitriou and Tsitsiklis, 1987), the past two decades have seen tremendous progress in developing approximately optimal solvers that trade optimality for computational tractability, enabling POMDPs to start to become practical for various robotic planning problems (Bai and Hsu, 2012; Horowitz and Burdick, 2013; Hsiao et al, 2007; Wandzel et al, 2019; Hoerger et al, 2019b).…”