Learning search polices from humans in a partially observable context

Abstract

Decision making and planning for which the state information is only partially available is a problem faced by all forms of intelligent entities they being either virtual, synthetic or biological. The standard approach to mathematically solve such a decisional problem is to formulate it as a partially observable decision process (POMDP) and apply the same optimisation techniques used in the Markov decision process (MDP). However, applying naively the same methodology to solve MDPs as with POMDPs makes the problem computationally intractable. To address this problem, we take a programming by demonstration approach to provide a solution to the POMDP in continuous state and action space. In this work, we model the decision making process followed by humans when searching blindly for an object on a table. We show that by representing the belief of the human's position in the environment by a particle filter (PF) and learning a mapping from this belief to their end effector velocities with a Gaussian mixture model (GMM), we can model the human's search process and reproduce it for any agent. We further categorize the type of behaviours demonstrated by humans as being either risk-prone or risk-averse and find that more than 70% of the human searches were considered to be risk-averse. We contrast the performance of this human-inspired search model with respect to greedy and coastal navigation search methods. Our evaluation metric is the distance taken to reach the goal and how each method minimises the uncertainty. We further analyse the control policy of the coastal navigation and GMM search models and argue that taking into account uncertainty is more efficient with respect to distance travelled to reach the goal.

Study on Data-Driven Approaches for the Automated Assembly of Board-to-Board Connectors

“…Abbu-Dakka [22] solved the peg-in-hole task by human demonstration using exception strategies. Moreover, De Chambier and Suomalainen [23,24] proposed learning compliance for assembly motion demonstration. The key idea of these methods is to learn from human demonstration and exploration for the distribution in their dynamic model, as Dennis mentioned in his research [25].…”

A Survey of Robot Manipulation in Contact

“…Whereas both are very popular in robotics in general, with different techniques available to reduce the computational complexity, there is a limited number of in-contact tasks they has been used for; popular examples being [168,169], who used POMDP to handle motions requiring both free space and in-contact motions. POMDP has also been used to leverage contact for localization both in classical tasks [170] and when teaching the robot to search for a goal from a human demonstration [171,172,173].…”

Learning Haptic Exploration Schemes for Adaptive Task Execution

“…In [25], autonomous exploration of the whole workspace is combined with tactile-based object discrimination. In [26], the search policy is learned from human behavior to act in the whole workspace, where we instead focus on the transfer of specialized skills to rapidly program and execute a novel task. The approach in [27] combines in-hand object localization using a tactile sensor array with tactile based manipulation.…”