MOMDP-Based Target Search Mission Taking into Account the Human Operator's Cognitive State

Abstract

This study discusses the application of sequential decision making under uncertainty and mixed observability in a mixed-initiative robotic target search application. In such a robotic mission, two agents, a ground robot and a human operator, must collaborate to reach a common goal using, each in turn, their recognized skills. The originality of the work relies in considering that the human operator is not a providential agent when the robot fails. Using the data from previous experiments, a Mixed Observability Markov Decision Process (MOMDP) model was designed, which allows to consider aleatory failure events and the partial observable human operator's state while planning for a long-term horizon. Results show that the collaborative system was in general able to successfully complete or terminate the mission, even when many simultaneous sensors, devices and operators failures happened. So, the mixed-initiative framework highlighted in this study shows the relevancy of taking into account the cognitive state of the operator, which permits to compute a policy for the sequential decision problem which prevents to re-planning when unexpected (but known) events occurs.

Towards Mixed-Initiative Human–Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction

“…Beyond that, and based on the achieved results of this paper, the next step in this research is to implement online methods to continuously infer the human operators' performance like-profile (low versus high performer) and to launch adaptation based on the classification results obtained possibly used to feed a sequential decision-making process. More precisely, we believe that such an online performance profile prediction could be used to feed a stochastic sequential decision-making model dedicated to trigger adaptation [4,12].…”

“…It is known that classification errors can possibly bring the system to launch inappropriate actions. A promising solution to avoid launching inappropriate actions (e.g., actions based only on short term reasoning, called greedy actions) is to exploit the Partially Observable Markov Decision Process (POMDP) framework [4,12] to drive the human-robot interactions because: (i) it can integrate the uncertainties related to system states transitions (e.g., human actions) and related to observation functions (e.g., output labels from classifiers that are not always accurate); (ii) it can maintain a belief state distribution over possible human states (or operators-like profile); this belief state is updated each time an action is performed and an observation received using the Bayes rule; and (iii) its solution is an optimal policy-a function mapping reachable belief states to actions-that aims to maximize the expected long-term gains (i.e., entire system's performance). In this sense, the belief state update process should estimate the system state (human, robot and mission states) during the interaction and choose to adapt the automation level of the robot with respect to the expected upcoming mission performance.…”

Online ECG-based Features for Cognitive Load Assessment

“…Given that any objective information about human operator's mental state is a valuable information, one could consequently consider the proposed online metrics as supplementary features for mixed-initiative system development. For instance de Souza and colleagues [6] have proposed to use such physiological data with a Partially Observable Markov Decision Process (POMDP) to improve human-machine interactions.…”

“…From these results, two branches of future work could be indicated: (i) broadening this research to develop a more comprehensive pool of candidate metrics and algorithms for online cardiac monitoring, and (ii) use of these results to experimentally validate the approach only simulated in [6]. Related to this last point, the experiment described in [9] is now taking place in our lab.…”

“…More recently, Scannella et al [5] successfully used the same metrics to evaluate pilots' flying-related mental workload in real, light aircraft. Building on the work of Régis and colleagues [4], de Souza et al [6] implemented a successful method to develop control policies that could take such human state estimations into account to improve a mixed-initiative mission's success rate. Success was achieved despite various system failures (human or machine), providing that both human and machine state estimations could be reliably inferred online.…”