Director: A User Interface Designed for Robot Operation with Shared Autonomy

Abstract

Operating a high degree of freedom mobile manipulator, such as a humanoid, in a field scenario requires constant situational awareness, capable perception modules, and effective mechanisms for interactive motion planning and control. A well-designed operator interface presents the operator with enough context to quickly carry out a mission and the flexibility to handle unforeseen operating scenarios robustly. By contrast, an unintuitive user interface can increase the risk of catastrophic operator error by overwhelming the user with unnecessary information. With these principles in mind, we present the philosophy and design decisions behind Director-the open-source user interface developed by Team MIT to pilot the Atlas robot in the DARPA Robotics Challenge (DRC). At the heart of Director is an integrated task execution system that specifies sequences of actions needed to achieve a substantive task, such as drilling a wall or climbing a staircase. These task sequences, developed a priori, make online queries to automated perception and planning algorithms with outputs that can be reviewed by the operator and executed by our whole-body controller. Our use of Director at the DRC resulted in efficient high-level task operation while being fully competitive with approaches focusing on teleoperation by highly-trained operators. We discuss the primary interface elements that comprise the Director and provide analysis of its successful use at the DRC.

A Virtual-Reality Driven Approach for Generating Humanoid Multi-Contact Trajectories

“…In Leeper et al [11] this is implemented through an interactable "rings-andarrows" widget that directly maps to the controller's endeffector setpoint. Similar strategies of direct setpoint control were widely used during the DARPA Robotics Challenge for teleoperating humanoids [3][4][5]. In these frameworks, the operator directly interfaced with an inverse kinematics solver's setpoint through similar interactable elements.…”

“…In these frameworks, the operator directly interfaced with an inverse kinematics solver's setpoint through similar interactable elements. These approaches, in particular [5], also had additional flexibility such as selecting hand, chest, pelvis and Center of Mass setpoints as well as the taskspace constraint set and base of the kinematic chain. Recently this approach has been adapted to VR for use in humanoid teleoperation [7,8].…”

“…5(b). This could be addressed by incorporating a motion preview similar such as [4,5]. Inverse kinematics failures occur for two reasons: getting "stuck" and going unstable.…”

“…Additionally it is challenging to construct generalized posture scoring functions that perform well over a large space of configurations. Alternatively, operator-driven approaches have been proven as a reliable means of commanding complex humanoid motion [3][4][5]. With many recent successful results using virtual reality (VR) [6][7][8][9][10] we believe it is a well-suited solution to the problem of generating multi-contact humanoid trajectories.…”