Human in the Loop Evaluation of a Mixed-Initiative System for Planning and Control of Multiple UAV Teams

Abstract

This paper describes a ‘Human in the Loop' evaluation of an early prototype mixed-initiative control system that generated plans for single operator supervision of multiple unmanned aerial vehicles (UAV) missions. The objectives of the evaluation were to assess to what extent human operators were able to understand the plans generated by the automated controllers as well as to point to additional support requirements to drive further development of mixed-initiative planning systems. Six former fighter pilots served as test participants. Multiple convergent measures were utilized to evaluate the ability of users to understand and evaluate the plans generated by the automated controller. While test participants were able to understand the plans, the results provided compelling evidence for the need to communicate more effectively the rationale behind plan elements proposed by the automated controller and to provide ‘levers' to enable the user to modify the plan. These results point to important challenges for design of ‘mixed-initiative’ controllers to enable the human and automated controller to function as effective ‘partners’.

The Effects of Degraded Digital Instrumentation and Control Systems on Human-system Interfaces and Operator Performance: HFE Review Guidance and Technical Basis

“…Willems and Heiney (2002) stated that "As errors involving automation tend to be more cataclysmic and costly, the human interface has become more important than ever" (p. 3). The HSIs typically provide insufficient information about automation's goals, current activities, and performance (Liu, Nakata & Furuta, 2004;Lee & See, 2004;Parasuraman & Riley, 1997;Roth et al, 2004;Rook & McDonnell, 1993).…”

Scaling Up Wide-Area-Search-Munition Teams

“…For example, Emilie Roth and her colleagues describe an "intent matrix" interface for specifying a target class's relative value to a modelbased architecture. 4 In this architecture, the operator could select particular targets only indirectly by changing class values and computing new plans until one was found that included a desired target.…”

Affecting operator trust in intelligent multirobot surveillance systems

“…However, high LOAs are prone to a loss of situational awareness and complacency by removing the operator from the task [5]. Also, operators often do not understand what autonomous systems are doing because the action selection algorithm is markedly different [6]. As a result, operators are more likely to intervene because they do not trust the system [7].…”

“…Exocentric views most often show an overhead view of a map with the positions and status of the robots [13], [14]. In addition to the current state, some interfaces also provide the future state of the robots by displaying the planned routes of the robots [6], [7], target or hazard locations [8], [15] and explored areas of the map [16]. The interface in [17] provided the explored area as well as the open regions around the robots and information about the selected robot's local situation.…”