Goal Reasoning and Trusted Autonomy

Abstract

An important consideration for any autonomous system is the need to react intelligently to unplanned events and observations. Doing so requires that the system be given the freedom and ability to adjust its behavior, without being commanded to do so by a human operator or external system. A good deal of research in robotics and autonomy focuses on developing systems that can change their actions and plans, to more reliably or more optimally achieve their goals. However, it is also important to develop systems that autonomously deliberate on the goals themselves. Goal Reasoning (e.g., [38]) is the study of how autonomous agents can dynamically reason about and adjust their goals. Doing so enables agents to adapt intelligently to changing conditions and unexpected events, allowing them to address a wider variety of complex problems. Section 1.3 argues for the generality of reward maximization for goals; Goal Reasoning, then, would allow the autonomous agent to adapt its own reward function. Goal Reasoning capabilities, in one form or another, may prove useful in many applications. In domains where the agents must operate autonomously for substantial periods of time or with limited communications, such as in unmanned underwater B. Johnson (B) • A.

Approaches for assessing communication in human-autonomy teams

“…For this scope, human-autonomy teams involve one or more humans and one or more autonomyenabled systems, or intelligent agents (IA), who coordinate and work interdependently over time in order to reach a common goal or complete a task (McNeese et al 2018). An IA is an entity that possesses the capability to sense, observe and act on the environment, and intelligently respond to unexpected, dynamic events (Johnson et al 2018;Mercado et al 2016). In future military operations, there is the potential for interaction with a wide variety of embodied IAs-from small ground, air, surface or subsurface unmanned vehicles or robots to large weaponized unmanned vehicles, troop or cargo carriers, and even computer-based IAs that assist with decision-making, planning, or communication.…”

Artificial Intelligence – The Big Picture

“…Many AI systems will have implicit objectives or explicitly programmed goals or at least a reward function. Depending on the degree of the autonomy of a system, the machine will pursue these goals or seek to maximize its rewards without further human guidance (Johnson et al 2018). The caveat is that humans tend to have shifting values, motivations, goals, alliances and priorities, requiring changing behaviors over time.…”

Foundations of Collaborative Task-Oriented Dialogue: What’s in a Slot?

“…Whereas much utterance variability and uncertainty can be captured via the above approach, we believe there is less variability at the level of the goal/intention lifecycle, which includes goal adoption, commitment, planning, achievement, failure, abandonment, reformulation, etc. (Galescu et al, 2018;Johnson et al, 2018). This goal lifecycle would be directly supported by the BDI architecture and therefore would be available for every domain.…”