“I'm Confident This Will End Poorly”: Robot Proficiency Self-Assessment in Human-Robot Teaming

Conlon, Nicholas; Szafir, Daniel; Ahmed, Nisar

doi:10.1109/iros47612.2022.9981653

Cited by 4 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods generally compute agent self-confidence based on simulation ( Israelsen et al, 2019 ; Ardón et al, 2020 ) or previous experience ( Frasca et al, 2020 ). Our recent work showed that reporting of a priori self-assessments lead to better choices of reliance ( Conlon et al, 2022c ) and improvements in performance and trust ( Conlon et al, 2022b ). However, in dynamic environments, a priori assessment is a poor predictor of the agent’s confidence due to factors that are not accounted for before execution, such as environmental changes, task changes, or interactions with other agents.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Recent work has shown that robots that report a priori competency self-assessments can align operator perception with the robot’s actual competency, thus improving decision-making and performance in a variable-autonomy navigation task ( Conlon et al, 2022b ). However, in dynamic and uncertain environments, like the SAR scenario outlined above, an a priori confidence assessment can quickly become stale due to environmental changes such as falling debris or unexpected obstacles.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Event-triggered robot self-assessment to aid in autonomy adjustment

Conlon,

Ahmed,

Szafir

2024

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Introduction: Human–robot teams are being called upon to accomplish increasingly complex tasks. During execution, the robot may operate at different levels of autonomy (LOAs), ranging from full robotic autonomy to full human control. For any number of reasons, such as changes in the robot’s surroundings due to the complexities of operating in dynamic and uncertain environments, degradation and damage to the robot platform, or changes in tasking, adjusting the LOA during operations may be necessary to achieve desired mission outcomes. Thus, a critical challenge is understanding when and how the autonomy should be adjusted.Methods: We frame this problem with respect to the robot’s capabilities and limitations, known as robot competency. With this framing, a robot could be granted a level of autonomy in line with its ability to operate with a high degree of competence. First, we propose a Model Quality Assessment metric, which indicates how (un)expected an autonomous robot’s observations are compared to its model predictions. Next, we present an Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm that uses changes in the Model Quality Assessment above a threshold to selectively execute and report a high-level assessment of the robot’s competency. We validated the Model Quality Assessment metric and the ET-GOA algorithm in both simulated and live robot navigation scenarios.Results: Our experiments found that the Model Quality Assessment was able to respond to unexpected observations. Additionally, our validation of the full ET-GOA algorithm explored how the computational cost and accuracy of the algorithm was impacted across several Model Quality triggering thresholds and with differing amounts of state perturbations.Discussion: Our experimental results combined with a human-in-the-loop demonstration show that Event-Triggered Generalized Outcome Assessment algorithm can facilitate informed autonomy-adjustment decisions based on a robot’s task competency.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Event-triggered robot self-assessment to aid in autonomy adjustment

Conlon,

Ahmed,

Szafir

2024

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Pilots mentioned several ways that they missed the ability of autonomous systems to communicate in ways similar to humanhuman dialogs (Zhang et al, 2021). Communication between humans and autonomous systems must allow the system to understand human intentions, think ahead, and react appropriately to complex and changing situations (Zhang et al, Human-like aspects Describes features and characteristics that an autonomous system should have to be perceived as human-like Zhang et al (2021), Conlon et al (2022), Ulusoy and Reisman (2022) Frontiers in Aerospace Engineering frontiersin.org 2021). This communication skill should be bi-directional (Shively et al, 2017).…”

Section: Recent Hat Empirical Study Literaturementioning

confidence: 99%

Comparison and synthesis of two aerospace case studies to develop human-autonomy teaming requirements

Tokadli

Dorneich

2023

Front. Aerosp. Eng.

View full text Add to dashboard Cite

This paper developed human-autonomy teaming (HAT) characteristics and requirements by comparing and synthesizing two aerospace case studies (Single Pilot Operations/Reduced Crew Operations and Long-Distance Human Space Operations) and the related recent HAT empirical studies. Advances in sensors, machine learning, and machine reasoning have enabled increasingly autonomous system technology to work more closely with human(s), often with decreasing human direction. As increasingly autonomous systems become more capable, their interactions with humans may evolve into a teaming relationship. However, humans and autonomous systems have asymmetric teaming capabilities, which introduces challenges when designing a teaming interaction paradigm in HAT. Additionally, developing requirements for HAT can be challenging for future operations concepts, which are not yet well-defined. Two case studies conducted previously document analysis of past literature and interviews with subject matter experts to develop domain knowledge models and requirements for future operations. Prototype delegation interfaces were developed to perform summative evaluation studies for the case studies. In this paper, a review of recent literature on HAT empirical studies was conducted to augment the document analysis for the case studies. The results of the two case studies and the literature review were compared and synthesized to suggest the common characteristics and requirements for HAT in future aerospace operations. The requirements and characteristics were grouped into categories of team roles, autonomous teammate types, interaction paradigms, and training. For example, human teammates preferred the autonomous teammate to have human-like characteristics (e.g., dialog-based conversation, social skills, and body gestures to provide cue-based information). Even though more work is necessary to verify and validate the requirements for HAT development, the case studies and recent empirical literature enumerate the types of functions and capabilities needed for increasingly autonomous systems to act as a teammate to support future operations.

show abstract

Learning Self-Confidence from Semantic Action Embeddings for Improved Trust in Human-Robot Interaction

Goubard,

Demiris

2024

2024 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

“I'm Confident This Will End Poorly”: Robot Proficiency Self-Assessment in Human-Robot Teaming

Cited by 4 publications

References 23 publications

Event-triggered robot self-assessment to aid in autonomy adjustment

Event-triggered robot self-assessment to aid in autonomy adjustment

Comparison and synthesis of two aerospace case studies to develop human-autonomy teaming requirements

Learning Self-Confidence from Semantic Action Embeddings for Improved Trust in Human-Robot Interaction

Contact Info

Product

Resources

About