A Survey on Trust in Autonomous Systems

Shahrdar, Shervin; Menezes, Luiza; Nojoumian, Mehrdad

doi:10.1007/978-3-030-01177-2_27

Cited by 31 publications

(16 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The loss of control over interpretability of decision making becoming a serious concern for high impact problems [9]. These expectations from policymakers are in line with results of other surveys done in the area of trust in autonomous systems [10] and where the end-users cite 'explainability' as one of the prerequisites for trust in such systems [11]. End-users and decision-makers who use the recommendations of the intelligent system require explanations to assure confidence and trust over the system through direct intervention [12].…”

Section: Introductionsupporting

confidence: 73%

Explainable Artificial Intelligence for Developing Smart Cities Solutions

et al. 2020

View full text Add to dashboard Cite

Traditional Artificial Intelligence (AI) technologies used in developing smart cities solutions, Machine Learning (ML) and recently Deep Learning (DL), rely more on utilising best representative training datasets and features engineering and less on the available domain expertise. We argue that such an approach to solution development makes the outcome of solutions less explainable, i.e., it is often not possible to explain the results of the model. There is a growing concern among policymakers in cities with this lack of explainability of AI solutions, and this is considered a major hindrance in the wider acceptability and trust in such AI-based solutions. In this work, we survey the concept of ‘explainable deep learning’ as a subset of the ‘explainable AI’ problem and propose a new solution using Semantic Web technologies, demonstrated with a smart cities flood monitoring application in the context of a European Commission-funded project. Monitoring of gullies and drainage in crucial geographical areas susceptible to flooding issues is an important aspect of any flood monitoring solution. Typical solutions for this problem involve the use of cameras to capture images showing the affected areas in real-time with different objects such as leaves, plastic bottles etc., and building a DL-based classifier to detect such objects and classify blockages based on the presence and coverage of these objects in the images. In this work, we uniquely propose an Explainable AI solution using DL and Semantic Web technologies to build a hybrid classifier. In this hybrid classifier, the DL component detects object presence and coverage level and semantic rules designed with close consultation with experts carry out the classification. By using the expert knowledge in the flooding context, our hybrid classifier provides the flexibility on categorising the image using objects and their coverage relationships. The experimental results demonstrated with a real-world use case showed that this hybrid approach of image classification has on average 11% improvement (F-Measure) in image classification performance compared to DL-only classifier. It also has the distinct advantage of integrating experts’ knowledge on defining the decision-making rules to represent the complex circumstances and using such knowledge to explain the results.

show abstract

Section: Introductionsupporting

confidence: 73%

Explainable Artificial Intelligence for Developing Smart Cities Solutions

et al. 2020

View full text Add to dashboard Cite

show abstract

“…This approach is important for operators to believe that the system will consider any deterioration in their health/ability to work and adjust their workload accordingly, such that they are never overwhelmed. The proposed framework therefore facilitates cooperation between agents and could be beneficial in studying inter-agent trust and trust between human operators and the autonomous systems itself following [72], [73] in MH-MR applications.…”

Section: Discussionmentioning

confidence: 99%

Adaptive Workload Allocation for Multi-Human Multi-Robot Teams for Independent and Homogeneous Tasks

Mina

Kannan

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Multi-human multi-robot (MH-MR) systems have the ability to combine the potential advantages of robotic systems with those of having humans in the loop. Robotic systems contribute precision performance and long operation on repetitive tasks without tiring, while humans in the loop improve situational awareness and enhance decision-making abilities. A system's ability to adapt allocated workload to changing conditions and the performance of each individual (human and robot) during the mission is vital to maintaining overall system performance. Previous works from literature including market-based and optimization approaches have attempted to address the task/workload allocation problem with focus on maximizing the system output without regarding individual agent conditions, lacking in real-time processing and have mostly focused exclusively on multi-robot systems. Given the variety of possible combination of teams (autonomous robots and human-operated robots: any number of human operators operating any number of robots at a time) and the operational scale of MH-MR systems, development of a generalized framework of workload allocation has been a particularly challenging task. In this paper, we present such a framework for independent homogeneous missions, capable of adaptively allocating the system workload in relation to health conditions and work performances of human-operated and autonomous robots in real-time. The framework consists of removable modular function blocks ensuring its applicability to different MH-MR scenarios. A new workload transition function block ensures smooth transition without the workload change having adverse effects on individual agents. The effectiveness and scalability of the system's workload adaptability is validated by experiments applying the proposed framework in a MH-MR patrolling scenario with changing human and robot condition, and failing robots.

show abstract

“…A literature review of research covering trust in autonomous systems, mostly on materials published in the last decade, was conducted to build on the previous studies carried out in [16] and [17] using combinations of key terms, such as trust, trust in autonomous systems, trust measurement, trust scales, trustworthiness, and trust calibration.…”

Section: Background and Definitionsmentioning

confidence: 99%

“…This layered and structured approach comprises dispositional, situational, and learned trust. Another review [17] summarized the research concerning autonomous systems by considering four categories of such systems, namely decision support systems, robots, self-driving cars, and autopilot systems. This article provided a quantitative definition of trust that appeared in the authors' prior work [18].…”

Section: Introductionmentioning

confidence: 99%

A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives

Gebru

Zeleke

Blankson

et al. 2022

IEEE Trans. Human-Mach. Syst.

View full text Add to dashboard Cite

As complex autonomous systems become increasingly ubiquitous, their deployment and integration into our daily lives will become a significant endeavor. Human-machine trust relationship is now acknowledged as one of the primary aspects that characterize a successful integration. In the context of human-machine interaction (HMI), proper use of machines and autonomous systems depends both on the human and machine counterparts. On one hand, it depends on how well the human relies on the machine regarding the situation or task at hand based on willingness and experience. On the other hand, it depends on how well the machine carries out the task and how well it conveys important information on how the job is done. Furthermore, proper calibration of trust for effective HMI requires the factors affecting trust to be properly accounted for and their relative importance to be rightly quantified. In this article, the functional understanding of humanmachine trust is viewed from two perspectives-human-centric and machine-centric. The human aspect of the discussion outlines factors, scales, and approaches, which are available to measure and calibrate human trust. The discussion on the machine aspect spans trustworthy artificial intelligence, built-in machine assurances, and ethical frameworks of trustworthy machines.

show abstract

A Survey on Trust in Autonomous Systems

Cited by 31 publications

References 36 publications

Explainable Artificial Intelligence for Developing Smart Cities Solutions

Explainable Artificial Intelligence for Developing Smart Cities Solutions

Adaptive Workload Allocation for Multi-Human Multi-Robot Teams for Independent and Homogeneous Tasks

A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives

Contact Info

Product

Resources

About