Towards Trusting Autonomous Systems

Winikoff, Michael

doi:10.1007/978-3-319-91899-0_1

Cited by 19 publications

(12 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The loss of control over interpretability of decision making becoming a serious concern for high impact problems [9]. These expectations from policymakers are in line with results of other surveys done in the area of trust in autonomous systems [10] and where the end-users cite 'explainability' as one of the prerequisites for trust in such systems [11]. End-users and decision-makers who use the recommendations of the intelligent system require explanations to assure confidence and trust over the system through direct intervention [12].…”

Section: Introductionsupporting

confidence: 69%

Explainable Artificial Intelligence for Developing Smart Cities Solutions

et al. 2020

View full text Add to dashboard Cite

Traditional Artificial Intelligence (AI) technologies used in developing smart cities solutions, Machine Learning (ML) and recently Deep Learning (DL), rely more on utilising best representative training datasets and features engineering and less on the available domain expertise. We argue that such an approach to solution development makes the outcome of solutions less explainable, i.e., it is often not possible to explain the results of the model. There is a growing concern among policymakers in cities with this lack of explainability of AI solutions, and this is considered a major hindrance in the wider acceptability and trust in such AI-based solutions. In this work, we survey the concept of ‘explainable deep learning’ as a subset of the ‘explainable AI’ problem and propose a new solution using Semantic Web technologies, demonstrated with a smart cities flood monitoring application in the context of a European Commission-funded project. Monitoring of gullies and drainage in crucial geographical areas susceptible to flooding issues is an important aspect of any flood monitoring solution. Typical solutions for this problem involve the use of cameras to capture images showing the affected areas in real-time with different objects such as leaves, plastic bottles etc., and building a DL-based classifier to detect such objects and classify blockages based on the presence and coverage of these objects in the images. In this work, we uniquely propose an Explainable AI solution using DL and Semantic Web technologies to build a hybrid classifier. In this hybrid classifier, the DL component detects object presence and coverage level and semantic rules designed with close consultation with experts carry out the classification. By using the expert knowledge in the flooding context, our hybrid classifier provides the flexibility on categorising the image using objects and their coverage relationships. The experimental results demonstrated with a real-world use case showed that this hybrid approach of image classification has on average 11% improvement (F-Measure) in image classification performance compared to DL-only classifier. It also has the distinct advantage of integrating experts’ knowledge on defining the decision-making rules to represent the complex circumstances and using such knowledge to explain the results.

show abstract

Section: Introductionsupporting

confidence: 69%

Explainable Artificial Intelligence for Developing Smart Cities Solutions

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In [151], the author argues that trust (in an autonomous system) at least requires a framework for recourse, the system's ability to give explanations, and verification and validation of the system.…”

Section: Trust and Transparencymentioning

confidence: 99%

Agents and Robots for Reliable Engineered Autonomy:A Perspective from the Organisers of AREA 2020

Cardoso

Ferrando

Briola

et al. 2021

JSAN

View full text Add to dashboard Cite

Multi-agent systems, robotics and software engineering are large and active research areas with many applications in academia and industry. The First Workshop on Agents and Robots for reliable Engineered Autonomy (AREA), organised the first time in 2020, aims at encouraging cross-disciplinary collaborations and exchange of ideas among researchers working in these research areas. This paper presents a perspective of the organisers that aims at highlighting the latest research trends, future directions, challenges, and open problems. It also includes feedback from the discussions held during the AREA workshop. The goal of this perspective is to provide a high-level view of current research trends for researchers that aim at working in the intersection of these research areas.

show abstract

“…Winikoff [37] considers the question of the trustability of autonomous systems, i.e., how humans can come to trust them, and proposes three prerequisites for such trust: there should be a social framework for recourse; if the system makes a decision with negative consequences for the user, the system should be able to explain its behaviour; and the system should be subject to verification and validation to give assurance that key behavioural properties hold.…”

Section: Related Conceptsmentioning

confidence: 99%

Accountability for Practical Reasoning Agents

Cranefield

Oren

Vasconcelos

2019

Agreement Technologies

View full text Add to dashboard Cite

Artificial intelligence has been increasing the autonomy of man-made artefacts such as software agents, self-driving vehicles and military drones. This increase in autonomy together with the ubiquity and impact of such artefacts in our daily lives have raised many concerns in society. Initiatives such as transparent and ethical AI aim to allay fears of a "free for all" future where amoral technology (or technology amorally designed) will replace humans with terrible consequences. We discuss the notion of accountable autonomy, and explore this concept within the context of practical reasoning agents. We survey literature from distinct fields such as management, healthcare, policy-making, and others, and differentiate and relate concepts connected to accountability. We present a list of justified requirements for accountable software agents and discuss research questions stemming from these requirements. We also propose a preliminary formalisation of one core aspect of accountability: responsibility.

show abstract

Towards Trusting Autonomous Systems

Cited by 19 publications

References 47 publications

Explainable Artificial Intelligence for Developing Smart Cities Solutions

Explainable Artificial Intelligence for Developing Smart Cities Solutions

Agents and Robots for Reliable Engineered Autonomy:A Perspective from the Organisers of AREA 2020

Accountability for Practical Reasoning Agents

Contact Info

Product

Resources

About