Uncanny and Unsafe Valley of Assistance and Automation: First Sketch and Application to Vehicle Automation

Flemisch, Frank; Altendorf, Eugen; Canpolat, Yigiterkut; Weßel, Gina; Baltzer, Marcel; López, Daniel; Herzberger, Nicolas; Voß, Gudrun M. I.; Schwalm, Maximilian; Schutte, Paul C.

doi:10.1007/978-3-662-53305-5_23

Cited by 25 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But this, as we argue, cannot be granted if ADS are not under some relevant form of human control and there are gaps in responsibility. When this happens, the relevant human actors may not be sufficiently able, motivated and willing to prevent undesired outcomes (Elish, 2019 ; Flemisch et al, 2017 ). In addition, we consider human responsibility also to be important in order to prevent legitimate discontent among victims of accidents and distrust towards technology more generally (Danaher, 2016 ).…”

Section: Automated Driving Systems and Responsibilitymentioning

confidence: 99%

Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

et al. 2022

View full text Add to dashboard Cite

The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding

show abstract

Section: Automated Driving Systems and Responsibilitymentioning

confidence: 99%

Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Systems that parallel human-like characteristics and personas (i.e., humanoid robots, intelligent agents such as Alexa) tend to have more trust than systems designed with the same capacities and purpose, but with non-anthropomorphized characteristics (Hancock et al, 2011;de Visser et al, 2017;Calhoun et al, 2019). However, there is a point where extreme similarity between a technology and human can result in a significant drop in trust levels, often referred to as the uncanny valley (Flemisch et al, 2017). Because of this interaction between trust and human-like characteristics in technology, there are observable differences in trusting behavior, founded on emotional connection to the system rather than system capability (Jensen et al, 2020).…”

Section: Basis Of Trust In Interpersonal Verses Technology Automationmentioning

confidence: 99%

Neural Correlates of Trust in Automation: Considerations and Generalizability Between Technology Domains

Hopko

Mehta

2021

Front. Neuroergonomics

View full text Add to dashboard Cite

Investigations into physiological or neurological correlates of trust has increased in popularity due to the need for a continuous measure of trust, including for trust-sensitive or adaptive systems, measurements of trustworthiness or pain points of technology, or for human-in-the-loop cyber intrusion detection. Understanding the limitations and generalizability of the physiological responses between technology domains is important as the usefulness and relevance of results is impacted by fundamental characteristics of the technology domains, corresponding use cases, and socially acceptable behaviors of the technologies. While investigations into the neural correlates of trust in automation has grown in popularity, there is limited understanding of the neural correlates of trust, where the vast majority of current investigations are in cyber or decision aid technologies. Thus, the relevance of these correlates as a deployable measure for other domains and the robustness of the measures to varying use cases is unknown. As such, this manuscript discusses the current-state-of-knowledge in trust perceptions, factors that influence trust, and corresponding neural correlates of trust as generalizable between domains.

show abstract

“…Leaving aside the issue of driver's responsibility, which will be discussed in "Responsibility" section below, it is interesting to note that the riskiness of these handover situations was well-known, and this kind of accidents largely predicted by scientific studies. As reported in the discussion of Recommendation 2, handover scenarios require that the driver is given sufficient time to regain situational awareness (Flemisch et al, 2017), and in one simulator study none of the participants was able to regain control of the car within the 2 seconds they were given to react to a sudden failure in proximity of a curve (Flemisch et al, 2008).…”

Section: Safety and Freedommentioning

confidence: 99%

The European Commission report on ethics of connected and automated vehicles and the future of ethics of transportation

Sio

2021

Ethics Inf Technol

View full text Add to dashboard Cite

The paper has two goals. The first is presenting the main results of the recent report Ethics of Connected and Automated Vehicles: recommendations on road safety, privacy, fairness, explainability and responsibility written by the Horizon 2020 European Commission Expert Group to advise on specific ethical issues raised by driverless mobility, of which the author of this paper has been member and rapporteur. The second is presenting some broader ethical and philosophical implications of these recommendations, and using these to contribute to the establishment of Ethics of Transportation as an independent branch of applied ethics. The recent debate on the ethics of Connected and Automated Vehicles (CAVs) presents a paradox and an opportunity. The paradox is the presence of a flourishing debate on the ethics of one very specific transportation technology without ethics of transportation being in itself a well-established academic discipline. The opportunity is that now that a spotlight has been switched on the ethical dimensions of CAVs it may be easier to establish a broader debate on ethics of transportation. While the 20 recommendations of the EU report are grouped in three macro-areas: road safety, data ethics, and responsibility, in this paper they will be grouped according to eight philosophical themes: Responsible Innovation, road justice, road safety, freedom, human control, privacy, data fairness, responsibility. These are proposed as the first topics for a new ethics of transportation.

show abstract

Uncanny and Unsafe Valley of Assistance and Automation: First Sketch and Application to Vehicle Automation

Cited by 25 publications

References 19 publications

Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

Neural Correlates of Trust in Automation: Considerations and Generalizability Between Technology Domains

The European Commission report on ethics of connected and automated vehicles and the future of ethics of transportation

Contact Info

Product

Resources

About