Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

Sio, Filippo Santoni de; Mecacci, Giulio; Calvert, Simeon C.; Heikoop, D.D.; Hagenzieker, Marjan; Arem, Bart van

doi:10.1007/s11023-022-09608-8

Cited by 8 publications

(7 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The physician must therefore not only be able to operate the AI-DSS, but also be able to assess in which cases the system should be used and how the recommendations of an AI-DSS should be interpreted in a context-specific manner. Careful attention should be devoted to how the interaction between an available AI-DSS and the user – in our case the physician – should be organized ( 40 ).…”

Section: Discussionmentioning

confidence: 99%

“Nothing works without the doctor:” Physicians’ perception of clinical decision-making and artificial intelligence

et al. 2022

View full text Add to dashboard Cite

IntroductionArtificial intelligence–driven decision support systems (AI–DSS) have the potential to help physicians analyze data and facilitate the search for a correct diagnosis or suitable intervention. The potential of such systems is often emphasized. However, implementation in clinical practice deserves continuous attention. This article aims to shed light on the needs and challenges arising from the use of AI-DSS from physicians’ perspectives.MethodsThe basis for this study is a qualitative content analysis of expert interviews with experienced nephrologists after testing an AI-DSS in a straightforward usage scenario.ResultsThe results provide insights on the basics of clinical decision-making, expected challenges when using AI-DSS as well as a reflection on the test run.DiscussionWhile we can confirm the somewhat expectable demand for better explainability and control, other insights highlight the need to uphold classical strengths of the medical profession when using AI-DSS as well as the importance of broadening the view of AI-related challenges to the clinical environment, especially during treatment. Our results stress the necessity for adjusting AI-DSS to shared decision-making. We conclude that explainability must be context-specific while fostering meaningful interaction with the systems available.

show abstract

Section: Discussionmentioning

confidence: 99%

“Nothing works without the doctor:” Physicians’ perception of clinical decision-making and artificial intelligence

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Simple accuracy, represented by (29), calculates the ratio of the number of true samples out of the total. Precision, represented by (30), is the ratio of the number of TP samples out of the predicted positives.…”

Section: Performance Evaluation Of the Modelmentioning

confidence: 99%

“…Recent AV-related accident reports have also pointed out the potential for human drivers to misjudge unfamiliar driving behaviors exhibited by AVs [12]. Therefore, AV driving strategies should be developed to resemble HV driving behaviors for better traffic flow in the near future [13].…”

Section: Introductionmentioning

confidence: 99%

“…Meanwhile, support vector machine (SVM) can enhance model performance while maintaining the interpretability of driving behaviors [29]. Mandalia and Salvucci [30] utilized SVM for LC detection and tested five input sets, each containing four to seven features (e.g., heading, lane position directly ahead of the vehicle, acceleration, and steering angle).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modeling Lane-Changing Behaviors for Autonomous Vehicles Based on Vehicle-to-Vehicle Communication

Lee,

Han,

Lee

et al. 2023

IEEE Access

View full text Add to dashboard Cite

With the advent of autonomous vehicles (AVs) and advanced driving assistance systems (ADAS), there has been a growing interest in studying driving behaviors within the field of transportation science. Given that the transition period of mixed traffic is expected to continue for more than 30 years, it is crucial to evolve AV technology to resemble human driving, especially in the freeway weaving sections. Lane-changing (LC) maneuvers in these sections could cause problems for traffic flow, such as traffic breakdown, oscillation, or bottleneck activation. This study proposes an interpretable LC implementation model for naturalistic driving behaviors of AVs based on vehicle-to-vehicle (V2V) communication. To achieve this objective, a systematic selection process is adopted to find optimal V2V features that resemble how human drivers assess LC situations. Based on the minimum redundancy maximum relevance (mRMR) algorithm, seven V2V features have been selected out of 25 candidates. Then, a support vector machine (SVM) is employed to investigate how these features exhibit in each of LC and lane-keeping (LK) situations. The proposed model was applied in a field case of a weaving section on freeway US 101. Performance measures of simple accuracy, precision, recall, and F1-score show high accuracy of 0.9814, 0.9150, 0.7955, and 0.8511, respectively. Subsequently, a strategy for naturalistic LC behaviors of AVs was simulated. The proposed model outperforms high prediction accuracy compared to other existing models. Particularly, errors in the lateral movements have significantly improved. These results suggest that the proposed model effectively simulates naturalistic LC behaviors based on V2V communication.

show abstract

“…In contrast to operationalizing the whole concept of meaningful human control, other studies operationalized only its tracing condition. For example, the cascade evaluation approach subjectively quantifies traceability ( Calvert et al, 2020 ; Calvert and Mecacci, 2020 ; de Sio et al, 2022 ). This approach is centered around four aspects: 1) The exertion of operational control; 2) the involvement of a human agent; 3) the ability of a human agent to understand and interact with a robot; and 4) the ability of a human agent to understand their moral responsibility over a robot.…”

Section: Introductionmentioning

confidence: 99%

Meaningful human control and variable autonomy in human-robot teams for firefighting

Verhagen,

Neerincx,

Tielman

2024

Front. Robot. AI

View full text Add to dashboard Cite

Introduction: Humans and robots are increasingly collaborating on complex tasks such as firefighting. As robots are becoming more autonomous, collaboration in human-robot teams should be combined with meaningful human control. Variable autonomy approaches can ensure meaningful human control over robots by satisfying accountability, responsibility, and transparency. To verify whether variable autonomy approaches truly ensure meaningful human control, the concept should be operationalized to allow its measurement. So far, designers of variable autonomy approaches lack metrics to systematically address meaningful human control.Methods: Therefore, this qualitative focus group (n = 5 experts) explored quantitative operationalizations of meaningful human control during dynamic task allocation using variable autonomy in human-robot teams for firefighting. This variable autonomy approach requires dynamic allocation of moral decisions to humans and non-moral decisions to robots, using robot identification of moral sensitivity. We analyzed the data of the focus group using reflexive thematic analysis.Results: Results highlight the usefulness of quantifying the traceability requirement of meaningful human control, and how situation awareness and performance can be used to objectively measure aspects of the traceability requirement. Moreover, results emphasize that team and robot outcomes can be used to verify meaningful human control but that identifying reasons underlying these outcomes determines the level of meaningful human control.Discussion: Based on our results, we propose an evaluation method that can verify if dynamic task allocation using variable autonomy in human-robot teams for firefighting ensures meaningful human control over the robot. This method involves subjectively and objectively quantifying traceability using human responses during and after simulations of the collaboration. In addition, the method involves semi-structured interviews after the simulation to identify reasons underlying outcomes and suggestions to improve the variable autonomy approach.

show abstract

Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

Cited by 8 publications

References 63 publications

“Nothing works without the doctor:” Physicians’ perception of clinical decision-making and artificial intelligence

“Nothing works without the doctor:” Physicians’ perception of clinical decision-making and artificial intelligence

Modeling Lane-Changing Behaviors for Autonomous Vehicles Based on Vehicle-to-Vehicle Communication

Meaningful human control and variable autonomy in human-robot teams for firefighting

Contact Info

Product

Resources

About