Design patterns for human-AI co-learning: A wizard-of-Oz evaluation in an urban-search-and-rescue task

Schoonderwoerd, Tjeerd; Zoelen, Emma M. van; Bosch, Karel Van den; Neerincx, Mark A.

doi:10.1016/j.ijhcs.2022.102831

Cited by 22 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 2 shows an example of what a simulated task could look like for dynamic task allocation using variable autonomy in human-robot teams for firefighting. This simulated task is especially valuable for evaluating human-robot collaboration before real-world deployment, enabling easier manipulation and evaluation of aspects like robot communication and behavior ( Van Zoelen et al, 2021 ; Schadd et al, 2022 ; Schoonderwoerd et al, 2022 ; Verhagen et al, 2022 ). For example, the simulated task in Figure 2 allows the evaluation of different robot explanations such as textual, visual, or hybrid explanations ( Szymanski et al, 2021 ).…”

Section: Discussionmentioning

confidence: 99%

Meaningful human control and variable autonomy in human-robot teams for firefighting

Verhagen,

Neerincx,

Tielman

2024

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Introduction: Humans and robots are increasingly collaborating on complex tasks such as firefighting. As robots are becoming more autonomous, collaboration in human-robot teams should be combined with meaningful human control. Variable autonomy approaches can ensure meaningful human control over robots by satisfying accountability, responsibility, and transparency. To verify whether variable autonomy approaches truly ensure meaningful human control, the concept should be operationalized to allow its measurement. So far, designers of variable autonomy approaches lack metrics to systematically address meaningful human control.Methods: Therefore, this qualitative focus group (n = 5 experts) explored quantitative operationalizations of meaningful human control during dynamic task allocation using variable autonomy in human-robot teams for firefighting. This variable autonomy approach requires dynamic allocation of moral decisions to humans and non-moral decisions to robots, using robot identification of moral sensitivity. We analyzed the data of the focus group using reflexive thematic analysis.Results: Results highlight the usefulness of quantifying the traceability requirement of meaningful human control, and how situation awareness and performance can be used to objectively measure aspects of the traceability requirement. Moreover, results emphasize that team and robot outcomes can be used to verify meaningful human control but that identifying reasons underlying these outcomes determines the level of meaningful human control.Discussion: Based on our results, we propose an evaluation method that can verify if dynamic task allocation using variable autonomy in human-robot teams for firefighting ensures meaningful human control over the robot. This method involves subjectively and objectively quantifying traceability using human responses during and after simulations of the collaboration. In addition, the method involves semi-structured interviews after the simulation to identify reasons underlying outcomes and suggestions to improve the variable autonomy approach.

show abstract

Section: Discussionmentioning

confidence: 99%

Meaningful human control and variable autonomy in human-robot teams for firefighting

Verhagen,

Neerincx,

Tielman

2024

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Yet, to the best of our knowledge, no research has examined the role of human-AI collaboration for context-aware systems that necessitates an actionable and holistic framework for designers seeking to combine AI and context-awareness technologies and enhance human-AI collaboration. In line with the prior literature (Peeters et al, 2021;Schoonderwoerd et al, 2022), we define human-AI collaboration as the process of developing a socio-technical system in which human intelligence and AI can effectively collaborate by exploiting their individual strengths and coevolve via mutual improvement (Loske and Klumpp, 2021). A higher degree of human-AI collaboration indicates that users are willing to promote the performance of AI by integrating their own intelligence; and the AI-enabled system keeps learning users to augment human intelligence at the same time.…”

Section: Research Directionsmentioning

confidence: 94%

“…In line with the prior literature (Peeters et al. , 2021; Schoonderwoerd et al. , 2022), we define human–AI collaboration as the process of developing a socio-technical system in which human intelligence and AI can effectively collaborate by exploiting their individual strengths and coevolve via mutual improvement (Loske and Klumpp, 2021).…”

Section: Research Directionsmentioning

confidence: 99%

Beyond AI-powered context-aware services: the role of human–AI collaboration

Jiang

Liu²,

Liu³

et al. 2022

IMDS

View full text Add to dashboard Cite

PurposeArtificial intelligence (AI) has gained significant momentum in recent years. Among AI-infused systems, one prominent application is context-aware systems. Although the fusion of AI and context awareness has given birth to personalized and timely AI-powered context-aware systems, several challenges still remain. Given the “black box” nature of AI, the authors propose that human–AI collaboration is essential for AI-powered context-aware services to eliminate uncertainty and evolve. To this end, this study aims to advance a research agenda for facilitators and outcomes of human–AI collaboration in AI-powered context-aware services.Design/methodology/approachSynthesizing the extant literature on AI and context awareness, the authors advance a theoretical framework that not only differentiates among the three phases of AI-powered context-aware services (i.e. context acquisition, context interpretation and context application) but also outlines plausible research directions for each stage.FindingsThe authors delve into the role of human–AI collaboration and derive future research questions from two directions, namely, the effects of AI-powered context-aware services design on human–AI collaboration and the impact of human–AI collaboration.Originality/valueThis study contributes to the extant literature by identifying knowledge gaps in human–AI collaboration for AI-powered context-aware services and putting forth research directions accordingly. In turn, their proposed framework yields actionable guidance for AI-powered context-aware service designers and practitioners.

show abstract

“…This approach is related to the Wizard of Oz technique in which the functionality of the system is simulated by a human (the "wizard"). This technique can be used to test the interaction between humans and intelligent systems that cannot be easily implemented or realized with available resources (Weiss et al, 2009;Wilson & Rosenberg, 1988;Schoonderwoerd et al, 2022). Here, the system's functionality was not simulated by a human in real time but manually implemented prior to the experiment.…”

Section: Methodsmentioning

confidence: 99%

Applying XAI to an AI-based system for candidate management to mitigate bias and discrimination in hiring

et al. 2022

View full text Add to dashboard Cite

Assuming that potential biases of Artificial Intelligence (AI)-based systems can be identified and controlled for (e.g., by providing high quality training data), employing such systems to augment human resource (HR)-decision makers in candidate selection provides an opportunity to make selection processes more objective. However, as the final hiring decision is likely to remain with humans, prevalent human biases could still cause discrimination. This work investigates the impact of an AI-based system’s candidate recommendations on humans’ hiring decisions and how this relation could be moderated by an Explainable AI (XAI) approach. We used a self-developed platform and conducted an online experiment with 194 participants. Our quantitative and qualitative findings suggest that the recommendations of an AI-based system can reduce discrimination against older and female candidates but appear to cause fewer selections of foreign-race candidates. Contrary to our expectations, the same XAI approach moderated these effects differently depending on the context.

show abstract

Design patterns for human-AI co-learning: A wizard-of-Oz evaluation in an urban-search-and-rescue task

Cited by 22 publications

References 22 publications

Meaningful human control and variable autonomy in human-robot teams for firefighting

Meaningful human control and variable autonomy in human-robot teams for firefighting

Beyond AI-powered context-aware services: the role of human–AI collaboration

Applying XAI to an AI-based system for candidate management to mitigate bias and discrimination in hiring

Contact Info

Product

Resources

About