Becoming Team Members: Identifying Interaction Patterns of Mutual Adaptation for Human-Robot Co-Learning

Abstract

Becoming a well-functioning team requires continuous collaborative learning by all team members. This is called co-learning, conceptualized in this paper as comprising two alternating iterative stages: partners adapting their behavior to the task and to each other (co-adaptation), and partners sustaining successful behavior through communication. This paper focuses on the first stage in human-robot teams, aiming at a method for the identification of recurring behaviors that indicate co-learning. Studying this requires a task context that allows for behavioral adaptation to emerge from the interactions between human and robot. We address the requirements for conducting research into co-adaptation by a human-robot team, and designed a simplified computer simulation of an urban search and rescue task accordingly. A human participant and a virtual robot were instructed to discover how to collaboratively free victims from the rubbles of an earthquake. The virtual robot was designed to be able to real-time learn which actions best contributed to good team performance. The interactions between human participants and robots were recorded. The observations revealed patterns of interaction used by human and robot in order to adapt their behavior to the task and to one another. Results therefore show that our task environment enables us to study co-learning, and suggest that more participant adaptation improved robot learning and thus team level learning. The identified interaction patterns can emerge in similar task contexts, forming a first description and analysis method for co-learning. Moreover, the identification of interaction patterns support awareness among team members, providing the foundation for human-robot communication about the co-adaptation (i.e., the second stage of co-learning). Future research will focus on these human-robot communication processes for co-learning.

Use of cross‐training in human–robot collaborative rescue

“…They achieve a potential communication by exchanging tasks to understand each other's priorities. The impact of cross‐training on human–robot team in our research conclusions has laid a theoretical foundation for the collaborative learning of human–robot team in the research of van Zoelen et al (2021). Chiou et al (2021) researched the trade‐offs of explanation‐based communication strategies in a virtual search and rescue task, which is dealing with a team of human and autonomous robot.…”

Meaningful human control and variable autonomy in human-robot teams for firefighting

“…Figure 2 shows an example of what a simulated task could look like for dynamic task allocation using variable autonomy in human-robot teams for firefighting. This simulated task is especially valuable for evaluating human-robot collaboration before real-world deployment, enabling easier manipulation and evaluation of aspects like robot communication and behavior ( Van Zoelen et al, 2021 ; Schadd et al, 2022 ; Schoonderwoerd et al, 2022 ; Verhagen et al, 2022 ). For example, the simulated task in Figure 2 allows the evaluation of different robot explanations such as textual, visual, or hybrid explanations ( Szymanski et al, 2021 ).…”

Feasibility study of personalized speed adaptation method based on mental state for teleoperated robots

“…For a "human-in-the-loop" reinforcement learning training process, it's arguably better for the algorithm to learn certain repetitive subsequences of actions (or patterns of actions) and store them in a rule-based fashion. Once an action pattern has been shown to be successful in multiple instances of a task context, it can be applied in similar other task contexts (Wen et al, 2020;van Zoelen et al, 2021). Therefore, we believed that research on reinforcement learning of "human-in-the-loop" should not use conventional convergence criteria as a criterion for whether a model is valid, but should focus on whether a successful action pattern emerges, and its sustainability.…”

“…Once an action pattern has been shown to be successful in multiple instances of a task context, it can be applied in similar other task contexts. This reinforcement learning convergence strategy can cope well with dynamic task environments (Wen et al, 2020;van Zoelen et al, 2021). It is worth noting that the mental state in this article does not refer to a specific discrete state, but a continuous state, which is mainly evaluated by the two indicators of arousal and valence in the dimension theory of psychology (Russell, 2003).…”