Cloud-based multimodal human-robot interaction simulator utilizing ROS and unity frameworks

Mizuchi, Yoshiaki; Inamura, Tetsunari

doi:10.1109/sii.2017.8279345

Cited by 39 publications

(16 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, studies have compared the interactive behaviors of robots in a VR environment and a real environment. The results confirm that the behaviors of a virtual robot and a robot in a real environment are the same (Mizuchi and Inamura, 2017). Virtual robots can also reduce the cost of experiments.…”

Section: Introductionsupporting

confidence: 69%

HAVEN: A Unity-based Virtual Robot Environment to Showcase HRI-based Augmented Reality

Cleaver,

Tang,

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Due to the COVID-19 pandemic, conducting Human-Robot Interaction (HRI) studies in person is not permissible due to social distancing practices to limit the spread of the virus. Therefore, a virtual reality (VR) simulation with a virtual robot may offer an alternative to real-life HRI studies. Like a real intelligent robot, a virtual robot can utilize the same advanced algorithms to behave autonomously. This paper introduces HAVEN (HRI-based Augmentation in a Virtual robot Environment using uNity), a VR simulation that enables users to interact with a virtual robot. The goal of this system design is to enable researchers to conduct HRI Augmented Reality studies using a virtual robot without being in a real environment. This framework also introduces two common HRI experiment designs: a hallway passing scenario and humanrobot team object retrieval scenario. Both reflect HAVEN's potential as a tool for future AR-based HRI studies.

show abstract

Section: Introductionsupporting

confidence: 69%

HAVEN: A Unity-based Virtual Robot Environment to Showcase HRI-based Augmented Reality

Cleaver,

Tang,

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We evaluated the data transfer performance (Mizuchi and Inamura, 2017) to compare the proposed method with the conventional JSON-based method. The experimental conditions presented in Figure 3 were adopted for the investigation, where a mobile robot tracked a walking person using an RGB-D sensor.…”

Section: Mechanism For Connecting Ros and Unitymentioning

confidence: 99%

SIGVerse: A Cloud-Based VR Platform for Research on Multimodal Human-Robot Interaction

Inamura

Mizuchi

2021

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Research on Human-Robot Interaction (HRI) requires the substantial consideration of an experimental design, as well as a significant amount of time to practice the subject experiment. Recent technology in virtual reality (VR) can potentially address these time and effort challenges. The significant advantages of VR systems for HRI are: 1) cost reduction, as experimental facilities are not required in a real environment; 2) provision of the same environmental and embodied interaction conditions to test subjects; 3) visualization of arbitrary information and situations that cannot occur in reality, such as playback of past experiences, and 4) ease of access to an immersive and natural interface for robot/avatar teleoperations. Although VR tools with their features have been applied and developed in previous HRI research, all-encompassing tools or frameworks remain unavailable. In particular, the benefits of integration with cloud computing have not been comprehensively considered. Hence, the purpose of this study is to propose a research platform that can comprehensively provide the elements required for HRI research by integrating VR and cloud technologies. To realize a flexible and reusable system, we developed a real-time bridging mechanism between the robot operating system (ROS) and Unity. To confirm the feasibility of the system in a practical HRI scenario, we applied the proposed system to three case studies, including a robot competition named RoboCup@Home. via these case studies, we validated the system’s usefulness and its potential for the development and evaluation of social intelligence via multimodal HRI.

show abstract

“…Figure 1 shows the configuration of the proposed platform, which is an extension of SIGVerse version 3 [2,4]. Unity 3D was used to construct the VR scene and implement the VR interface.…”

Section: Configuration Of the Vr Platformmentioning

confidence: 99%

“…The robot's voice is relayed through the speakers of the HMD. The robot itself is controlled using the ROS software written by the competitors -the control commands and the emulated sensor information are communicated between Unity and ROS in real time [4].…”

Section: Configuration Of the Vr Platformmentioning

confidence: 99%

VR platform enabling crowdsourcing of embodied HRI experiments – case study of online robot competition

2021

Self Cite

View full text Add to dashboard Cite

The COVID-19 pandemic has made it difficult to conduct research involving human-robot interactions (HRIs). As a result, many robot competitions have been canceled in the past year. To address this problem, we propose a cloud-based VR platform for the crowdsourcing of embodied cognition with multiple human users and a virtual robot. The feasibility of the platform is evaluated from the latency of the interactive events and the motions of avatars, based on the number of users logging in to the VR scene and the distance between the local and server environments. We also discuss the feasibility of our system when dealing with large-scale datasets in embodied HRI experiments.

show abstract

Cloud-based multimodal human-robot interaction simulator utilizing ROS and unity frameworks

Cited by 39 publications

References 20 publications

HAVEN: A Unity-based Virtual Robot Environment to Showcase HRI-based Augmented Reality

HAVEN: A Unity-based Virtual Robot Environment to Showcase HRI-based Augmented Reality

SIGVerse: A Cloud-Based VR Platform for Research on Multimodal Human-Robot Interaction

VR platform enabling crowdsourcing of embodied HRI experiments – case study of online robot competition

Contact Info

Product

Resources

About