[POSTER] Remote Mixed Reality System Supporting Interactions with Virtualized Objects

Abstract

Figure 1: Example of remote collaborative work. From left to right: A local user with real objects. A remote user without real objects. The local user observes the operation performed by the remote user. Result of operation performed by the remote user. The local user operates the real objects. Final result of the remote collaborative work. ABSTRACTMixed Reality (MR) can merge real and virtual worlds seamlessly. This paper proposes a method to realize smooth collaboration using a remote MR, which makes it possible for geographically distributed users to share the same objects and communicate in real time as if they are at the same place. In this paper, we consider a situation where the users at local and remote sites perform a collaborative work, and real objects to be operated exist only at the local site. It is necessary to share the real objects between the two sites. However, prior studies have shown sharing real objects by duplication is either too costly or unrealistic. Therefore, we propose a method to share the objects by virtualizing the real objects using Computer Vision (CV) and then rendering the virtualized objects using MR. We have proposed a remote collaborative work system to create a smoother user experience for collaborative work with virtualized objects for remote users. Through experiments, we confirmed the effectiveness of our approach.

Mixed reality in design prototyping: A systematic review

“…MR can enable remote working and editing of prototypes by providing a method to edit multiple instances in multiple locations in parallel. Two remote teams may simultaneously view and interact with design changes, notes, and information, thereby creating a real-time network of prototypes and environments (Yang et al, 2015;Zillner et al, 2018). Peng et al (2018) have created a system that allows gestural remote 3D printing using a robotic 3D printer and an AR controller.…”

“…Zhang and Kwok (2018) demonstrate using a mobile-based AR interface to sketch over a physical prototype, with a 3D printer able to realise the changes directly onto a set of remote 'twinned' prototype 'siblings'. Entire subsystems being worked on by remote teams may interface virtually with local subsystems, communicating in real-time with the remote team (Yang et al, 2015).…”

A Review on Methods and Systems for Remote Collaboration

“…Yang and Peng in [81] presented a remote collaboration solution based on mixed reality. The real objects were virtualized using computing virtualization and displayed at the remote user's site.…”