Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction

Teo, Theophilus; Lawrence, Louise; Lee, Gun; Billinghurst, Mark; Adcock, Matt

doi:10.1145/3290605.3300431

Cited by 147 publications

(53 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonverbal cues are often missing in remote collaboration solutions in XR. They are important since they can affect user performance even more than lack of verbal communication (Teo et al 2019). (Jo et al 2017) found that presenting avatars in a real/AR background helped to improve trust in collaboration and co-presence.…”

Section: Related Workmentioning

confidence: 99%

Running an XR lab in the context of COVID-19 pandemic: Lessons learned from a Norwegian university

Estrada

Prasolova‐Førland

2021

Educ Inf Technol

View full text Add to dashboard Cite

Universities and companies were not prepared to the changes introduced to limit the spread of COVID-19 in Norway. Universities had to switch to online teaching overnight. There is still uncertainty how measures to control the pandemic will keep affecting universities in the short and middle term. Such measures have consequences on how to carry out research that usually relies on students, researcher and volunteers using the equipment and applications. Our group carries out research on virtual/augmented/extended reality (VR/AR/XR) for immersive training and learning. This research often involves user studies. We had established procedures on how to use the equipment, carry out demonstrations and teaching for students, teachers and visitors, develop projects as part of bachelor and master projects and test new applications with volunteers. The measures taken by authorities to control the spread of the pandemic made it difficult or unfeasible to carry out some of those activities. In this paper we describe how our group and XR lab reacted after universities were closed to students’ presence in campus in March 2020. We present our actions to keep research ongoing, our evaluation of some of those actions and discuss how we had to change the way we operate our XR lab in order to continue teaching and research in the near future, under the assumption that restrictions due to the pandemic can be re-implemented at short notice. We propose procedures to run an XR lab in a manner that inspires visitors to feel safe and confident of using the equipment. Our contribution is the proposal of procedures to run an educational XR lab safely and contribute towards the conversation about how to carry out research involving users in XR under pandemic restrictions.

show abstract

Section: Related Workmentioning

confidence: 99%

Running an XR lab in the context of COVID-19 pandemic: Lessons learned from a Norwegian university

Estrada

Prasolova‐Førland

2021

Educ Inf Technol

View full text Add to dashboard Cite

show abstract

“…This is especially important in situations where contextual information is required to provide a rich understanding of the environment in order for people to communicate and interact with both other users and the environment itself. However, this can be especially challenging for the virtual environment that recreates real physical spaces using depthcameras, as it may limit the sense of the presence of the user [62].…”

Section: Motivationmentioning

confidence: 99%

TeleGate: Multi-user Semi-teleportation for Remote Collaboration in Mixed Reality 360-Videos

Kim¹

2021

Preprint

View full text Add to dashboard Cite

Telepresence systems enable people to feel present in a remote space while their bodies remain in their local space. To enhance telepresence, the remote environment needs to be captured and visualised in an immersive way. For instance, 360-degree videos (360-videos) shown on head-mounted displays (HMDs) provide high fidelity telepresence in a remote place. Mixed reality (MR) in 360-videos enables interactions with virtual objects blended in the captured remote environment while it allows telepresence only for a single user wearing HMD. For this reason, it has limitations when multiple users want to experience telepresence together and naturally collaborate within a teleported space. This thesis presents TeleGate, a novel multi-user teleportation platform for remote collaboration in a MR space. TeleGate provides "semi-teleportation" into the MR space using large-scale displays, acting as a bridge between the local physical communication space and the remote collaboration space created by MR with captured 360-videos. Our proposed platform enables multi-user semi-teleportation to perform collaborative tasks in the remote MR collaboration (MRC) space while allowing for natural communication between collaborators in the same local physical space. We implemented a working prototype of TeleGate and then conducted a user study to evaluate our concept of semi-teleportation. We measured the spatial presence, social presence while participants performed remote collaborative tasks in the MRC space. Additionally, we also explored the different control mechanisms within the platform in the remote MR collaboration scenario. In conclusion, TeleGate enabled multiple co-located users to semi-teleport together using large-scale displays for remote collaboration in MR 360-videos.

show abstract

“…These are not trivial challenges, and proposed solutions are largely experimental [122,123]. Ideally, in a collaborative XR, people should (1) experience the presence of others (e.g., using avatars of the full body, or parts of the body such as hands) [124,125]; (2) be able to detect the gaze direction of others [126], and eventually, experience 'eye contact' [127]; (3) have on-demand access to what the others see ('shared field of view') [128,129]; (4) be able to share spatial context [123], especially in the case of remote collaboration (i.e., does it 'rain or shine' in one person's location, are they on the move, is it dark or light, are they looking at a water body? ); (5) be able to use virtual gestures (handshake, wave, nod, other nonverbal communication) [129,130]; (6) be able to add proper annotations to scenes and objects and see others' annotations; and last but not least (7), be able to 'read' the emotional reactions of their collaboration partner [131].…”

Section: Interaction Designmentioning

confidence: 99%

Extended Reality in Spatial Sciences: A Review of Research Challenges and Future Directions

Çöltekin

Lochhead

Madden

et al. 2020

IJGI

186

View full text Add to dashboard Cite

This manuscript identifies and documents unsolved problems and research challenges in the extended reality (XR) domain (i.e., virtual (VR), augmented (AR), and mixed reality (MR)). The manuscript is structured to include technology, design, and human factor perspectives. The text is visualization/display-focused, that is, other modalities such as audio, haptic, smell, and touch, while important for XR, are beyond the scope of this paper. We further narrow our focus to mainly geospatial research, with necessary deviations to other domains where these technologies are widely researched. The main objective of the study is to provide an overview of broader research challenges and directions in XR, especially in spatial sciences. Aside from the research challenges identified based on a comprehensive literature review, we provide case studies with original results from our own studies in each section as examples to demonstrate the relevance of the challenges in the current research. We believe that this paper will be of relevance to anyone who has scientific interest in extended reality, and/or uses these systems in their research.

show abstract

Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction

Cited by 147 publications

References 27 publications

Running an XR lab in the context of COVID-19 pandemic: Lessons learned from a Norwegian university

Running an XR lab in the context of COVID-19 pandemic: Lessons learned from a Norwegian university

TeleGate: Multi-user Semi-teleportation for Remote Collaboration in Mixed Reality 360-Videos

Extended Reality in Spatial Sciences: A Review of Research Challenges and Future Directions

Contact Info

Product

Resources

About