Interactive Augmented Live Virtual Reality Streaming

Teng, Cecilia; Jensen, Nathan; Smith, Timothy B.; Forbush, Talon; Fletcher, Kyle T; Hoover, Michael

doi:10.1145/3239438.3239456

Cited by 11 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This system recognizes body motions, allowing the user to see his/her movements, and provides health status using the worn medical sensors. In [47], a proof-of-concept for a tele-medicine technology platform was presented. It provides an interactive experience through a 360 • video in an augmented virtual world.…”

Section: B Immersive Remote Servicesmentioning

confidence: 99%

VR-Based Immersive Service Management in B5G Mobile Systems: A UAV Command and Control Use Case

Taleb

Sehad

Nadir

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

The management of remote services, such as remote surgery, remote sensing, or remote driving, has become increasingly important, especially with the emerging 5G and Beyond 5G technologies. However, the strict network requirements of these remote services represent one of the major challenges that hinder their fast and large-scale deployment in critical infrastructures. This article addresses certain issues inherent in remote and immersive control of virtual reality (VR)-based unmanned aerial vehicles (UAVs), whereby a user remotely controls UAVs, equipped with 360 • cameras, using their head-mounted devices (HMD) and their respective controllers. Remote and immersive control services, using 360 • video streams, require much lower latency and higher throughput for true immersion and high service reliability. To assess and analyze these requirements, this article introduces a real-life testbed system that leverages different technologies (e.g., VR, 360 • video streaming over 4G/5G, and edge computing). In the performance evaluation, different latency types are considered. They are namely: 1) glass-to-glass latency between the 360 • camera of a remote UAV and the HMD display; 2) user/pilot's reaction latency; and 3) the command/execution latency. The obtained results indicate that the responsiveness (dubbed Glass-to-Reaction-to-Execution-GRElatency) of a pilot, using our system, to a sudden event is within an acceptable range, i.e., around 900 ms.

show abstract

Section: B Immersive Remote Servicesmentioning

confidence: 99%

VR-Based Immersive Service Management in B5G Mobile Systems: A UAV Command and Control Use Case

Taleb

Sehad

Nadir

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…The study carried out by Teng et al exposes a telemedicine application for streaming video captured by a camera to a virtual reality headset [17]. In this case they use the ''Acer Windows MR'' headset and use the RTSP and HLS protocols to achieve integration with virtual reality, since RTSP itself is not generally supported by VR applications.…”

Section: Related Workmentioning

confidence: 99%

Health-5G: A Mixed Reality-Based System for Remote Medical Assistance in Emergency Situations

García,

Moraleda,

Schez-Sobrino

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Mixed reality is the combination of virtual and augmented reality to interactively and believably merge physical and computer-generated environments. This paper discusses the design of Health-5G, a scalable mixed reality-based system that facilitates and supports emergency response by medical emergency teams. Health-5G is supported by a distributed architecture divided into four interrelated applications responsible for advanced computer-human interaction, effective real-time videoconference, medical device integration, and communication infrastructure, respectively. The mixed reality layer is provided by the headset Microsoft Hololens 2™. Health-5G is based on scenarios in which emergency personnel wear mixed reality glasses that can transmit audio, video, and data streams bidirectionally over a 5G network to medical specialists stationed in a hospital at any distance. Thanks to Health-5G, the specialist will be able to access the emergency team's point of view at any time and provide verbal and visual instructions, including gestures and positioning of graphical markers in 3D space. In this way, emergency personnel can provide the best possible care to the patient without having to wait for them to arrive at the hospital, saving a lot of time in scenarios where every second can make a difference. Health-5G also addresses the integration of medical devices and the collection of the patient's medical data in a scalable way through optical character recognition. A case study is discussed where Health-5G is used to attend a patient in the street suffering from syncope due to third-degree atrioventricular block. Latency and performance tests over a 5G network are also discussed. To the best of our knowledge, there is no comprehensive solution in the literature that provides all the capabilities offered by Health-5G in terms of functionality and advanced interaction mechanisms within the context of remote, immersive support in emergency situations.

show abstract

“…AR confers the advantages of interactivity and immersivity, enabling users to closely interact with the information needed by superimposing the elements in the physical reality [Ferrario et al, 2020, Teng et al, 2018. That is, choosing the reasonable information to be visualized through AR interfaces is critical for both users and the on-going processes.…”

Section: Related Workmentioning

confidence: 99%

“…Augmented reality (AR), as a variation of virtual reality (VR), in which virtual objects are superimposed in the real environment [Azuma, 1997], is one of the leading techniques being proved and applied in numerous fields, like medicine and education [Billinghurst et al, 2015]. However, due to some specific deficiencies such as hardware limitations, computation power, and low knowledge convertibility, AR has still not become a prevalent tool in most industrial application domains [de Souza Cardoso et al, 2020, Teng et al, 2018. Nonetheless, AR is capable of providing an interactive interface of displayed digital content, as well as offering ways to interact with information in a multimodal and collaborative manner [Ferrario et al, 2020].…”

Section: Introductionmentioning

confidence: 99%

A Novel Augmented Reality System to Support Volumetric Visualization in Industrial Process Tomography

Zhang

Yadav

Omrani

et al. 2021

Proceedings of the 15th International Conference on Interfaces and Human Computer Interaction 2021 and 14th International Confe

View full text Add to dashboard Cite

Augmented Reality (AR), as a variation of Virtual Reality (VR), has been proved useful for decades. However, it is not widely utilized in most industries. To fill the gap between this technique and industrial usage, we propose a novel AR system in the context of industrial process tomography (IPT). As the pioneering AR deployment in IPT, this system offers a new solution to underpin the onsite data analysis regarding volumetric visualization. In our work, an endeavor to provide intelligent control for an industrial drying system is pursued by using microwave tomography (MWT), a breed of IPT, as an imaging modality. Here, the AR system is integrated with the MWT for post processing of its volumetric images, containing critical information of the industrial process. The core part of the AR system is implemented by an interactive mobile app that is supported on iOS/Android platforms. The overall system is generalized by four distinctive findings: interactivity, mobility, information richness, and mutual collaboration. Our proposed system opens the horizon of leveraging AR in IPT to benefit domain-related users regarding onsite data analysis and visualization.

show abstract

Interactive Augmented Live Virtual Reality Streaming

Cited by 11 publications

References 8 publications

VR-Based Immersive Service Management in B5G Mobile Systems: A UAV Command and Control Use Case

VR-Based Immersive Service Management in B5G Mobile Systems: A UAV Command and Control Use Case

Health-5G: A Mixed Reality-Based System for Remote Medical Assistance in Emergency Situations

A Novel Augmented Reality System to Support Volumetric Visualization in Industrial Process Tomography

Contact Info

Product

Resources

About