VRComm

Gunkel, Simon; Hindriks, Rick; Assal, Karim M. El; Stokking, Hans; Dijkstra-Soudarissanane, Sylvie; Haar, Frank ter; Niamut, Omar

doi:10.1145/3458305.3459595

Cited by 30 publications

(8 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The concept of our approach is based on converting any depth information into a 2D RGB image format (see Section 3.2.4) that can be transmitted via existing 2D encoding and distribution technology. This concept was previously reported in (Gunkel et al, 2018;Gunkel et al, 2021). Compared to newer volumetric streaming formats (such as V-PCC (Schwarz et al, 2019) and G-PCC (Mekuria et al, 2017)) this allows the use of reliable and established real-time streaming components (including full hardware accelerating), allowing high quality stream decoding even on low-powered end devices (like the Microsoft HoloLens 2 12 and Oculus Quest 2 13 ).…”

Section: Transmissionmentioning

confidence: 95%

“…The ease of deployment is still hindered by complex calibration and preprocessing steps. There are further solutions based on photorealistic social VR communication (Gunkel et al, 2021;Langa et al, 2021;Reimat et al, 2022). For example, (Prins et al, 2018), introduces a web-based framework for communicative and Social VR experiences.…”

Section: Introduction and Methodologymentioning

confidence: 99%

“…In order to transmit depth over existing video/image compression formats, the 16-bit depth data is converted into an 8-bit RGB color format, with a window of 1,530 values. Currently we offer two flavors of this conversion GrayAVG, presented in (Gunkel et al, 2021), and Intel HSV 8 , both implemented in Numba 2 to offer maximum hardware acceleration with Python.…”

Section: Depth Conversionmentioning

confidence: 99%

“…We use the GrayAVG depth conversion algorithm as presented in (Gunkel et al, 2021). GrayAVG maps depth values with a range of 0-1.5 m (i.e., 1,530 values) into gray-color values on the entire RGB color space.…”

Section: Grayavgmentioning

confidence: 99%

See 3 more Smart Citations

From 2D to 3D video conferencing: modular RGB-D capture and reconstruction for interactive natural user representations in immersive extended reality (XR) communication

Gunkel

Dijkstra-Soudarissanane

Stokking

et al. 2023

Front. Signal Process.

View full text Add to dashboard Cite

With recent advancements in Virtual Reality (VR) and Augmented Reality (AR) hardware, many new immersive Extended Reality (XR) applications and services arose. One challenge that remains is to solve the social isolation often felt in these extended reality experiences and to enable a natural multi-user communication with high Social Presence. While a multitude of solutions exist to address this issue with computer-generated “artificial” avatars (based on pre-rendered 3D models), this form of user representation might not be sufficient for conveying a sense of co-presence for many use cases. In particular, for personal communication (for example, with family, doctor, or sales representatives) or for applications requiring photorealistic rendering. One alternative solution is to capture users (and objects) with the help of RGBD sensors to allow real-time photorealistic representations of users. In this paper, we present a complete and modular RGBD capture application and outline the different steps needed to utilize RGBD as means of photorealistic 3D user representations. We outline different capture modalities, as well as individual functional processing blocks, with its advantages and disadvantages. We evaluate our approach in two ways, a technical evaluation of the operation of the different modules and two small-scale user evaluations within integrated applications. The integrated applications present the use of the modular RGBD capture in both augmented reality and virtual reality communication application use cases, tested in realistic real-world settings. Our examples show that the proposed modular capture and reconstruction pipeline allows for easy evaluation and extension of each step of the processing pipeline. Furthermore, it allows parallel code execution, keeping performance overhead and delay low. Finally, our proposed methods show that an integration of 3D photorealistic user representations into existing video communication transmission systems is feasible and allows for new immersive extended reality applications.

show abstract

Section: Transmissionmentioning

confidence: 95%

Section: Introduction and Methodologymentioning

confidence: 99%

Section: Depth Conversionmentioning

confidence: 99%

Section: Grayavgmentioning

confidence: 99%

See 2 more Smart Citations

From 2D to 3D video conferencing: modular RGB-D capture and reconstruction for interactive natural user representations in immersive extended reality (XR) communication

Gunkel

Dijkstra-Soudarissanane

Stokking

et al. 2023

Front. Signal Process.

View full text Add to dashboard Cite

show abstract

“…https://www.comunicacionsocial.es Enlace de adquisición de la Monografía: https://doi.org/10.52495/c7.emcs.15.c45 en la nube para minimizar la carga computacional y ancho de banda para los clientes (ej.,Cernigliario et al., 2020;Gunkel et al, 2021). Tercero, las sesiones de RV se pueden retransmitir a través de plataformas 2D, como se hace en el ámbito del gaming a través de plataformas como Youtube o Twitch.…”

unclassified

Capítulo 7. Realidad virtual social y comunicaciones holográficas en 3D: oportunidades y retos pendientes en el sector de la educación

Montagud

Cernigliaro

Arevalillo‐Herráez

et al. 2023

Espejo de Monografías

View full text Add to dashboard Cite

En los últimos años se está explorando el uso y adopción de tecnologías innovadoras en el sector de la educación y aprendizaje. Asimismo, modelos de educación y docencia online están ganando protagonismo, ya sea debido a una implantación voluntaria o forzosa de programas de educación remota y/o híbrida. Este auge ha hecho palpable las limitaciones de los sistemas de comunicaciones audiovisuales 2D, como la videoconferencia, en varios sentidos: confort, calidad de interacción, conciencia situacional, (co-)presencia, etc. Este trabajo introduce un nuevo medio de comunicación, interacción y colaboración que promete ser un candidato idóneo para superar estas limitaciones haciendo uso de tecnologías inmersivas: la Realidad Virtual (RV) Social. En primer lugar, se aportará una revisión de evidencias científicas recientes que vislumbran y demuestran potenciales beneficios de este medio para diferentes casos de uso de interés, en comparación al uso de sistemas de videoconferencia 2D tradicionales. Asimismo, se destacarán los potenciales beneficios que puede aportar la disponibilidad de representaciones holográficas 3D de los usuarios en comparación al uso de avatares sintéticos (y por tanto, poco realistas) en entornos de RV Social. A continuación, se destacarán una serie de retos a nivel tecnológico y de experiencia de usuario a superar con tal de conseguir una implantación efectiva y masiva de este nuevo medio de comunicación. Palabras clave: realidad virtual social, comunicaciones holográficas, metaverso, holoportación, educación inmersiva, educación virtual.

show abstract

AHP-Based Assessment of Developing Online Virtual Reality Services with Progressive Web Apps

Wang

Yaqin

Hsu

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

VRComm

Abstract: Figure 1: 16 user streams in a Virtual Experience (left; anonymous; 3D Background Model "Olam Conference Room" by Gideon Abochie licensed under CC Attribution) and RGBD user transmission of RGB-part (middle) and depth-part (right)

Cited by 30 publications

References 53 publications

From 2D to 3D video conferencing: modular RGB-D capture and reconstruction for interactive natural user representations in immersive extended reality (XR) communication

From 2D to 3D video conferencing: modular RGB-D capture and reconstruction for interactive natural user representations in immersive extended reality (XR) communication

Capítulo 7. Realidad virtual social y comunicaciones holográficas en 3D: oportunidades y retos pendientes en el sector de la educación

AHP-Based Assessment of Developing Online Virtual Reality Services with Progressive Web Apps

Contact Info

Product

Resources

About