Novel View Synthesis with Multiple 360 Images for Large-Scale 6-DOF Virtual Reality System

Cho, Hochul; Kim, Jangyoon; Woo, Woontack

doi:10.1109/vr.2019.8798142

Cited by 9 publications

(6 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With 50 input images along a ring, they can create a dense reconstruction which allows them to perform view synthesis with depth and normal map estimations. Similar approaches have also been applied to sets of 360 • images [10,65] and 360 • video [26].…”

Section: • View Synthesismentioning

confidence: 99%

OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramas

Li¹,

Zhang²,

Häne³

et al. 2022

Preprint

View full text Add to dashboard Cite

Immersive maps such as Google Street View and Bing Streetside provide true-to-life views with a massive collection of panoramas. However, these panoramas are only available at sparse intervals along the path they are taken, resulting in visual discontinuities during navigation. Prior art in view synthesis is usually built upon a set of perspective images, a pair of stereoscopic images, or a monocular image, but barely examines wide-baseline panoramas, which are widely adopted in commercial platforms to optimize bandwidth and storage usage. In this paper, we leverage the unique characteristics of wide-baseline panoramas and present OmniSyn, a novel pipeline for 360 • view synthesis between wide-baseline panoramas. OmniSyn predicts omnidirectional depth maps using a spherical cost volume and a monocular skip connection, renders meshes in 360 • images, and synthesizes intermediate views with a fusion network. We demonstrate the effectiveness of OmniSyn via comprehensive experimental results including comparison with the state-of-the-art methods on CARLA and Matterport datasets, ablation studies, and generalization studies on street views. We envision our work may inspire future research for this unheeded real-world task and eventually produce a smoother experience for navigating immersive maps.

show abstract

Section: • View Synthesismentioning

confidence: 99%

OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramas

Li¹,

Zhang²,

Häne³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To manage the city efficiently, it is necessary to observe the city in detail from a micro perspective, and expression using 360-degree videos is one of the tools that can effectively show these details [26,27]. Cho et al [28] created a six-degrees-offreedom (6-DoF) virtual environment, using multiple 360 images. In this environment, a user can freely walk around by using novel view synthesis methods based on the estimated camera pose and point clouds via structure from motion (SfM) [29].…”

Section: Micro Perspective Of Udt Using 360 Techniquesmentioning

confidence: 99%

“…Normally, a user cannot be away from a captured camera path, which means that the user has a movement limitation. To overcome this limitation and provide the freedom of movement, we implemented a novel view synthesis method [28] that makes it possible to remove the camera-captured path by synthesizing a novel view image. Finally, we implemented it into Hololens 2 to maximize the walking experience in a virtual environment created by multiple 360 images by synchronizing HMD movements with the viewpoint.…”

Section: -Degree Video For Micro-perspective Visualizationmentioning

confidence: 99%

GeoACT: Augmented Control Tower using Virtual and Real Geospatial Data

Ssin¹,

Cho²,

Woo³

2021

IxD&A

Self Cite

View full text Add to dashboard Cite

Despite prior efforts to solve urban problems and build smart cities using large-scale urban information, researchers have yet to develop tools for city administrators to easily access a diverse mix of city information. To address this limitation, this article proposes an augmented reality visualization system, GeoACT, which can develop a digital twin to present city information from both macro and micro perspectives. GeoACT employs a 3D virtual city miniature with real and virtual IoT information to provide a macro perspective view, and a 360-degree-video-based visualization technique is used for a micro perspective view. We applied GeoACT to an urban space and demonstrated that it could effectively visualize city information through an augmented reality Head Mounted Display (HMD). Furthermore, GeoACT can be used in urban control centers.

show abstract

“…For instance, a documentary produced by Chris Milk, Clouds Over Sidra, allows people to experience the Syrian refugee crisis through a 360-degree VR film with either VR headsets, Google Cardboard, or their own computers. When users watch 360-degree videos with VR headsets, users engage in the movement by moving their heads, and they can look in any direction due to either three degrees of freedom (3DoF) or six Degrees of Freedom (6DoF) offered in 360-degree videos (Cho et al, 2019). 360-degree videos can be applied to various topics, such as environmental issues, nature and sustainability, crisis and disasters.…”

mentioning

confidence: 99%

The new ethical thinking in CGI immersive journalism

Lin

Hsu

2023

Convergence: The International Journal of Research into New Med

View full text Add to dashboard Cite

Virtual reality (VR) journalism has become one of the popular formats of journalism, where people experience presence, empathy, and immersion through VR technology in digital environments. Although the term ‘virtual’ may appear to be the opposite of ‘real,’ the boundary between virtuality and reality has become blurred due to the virtual embodiment being part of reality for participants. As VR and real virtuality converge, little is known about the ethical issues in producing computer-generated imagery (CGI) immersive journalism and the viewers’ responses to CGI VR news reporting. Our study applied a concurrent mixed-method approach, including in-depth interviews with eight CGI professionals from the U.K. and a quantitative survey in Taiwan, to understand the production issues, ethical concerns, and audience perceptions toward CGI immersive journalism. Results from the interviews suggested that digital journalists need to rethink their ethical codes, particularly the representation of truth while producing news stories using CGI images and VR. Findings from the post-test survey showed that both immersive presence and interactivity in VR news resulted in a greater formation of believing and motivations to take action regarding the topic presented in the VR news.

show abstract

Novel View Synthesis with Multiple 360 Images for Large-Scale 6-DOF Virtual Reality System

Cited by 9 publications

References 3 publications

OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramas

OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramas

GeoACT: Augmented Control Tower using Virtual and Real Geospatial Data

The new ethical thinking in CGI immersive journalism

Contact Info

Product

Resources

About