Enabling Five-Degree-of-Freedom From Low-Dimensional Light Fields With Extended Light Ray Acquisition Field-of-View

Abstract

Conventional captured-image-based virtual reality (VR) systems hardly support free movement of the viewpoint, greatly reducing the user's sense of reality. A light field (LF), which generates a novel view at a free viewpoint through a combination of light rays, is a suitable approach for increasing the degree of freedom (DoF) in captured-image-based VR. Previous LF-based VR systems have increased LF dimensions or used various acquisition layouts to obtain as many light rays as possible and thereby expand the renderable view range. However, these attempts are costly and time-consuming. This paper enables 5-DoF outcomes from a low-dimensional LF by increasing the field-of-view during light ray acquisition. The method starts with the simple idea that if a three-dimensional (3D) LF is constructed with 360 • images that can capture all directions, the renderable range can be widened in a cost-efficient manner. However, using light rays with a large incident angle in a 3D LF can cause visual artifacts. To cope with this problem, this paper initially defines and analyzes the visual artifacts. While these errors were simply ignored in the past, they become particularly noticeable when using light rays with large incident angles. Thus, a careful analysis of the errors and their impacts is required. Additionally, this paper proposes two schemes to deal with visual artifacts based on the error analysis: a horizontal LF and vertical depth (HLF-VD) view generation scheme that complements the structural limitation of 3D LFs and can address all types of errors, and a hybrid 3D LF connection scheme that effectively reduces connection errors by controlling the connection position of neighboring 3D LFs. The proposed system is implemented and verified in various real-world scenarios, and the experimental results show that the user's movement is naturally reflected at a level similar to that of high-dimensional LF, with minimal visual artifacts. INDEX TERMSFree viewpoint, Image-based rendering, Light field, View navigation, Virtual reality I. INTRODUCTIONA few years ago, virtual reality (VR) technology gained significant attention. However, problems such as insufficient technical maturity, lack of VR content, and motion sickness drastically reduced interest in VR. Nevertheless, with the onset of the COVID-19 pandemic, VR technology is experiencing a resurgence. The pandemic has changed the way people interact, and to avoid direct contact, telecommut-ing, video conferencing, and video education have become increasingly popular. VR technology can provide an allencompassing solution in the COVID-19 era as it allows people to work, meet, and play in a virtual space separated from the real world. However, the level of VR technology is still not as advanced as people expect, and there are numerous challenges that need to be addressed.Computer graphic-based virtual reality (CG-based VR)