Fast Hole Filling for View Synthesis in Free Viewpoint Video

Abstract

The recent emergence of three-dimensional (3D) movies and 3D television (TV) indicates an increasing interest in 3D content. Stereoscopic displays have enabled visual experiences to be enhanced, allowing the world to be viewed in 3D. Virtual view synthesis is the key technology to present 3D content, and depth image-based rendering (DIBR) is a classic virtual view synthesis method. With a texture image and its corresponding depth map, a virtual view can be generated using the DIBR technique. The depth and camera parameters are used to project the entire pixel in the image to the 3D world coordinate system. The results in the world coordinates are then reprojected into the virtual view, based on 3D warping. However, these projections will result in cracks (holes). Hence, we herein propose a new method of DIBR for free viewpoint videos to solve the hole problem due to these projection processes. First, the depth map is preprocessed to reduce the number of holes, which does not produce large-scale geometric distortions; subsequently, improved 3D warping projection is performed collectively to create the virtual view. A median filter is used to filter the hole regions in the virtual view, followed by 3D inverse warping blending to remove the holes. Next, brightness adjustment and adaptive image blending are performed. Finally, the synthesized virtual view is obtained using the inpainting method. Experimental results verify that our proposed method can produce a pleasant visibility of the synthetized virtual view, maintain a high peak signal-to-noise ratio (PSNR) value, and efficiently decrease execution time compared with state-of-the-art methods.

A dichotomy color quantization algorithm for the HSI color space

“…Therefore, to verify the performance of each algorithm, we use a uniform quantization algorithm to compare the dichotomy algorithm using the PSNR 27 . The PSNR calculation method is as follows: …”

View Synthesis Tool for VR Immersive Video

“…One solution is to add more input images in the hope to obtain this missing information [15,17]. Another approach is to inpaint the empty pixels [18,19]. In RVS, it is possible to choose any number of viewpoints and a basic inpainting fills the remaining disocclusions.…”

Quality Assessment of View Synthesis Based on Visual Saliency and Texture Naturalness

“…Through these, 3D applications support the ability for people to see the same scene from different perspectives, leading to information redundancy and costly storage space. Hence, researchers often only transmit and save two texture images and a depth map, while the others are synthesized by utilizing the DIBR techniques at the receiving terminal [4]. The complete view synthesis includes the collection, processing, and transmission of texture images and depth maps, as well as DIBR view synthesis [5].…”