Color Correction and Depth-Based Hierarchical Hole Filling in Free Viewpoint Generation

Qiao, Yiguo; Jiao, Licheng; Yang, Shuyuan; Hou, Biao; Feng, Jie

doi:10.1109/tbc.2019.2901391

Cited by 9 publications

(3 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiview color correction can be also performed by reprojecting points between different views [47] and is usually performed as a part of the virtual view synthesis process [7], [12], [67], [68], [69]. Such an approach allows to properly define corresponding areas even for systems with a very sparse arrangement of cameras.…”

Section: Related Workmentioning

confidence: 99%

Color Correction for Immersive Video Applications

et al. 2021

View full text Add to dashboard Cite

In this paper, the color correction method developed for immersive video systems is presented. The proposed method significantly increases the consistency of color characteristics of multiview sequences, understood both as the temporal and the inter-view consistency, what highly improves the subjective quality of the synthesized virtual views presented to the final user of the immersive video system. Moreover, the proposal allows to significantly increase the quality of the depth maps calculated for natural sequences, e.g., for views with colors inconsistent due to different lighting conditions. It enables more efficient compression of natural multiview video, as the newest encoding standards highly depend on the quality of depth maps. In order to evaluate the performance of the proposal, three experiments were conducted. In the first one, the proposal was compared to state of the art in the typical immersive video applicationcolor correction of a natural multiview sequence. In the second experiment, the performance of the proposal was tested on the Middlebury stereo dataset. In both experiments, the quality of synthesized virtual views was assessed subjectively by a group of 70 naïve viewers. The third experiment assessed the influence of color correction on the quality of estimated depth maps. All the experiments showed that the proposal significantly increases the color consistency of the multiview content. Due to the high usefulness and robustness, the proposed color correction method became the MPEG reference software for color correction. The implementation of the method is available for other researchers on the public repository.INDEX TERMS color correction, color refinement, immersive video, 3DoF+, inter-view consistency, virtual navigation.

show abstract

Section: Related Workmentioning

confidence: 99%

Color Correction for Immersive Video Applications

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Unfortunately, limited by equipment and cost, capturing all views of FVV via camera is unrealistic and needs the existence of virtual synthesized viewpoints to enhance the scene switching continuity. Multiview video plus depth is one of the mainstream representations of 3D scenes, which generate virtual synthesized images through depth-image-based-rendering (DIBR) techniques [ 2 ]. At this stage, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions which seriously deteriorate the users’ visual perception.…”

Section: Introductionmentioning

confidence: 99%

No-Reference Quality Assessment for 3D Synthesized Images Based on Visual-Entropy-Guided Multi-Layer Features Analysis

Jin

Peng

Zou

et al. 2021

Entropy

View full text Add to dashboard Cite

Multiview video plus depth is one of the mainstream representations of 3D scenes in emerging free viewpoint video, which generates virtual 3D synthesized images through a depth-image-based-rendering (DIBR) technique. However, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions that seriously deteriorate the users’ visual perception. An effective 3D synthesized image quality assessment (IQA) metric can simulate human visual perception and determine the application feasibility of the synthesized content. In this paper, a no-reference IQA metric based on visual-entropy-guided multi-layer features analysis for 3D synthesized images is proposed. According to the energy entropy, the geometric distortions are divided into two visual attention layers, namely, bottom-up layer and top-down layer. The feature of salient distortion is measured by regional proportion plus transition threshold on a bottom-up layer. In parallel, the key distribution regions of insignificant geometric distortion are extracted by a relative total variation model, and the features of these distortions are measured by the interaction of decentralized attention and concentrated attention on top-down layers. By integrating the features of both bottom-up and top-down layers, a more visually perceptive quality evaluation model is built. Experimental results show that the proposed method is superior to the state-of-the-art in assessing the quality of 3D synthesized images.

show abstract

“…However, the FG-BG classification could not distinguish precisely; it could not remove ghosts on the edges between the FG and BG. Many other approaches fill all holes in a single step using all valid image contents [28,29].…”

Section: Introductionmentioning

confidence: 99%

Fast Hole Filling for View Synthesis in Free Viewpoint Video

Huang

ShaoYu

2020

Electronics

View full text Add to dashboard Cite

The recent emergence of three-dimensional (3D) movies and 3D television (TV) indicates an increasing interest in 3D content. Stereoscopic displays have enabled visual experiences to be enhanced, allowing the world to be viewed in 3D. Virtual view synthesis is the key technology to present 3D content, and depth image-based rendering (DIBR) is a classic virtual view synthesis method. With a texture image and its corresponding depth map, a virtual view can be generated using the DIBR technique. The depth and camera parameters are used to project the entire pixel in the image to the 3D world coordinate system. The results in the world coordinates are then reprojected into the virtual view, based on 3D warping. However, these projections will result in cracks (holes). Hence, we herein propose a new method of DIBR for free viewpoint videos to solve the hole problem due to these projection processes. First, the depth map is preprocessed to reduce the number of holes, which does not produce large-scale geometric distortions; subsequently, improved 3D warping projection is performed collectively to create the virtual view. A median filter is used to filter the hole regions in the virtual view, followed by 3D inverse warping blending to remove the holes. Next, brightness adjustment and adaptive image blending are performed. Finally, the synthesized virtual view is obtained using the inpainting method. Experimental results verify that our proposed method can produce a pleasant visibility of the synthetized virtual view, maintain a high peak signal-to-noise ratio (PSNR) value, and efficiently decrease execution time compared with state-of-the-art methods.

show abstract

Color Correction and Depth-Based Hierarchical Hole Filling in Free Viewpoint Generation

Cited by 9 publications

References 40 publications

Color Correction for Immersive Video Applications

Color Correction for Immersive Video Applications

No-Reference Quality Assessment for 3D Synthesized Images Based on Visual-Entropy-Guided Multi-Layer Features Analysis

Fast Hole Filling for View Synthesis in Free Viewpoint Video

Contact Info

Product

Resources

About