An enhanced depth map based rendering method with directional depth filter and image inpainting

Liu, Wei; Zhang, Dehua; Cui, Mingyue; Ding, Jianwei

doi:10.1007/s00371-015-1074-2

Cited by 7 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the novel three-level progressive reconstruction procedure scheme described above, in this section we further exploit two other scene constraints { ξ c 1 , ξ c 2 } that are especially effective for the stereoscopic synthesis in this paper. In fact, these two constraints are always to be regarded as important scene cues and have been effectively used in some traditional DIBR approaches [ 5 , 6 ]. While most of the current deep-learning based methods assume that these constraints can be implicitly learned by CNNs without any further supervision, and therefore do not provide any additional information to the model.…”

Section: The Proposed Approachmentioning

confidence: 99%

“…On the other hand, removing occlusion can be realized by pre-filtering the depth map to reduce the discontinuity of depth data in such a way that disocclusions decrease, and then post-processing the warped image to replace the missing area with some color information. This is a common pipeline which current DIBR systems almost always follow [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Stereoscopic view synthesis with progressive structure reconstruction and scene constraints

Liu

Qiu

et al. 2022

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Depth image-based rendering (DIBR) is an important technology in the process of 2D-to-3D conversion. It uses texture images and related depth maps to render virtual views. While there are still some challenging problems in the current DIBR systems, such as disocclusion occurrences. Inpainting methods based on deep learning have recently shown significant improvements and generated plausible images. However, most of these methods may not deal well with the disocclusion holes in the synthesized views, because on the one hand they only treat this issue as generative inpainting after 3D warping, rather than following the full DIBR processing procedures. While on the other hand the distributions of holes on the virtual views are always around the transition regions of foreground and background, which makes them more difficult to distinguish without special constraints. Motivated by these observations, this paper proposes a novel learning-based method for stereoscopic view synthesis, in which the disocclusion regions are restored by a progressive structure reconstruction strategy instead of direct texture inpainting. Additionally, some special cues in the synthesized scenes are further exploited as constraints for the network to alleviate hallucinated structure mixtures among different layers. Extensive empirical evaluations and comparisons validate the strengths of the proposed approach and demonstrate that the model is more suitable for stereoscopic synthesis in the 2D-to-3D conversion applications.

show abstract

Section: The Proposed Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Stereoscopic view synthesis with progressive structure reconstruction and scene constraints

Liu

Qiu

et al. 2022

PLoS ONE

Self Cite

View full text Add to dashboard Cite

show abstract

“…Basis on above, some researchers proposed the technology of processing depth maps as described in [15]- [18] to reduce the holes in the virtual viewpoint. Experimental results show that this added pre-processing step can reduce the initial errors of the depth map and the warped holes in the virtual image at some degree.…”

Section: Introductionmentioning

confidence: 99%

An Improved Depth Image Based Virtual View Synthesis Method for Interactive 3D Video

Zhu

Yan

2019

IEEE Access

View full text Add to dashboard Cite

Depth image-based rendering (DIBR) is one of the main techniques in interactive 3D video systems. But a critical problem of DIBR is that how to eliminate the holes appearing in virtual view images. In this paper, an improved depth image based virtual view synthesis algorithm is proposed. In depth map pre-processing, only local areas of the depth map with sudden changes are pre-processed by the asymmetric filter. After this, we combine the forward and inverse 3D-warping to synthesize the virtual view. Firstly, it warps the left image to the virtual view and then only reversely warps the region of the holes in the virtual view to the right view to get the lost information of holes. Compared with the traditional method which have to warp the two whole images, computation cost can be reduced by 30% efficiently. Finally, we employ an improved image inpainting based on depth information to fill hole regions. We have improved the method in two portions: 1) depth information is added to the priority computation and searching the best matching patch and 2) as the filling proceeds, the confidence term of the newly inpainted pixels is updated with the mean square error (MSE) of pixels in the target matching patch and the source matching patch. Experimental results show significant objective and subjective gains of the proposed method in comparison to the reference methods.INDEX TERMS DIBR, 3D warping, hole filling, depth map.

show abstract

Superpixel-based color–depth restoration and dynamic environment modeling for Kinect-assisted image-based rendering systems

et al. 2016

View full text Add to dashboard Cite

An enhanced depth map based rendering method with directional depth filter and image inpainting

Cited by 7 publications

References 15 publications

Stereoscopic view synthesis with progressive structure reconstruction and scene constraints

Stereoscopic view synthesis with progressive structure reconstruction and scene constraints

An Improved Depth Image Based Virtual View Synthesis Method for Interactive 3D Video

Superpixel-based color–depth restoration and dynamic environment modeling for Kinect-assisted image-based rendering systems

Contact Info

Product

Resources

About