Hole Filling With Multiple Reference Views in DIBR View Synthesis

Li, Shuai; Zhu, Ce; Sun, Ming–Ting

doi:10.1109/tmm.2018.2791810

Cited by 71 publications

(22 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…D M Motiur Rahaman et al [11] used the number of models in Gaussian mixture modelling (GMM) to separate background and foreground pixels, subsequently the missing pixels were recovered from the adaptive weighted average of the pixel intensities from the corresponding model(s) of the GMM and the warped image. Li S et al [12] located useful pixels in the complementary views to reduce the holes.…”

Section: Related Workmentioning

confidence: 99%

View synthesis based on spatio-temporal continuity

Yao

2019

J Image Video Proc.

View full text Add to dashboard Cite

Free viewpoint video is generated on the basis of a video plus depth (V+D) virtual point rendering framework. Because of the limited bandwidth of video transmission, depth-image-based rendering (DIBR) has become a common method. Most DIBR methods not only have holes and ghost artifacts but also have problems with time continuity, leading to frequent flickers. In this paper, we make full use of time domain information in video sequences and adjacent frames to extract the static background image of the whole scene. Furthermore, we propose a weighted-fusion hole-filling method based on static background to fill holes and maintain time continuity. Experimental results show that the proposed method can improve the quality of virtual view images and strengthen spatio-temporal continuity.

show abstract

Section: Related Workmentioning

confidence: 99%

View synthesis based on spatio-temporal continuity

Yao

2019

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…Actually, these spatial methods always focus on filling the narrow gaps or small holes in the texture image, while texture synthesis can fill the large-scale holes [12][13][14]. Since large holes are observed when areas that are occluded by foreground objects in the reference view become exposed in the synthesized view, view-blending approaches can be used to alleviate this problem, as two adjacent cameras can cover a relatively wider viewing angle [15][16][17].…”

Section: Related Workmentioning

confidence: 99%

“…Although [17] indicates that large holes in a target virtual view can be greatly reduced by using other more neighboring complementary views in addition to the two (commonly used) most neighboring primary views, we still employ only two reference views to render virtual views in our proposed framework. The occlusions that appear on one warped image will be filled by another reference viewpoint in the weighted blending process.…”

Section: Related Workmentioning

confidence: 99%

Reliability-Based View Synthesis for Free Viewpoint Video

Deng

Wang

2018

Applied Sciences

View full text Add to dashboard Cite

Featured Application: View synthesis technology is important for free viewpoint video (FVV) and multiview video coding (MVC). It is a practical approach to reduce storage and transmission bandwidth for multiview videos.Abstract: View synthesis is a crucial technique for free viewpoint video and multi-view video coding because of its capability to render an unlimited number of virtual viewpoints from adjacent captured texture images and corresponding depth maps. The accuracy of depth maps is very important to the rendering quality, since depth image-based rendering (DIBR) is the most widely used technology among synthesis algorithms. There are some issues due to the fact that stereo depth estimation is error-prone. In addition, filling occlusions is another challenge in producing desirable synthesized images. In this paper, we propose a reliability-based view synthesis framework. A depth refinement method is used to check the reliability of depth values and refine some of the unreliable pixels, and an adaptive background modeling algorithm is utilized to construct a background image aiming to fill the remaining empty regions after a proposed weighted blending process. Finally, the proposed approach is implemented and tested on test video sequences, and experimental results indicate objective and subjective improvements compared to previous view synthesis methods.

show abstract

“…Multiple views can cover a larger capture range, and the occluded regions in the primary view may be visible in the auxiliary view. Li et al [23] verify that the hole size can be further reduced by using two or more auxiliary views. In [24], an improved multiview synthesis method is proposed.…”

Section: Introductionmentioning

confidence: 99%

Virtual View Synthesis Based on Asymmetric Bidirectional DIBR for 3D Video and Free Viewpoint Video

Chen

Liang

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

Depth image-based rendering (DIBR) plays an important role in 3D video and free viewpoint video synthesis. However, artifacts might occur in the synthesized view due to viewpoint changes and stereo depth estimation errors. Holes are usually out-of-field regions and disocclusions, and filling them appropriately becomes a challenge. In this paper, a virtual view synthesis approach based on asymmetric bidirectional DIBR is proposed. A depth image preprocessing method is applied to detect and correct unreliable depth values around the foreground edges. For the primary view, all pixels are warped to the virtual view by the modified DIBR method. For the auxiliary view, only the selected regions are warped, which contain the contents that are not visible in the primary view. This approach reduces the computational cost and prevents irrelevant foreground pixels from being warped to the holes. During the merging process, a color correction approach is introduced to make the result appear more natural. In addition, a depth-guided inpainting method is proposed to handle the remaining holes in the merged image. Experimental results show that, compared with bidirectional DIBR, the proposed rendering method can reduce about 37% rendering time and achieve 97% hole reduction. In terms of visual quality and objective evaluation, our approach performs better than the previous methods.

show abstract

Hole Filling With Multiple Reference Views in DIBR View Synthesis

Cited by 71 publications

References 23 publications

View synthesis based on spatio-temporal continuity

View synthesis based on spatio-temporal continuity

Reliability-Based View Synthesis for Free Viewpoint Video

Virtual View Synthesis Based on Asymmetric Bidirectional DIBR for 3D Video and Free Viewpoint Video

Contact Info

Product

Resources

About