Depth Map Driven Hole Filling Algorithm Exploiting Temporal Correlation Information

Yao, Chao; Tillo, Tammam; Xiao, Jimin; Bai, Huihui; Lin, Chunyu

doi:10.1109/tbc.2014.2321671

Cited by 62 publications

(30 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The spatial resolution of "Ballet," "Break Dancers," and "Book Arrival" is 1024 × 768, and that of "Car Park" and "Street" is 1920 × 1088. The proposed hole-filling algorithm was compared to the state-of-the-art spatio-temporal hole-filling algorithms, including those of Xi [17], Koppel [16], and Yao [18]. In addition, the proposed priority function used for the inpainting was compared to other priority functions, including those of Criminisi [13] and Ramirez [22].…”

Section: Resultsmentioning

confidence: 99%

“…This method was hampered by the geometric distortions around isolated disocclusions because BG information existing for short periods in the previous frames was not taken into account in the hole-filling. Yao et al [18] presented Gaussian mixture model (GMM) to estimate depthmap driven BG sprite, which was then used to fill the BG pixels of a disoccluded region. However, the GMM did not correctly extract the BG region.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

Doan

Nguyen

Hong

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper presents a new hole-filling method that uses extrapolated spatio-temporal background information to obtain a synthesized free-view. A new background codebook for extracting reliable temporal background information is introduced. In addition, the paper addresses estimating spatial local background to distinguish background and foreground regions so that spatial background information can be extrapolated. Background holes are filled by combining spatial and temporal background information. Finally, exemplar-based inpainting is applied to fill in the remaining holes using a new priority function. The experimental results demonstrated that satisfactory synthesized views can be obtained using the proposed algorithm. key words: hole-filling, spatio-temporal background information, background codebook, spatial local background estimation, priority function

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

Doan

Nguyen

Hong

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…In [20,21], a background sprite was generated by the original texture and synthesized images from the temporal previous frames using disocclusion filling, but the temporal consistency of the synthesized images needs further investigation, as described in [22]. In [23], Yao proposed a disocclusion filling approach based on temporal correlation and depth information. Experimental results showed that this approach yields better subjective and objective performance beyond the above-mentioned spatial methods of filling disocclusions.…”

Section: Related Workmentioning

confidence: 99%

“…Besides, some disocclusion regions that are not included in a single reference view may easily be spotted in another virtual viewpoint, and reverse mapping [4] may be more reliable. In [23], Luo and Zhu et al proposed the use of a constructed background video with a modified Gaussian mixture model (GMM) to eliminate the holes in synthesized video. The foreground objects are detected and removed, then motion compensation and modified GMMs are applied to construct a stable background.…”

Section: Related Workmentioning

confidence: 99%

Reliability-Based View Synthesis for Free Viewpoint Video

Deng

Wang

2018

Applied Sciences

View full text Add to dashboard Cite

Featured Application: View synthesis technology is important for free viewpoint video (FVV) and multiview video coding (MVC). It is a practical approach to reduce storage and transmission bandwidth for multiview videos.Abstract: View synthesis is a crucial technique for free viewpoint video and multi-view video coding because of its capability to render an unlimited number of virtual viewpoints from adjacent captured texture images and corresponding depth maps. The accuracy of depth maps is very important to the rendering quality, since depth image-based rendering (DIBR) is the most widely used technology among synthesis algorithms. There are some issues due to the fact that stereo depth estimation is error-prone. In addition, filling occlusions is another challenge in producing desirable synthesized images. In this paper, we propose a reliability-based view synthesis framework. A depth refinement method is used to check the reliability of depth values and refine some of the unreliable pixels, and an adaptive background modeling algorithm is utilized to construct a background image aiming to fill the remaining empty regions after a proposed weighted blending process. Finally, the proposed approach is implemented and tested on test video sequences, and experimental results indicate objective and subjective improvements compared to previous view synthesis methods.

show abstract

“…Hierarchical hole-filling and depth adaptive hierarchical hole-filling approaches are proposed in [9], which can avoid geometric distortion in the rendered image through the use of a pyramid-like approach for estimating the hole pixels from the lower resolution estimation of the 3D wraped image. Holes in the scene are filled using the temporal correlation of texture and depth information from one view, while the regions along the static objects are filled using the inpainting technique in [10]. The algorithm in [10] is more suitable for 3D video systems with single-view-plus-depth format.…”

Section: Introductionmentioning

confidence: 99%

Hole-Filling for DIBR Based on Depth and Gradient Information

Wang

Zhao

Wang

et al. 2015

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Depth image-based rendering (DIBR) is a method for generating new virtual images from known viewpoints. However, holes often appear in the rendered virtual images due to occlusion and inaccurate depth information. In this paper, we present a novel hole-filling algorithm to improve the image quality of DIBR. In the proposed method, depth information is added to the priority calculation function when determining the order of hole-filling. Then, the gradient information is used as auxiliary information when searching for the optimal matching block. Experimental results show that the proposed algorithm achieves better objective quality and also improves the subjective quality of the rendered images.

show abstract

Depth Map Driven Hole Filling Algorithm Exploiting Temporal Correlation Information

Cited by 62 publications

References 21 publications

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

Reliability-Based View Synthesis for Free Viewpoint Video

Hole-Filling for DIBR Based on Depth and Gradient Information

Contact Info

Product

Resources

About