Plane-based multi-view inpainting for image-based rendering in large scenes

Philip, Julien; Drettakis, George

doi:10.1145/3190834.3190846

Cited by 19 publications

(19 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, our approach allows automatic 6-DoF capture for dynamic scenes and videos, being agnostic to the capture hardware, including capture setups with a very narrow baseline. Another interesting approach is that of Philip and Drettakis [54], in which multi-view inpainting is performed on an intermediate representation formed by locally planar spaces shared between the input images; the work focuses on inpainting large unknown regions in large datasets with wider baselines than ours.…”

Section: Related Workmentioning

confidence: 99%

Motion parallax for 360° RGBD video

Serrano

Kim

Chen

et al. 2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1. We add parallax for 360 • videos, for viewing in virtual reality head-mounted displays (HMDs). This translates into a more compelling viewing experience, as our user studies confirm. Left: Captured point of view as shown in the HMD (top), and a novel view as the user moves their head (bottom); this novel view is generated with our method and was not captured by the camera. Right, top row: Straightforward approaches based on image-based rendering do not work well due to suboptimal quality of the depth information. Original view (left) captured with the GoPro Odyssey, and a close-up of novel views generated with three different methods (right): (A) naive reprojection of RGB information, (B) naive handling of disocclusions, and (C) our method, relying on a robust layered representation. Right, bottom row: We also propose a depth map improvement step to correct for errors that have a high impact on reprojection. Original view (left) from a YouTube video (https://youtu.be/iWyvlkWYXhY), and close-ups showing depth maps and a displaced view computed with them, for the original estimated depth map (top row), and for our improved depth map (bottom row).

show abstract

Section: Related Workmentioning

confidence: 99%

Motion parallax for 360° RGBD video

Serrano

Kim

Chen

et al. 2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…Other approaches such as video inpainting 19 for recovering consistent color using dense sequential data, however, are not applicable on sparse data or wide-baseline data used for IBR. 1,9,3…”

Section: Multiview Inpaintingmentioning

confidence: 99%

“…The rendering of real indoor spaces is typically achieved via an image-based rendering (IBR) method [1][2][3] that allows a free-viewpoint exploration in a virtual world. Recent IBR applications 1 have reduced geometric complexity for the real-time rendering of large-scale indoor spaces.…”

Section: Introductionmentioning

confidence: 99%

Generative multiview inpainting for object removal in large indoor spaces

Kim

Hyeon

Doh

2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

As interest in image-based rendering increases, the need for multiview inpainting is emerging. Despite of rapid progresses in single-image inpainting based on deep learning approaches, they have no constraint in obtaining color consistency over multiple inpainted images. We target object removal in large-scale indoor spaces and propose a novel pipeline of multiview inpainting to achieve color consistency and boundary consistency in multiple images. The first step of the pipeline is to create color prior information on masks by coloring point clouds from multiple images and projecting the colored point clouds onto the image planes. Next, a generative inpainting network accepts a masked image, a color prior image, imperfect guideline, and two different masks as inputs and yields the refined guideline and inpainted image as outputs. The color prior and guideline input ensure color and boundary consistencies across multiple images. We validate our pipeline on real indoor data sets quantitatively using consistency distance and similarity distance, metrics we defined for comparing results of multiview inpainting and qualitatively.

show abstract

“…Similarly, Thonat et al [10] enable free-viewpoint image based rendering with reprojected information from neighboring views. Also a refined method is proposed in the following work [11] that performs inpainting on intermediate, local planes in order to preserve perspective as well as to ensure multi-view coherence. In contrast, we use local homography for achieving pixel-wise correspondences, with which the information loss caused by SfM could be effectively avoided.…”

Section: Multi-view Based Inpaintingmentioning

confidence: 99%

“…As for the video inpainting method, we have to down-sample the images and use the neighboring 100 frames of the target as input because of its extremely high time cost. We also present comparisons among our methods and those pro- posed in [10,11], as shown in Fig. 6.…”

Section: Comparisonsmentioning

confidence: 99%

Multi-view Inpainting for RGB-D Sequence

Li¹,

Ricardez

Takamatsu

et al. 2018

2018 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

In this work we propose a novel approach to remove undesired objects from RGB-D sequences captured with freely moving cameras, which enables static 3D reconstruction. Our method jointly uses existing information from multiple frames as well as generates new one via inpainting techniques. We use balanced rules to select source frames; local homography based image warping method for alignment and Markov random field (MRF) based approach for combining existing information. For the left holes, we employ exemplar based multi-view inpainting method to deal with the color image and coherently use it as guidance to complete the depth correspondence. Experiments show that our approach is qualified for removing the undesired objects and inpainting the holes.

show abstract

Plane-based multi-view inpainting for image-based rendering in large scenes

Cited by 19 publications

References 43 publications

Motion parallax for 360° RGBD video

Motion parallax for 360° RGBD video

Generative multiview inpainting for object removal in large indoor spaces

Multi-view Inpainting for RGB-D Sequence

Contact Info

Product

Resources

About