Image inpainting techniques have been widely investigated to remove undesired objects in an image. Conventionally, missing parts in an image are completed by optimizing the objective function using pattern similarity. However, unnatural textures are easily generated due to two factors: (1) available samples in the image are quite limited, and (2) pattern similarity is one of the required conditions but is not sufficient for reproducing natural textures. In this paper, we propose a new energy function based on the pattern similarity considering brightness changes of sample textures (for (1)) and introducing spatial locality as an additional constraint (for (2)). The effectiveness of the proposed method is successfully demonstrated by qualitative and quantitative evaluation. Furthermore, the evaluation methods used in much inpainting research are discussed.

Diminished reality aims to remove real objects from video images and fill in the missing regions with plausible background textures in real time. Most conventional methods based on image inpainting achieve diminished reality by assuming that the background around a target object is almost planar. This paper proposes a new diminished reality method that considers background geometries with less constraints than the conventional ones. In this study, we approximate the background geometry by combining local planes, and improve the quality of image inpainting by correcting the perspective distortion of texture and limiting the search area for finding similar textures as exemplars. The temporal coherence of texture is preserved using the geometries and camera pose estimated by visual-simultaneous localization and mapping (SLAM). The mask region that includes a target object is robustly set in each frame by projecting a 3D region, rather than tracking the object in 2D image space. The effectiveness of the proposed method is successfully demonstrated using several experimental environments.

This paper proposes a new diminished reality technique which removes AR markers from a user's view image. In order to achieve natural marker hiding, three factors should be considered; (1) naturalness of texture generated on a marker area. (2) geometric consistency between consecutive frames, (3) photometric consistency between a marker area and its surrounding. In this study, assuming that an area around a marker is locally planar, the marker area in the first frame image is inpainted using the rectified image to achieve high-quality inpainting. The unique inpainted texture is overlaid on the marker region in subsequent frames according to camera pose for temporal geometric consistency. Global and local luminance changes around the marker are reflected to the inpainted texture for photometric consistency.

This paper proposes a new method of diminished reality which removes AR markers from a user's view image in real time. To achieve natural marker hiding, assuming that an area around a marker is locally planar, the marker area in the first frame is inpainted using the rectified image to achieve high-quality inpainting. The unique inpainted texture is overlaid on the marker region in each frame according to camera motion for geometric consistency. Both global and local luminance changes around the marker are separately detected and reflected to the inpainted texture for photometric consistency.

Augmented reality (AR) marker hiding is a technique to visually remove AR markers in a real-time video stream. A conventional approach transforms a background image with a homography matrix calculated on the basis of a camera pose and overlays the transformed image on an AR marker region in a real-time frame, assuming that the AR marker is on a planar surface. However, this approach may cause discontinuities in textures around the boundary between the marker and its surrounding area when the planar surface assumption is not satisfied. This paper proposes a method for AR marker hiding without discontinuities around texture boundaries even under nonplanar background geometry without measuring it. For doing this, our method estimates the dense motion in the marker's background by analyzing the motion of sparse feature points around it, together with a smooth motion assumption, and deforms the background image according to it. Our experiments demonstrate the effectiveness of the proposed method in various environments with different background geometries and textures.