Efficient affinity-based edit propagation using K-D tree

Abstract

Figure 1: Affinity-based edit propagation methods such as allow one to change the appearance of an image or video (e.g., the color of the bird here) using only a few strokes, yet consuming prohibitive amount of time and memory for large data (e.g., 48 minutes and 23GB for this video containing 61M pixels). Our approximation scheme drastically reduces the cost of edit propagation methods (to 8 seconds and 22MB in this example) by exploring adaptive clustering in the affinity space. Video courtesy of BBC Motion Gallery (UK). AbstractImage/video editing by strokes has become increasingly popular due to the ease of interaction. Propagating the user inputs to the rest of the image/video, however, is often time and memory consuming especially for large data. We propose here an efficient scheme that allows affinity-based edit propagation to be computed on data containing tens of millions of pixels at interactive rate (in matter of seconds). The key in our scheme is a novel means for approximately solving the optimization problem involved in edit propagation, using adaptive clustering in a high-dimensional, affinity space. Our approximation significantly reduces the cost of existing affinitybased propagation methods while maintaining visual fidelity, and enables interactive stroke-based editing even on high resolution images and long video sequences using commodity computers.

A framework for interactive image color editing

“…(D + 1)N indices, where N is the number of pixels. Our method is also generalizable to video input, similar as proposed by Levin et al or Xu et al [12,38]. We relegate the implementation of video-processing to future work, but we do expect to retain the same performance.…”

“…Further work are speed-up methods, like Xu et al [38] who proposed an acceleration to the approach of [2] based on kd-tree-subdivision of the image, but they still utilize the dense solver. Li et al [14] formulate the problem as Radial-Basis-Function kernels interpolation.…”

“…In addition, we introduce an efficient sub-sampling strategy in order to achieve interactive performance. Xu et al [38] proposed a speed-up approach to the formulation of An and Pellacini [2] which exploits the fact that often pixels in the image can be approximated by a much smaller set of clusters. Driven by similar observations we sub-sample the image in order to greatly reduce the number of color points to be processed.…”

Light Field Image Processing: An Overview

“…5). This structure is the key difference with other methods targeting non-structured media, such as unstructured light fields [217], [218], video [209], [219], [220], [221] or image collections [222], [223], [224]. These are more complex structures, and might require the use of techniques such as depth estimation, optical flow or feature matching, to transfer edits between images.…”

Deep Colorization for Surveillance Images

“…For a given illumination, we try to infer the a and b colour channels of input images based on a convolutional neural networks. [11][12][13] In the rest part of this paper, CNN-based colorization scheme will be introduced in section 2. Experimental results are provided in section 3.…”