Efficient affinity-based edit propagation using K-D tree

Abstract

Figure 1: Affinity-based edit propagation methods such as allow one to change the appearance of an image or video (e.g., the color of the bird here) using only a few strokes, yet consuming prohibitive amount of time and memory for large data (e.g., 48 minutes and 23GB for this video containing 61M pixels). Our approximation scheme drastically reduces the cost of edit propagation methods (to 8 seconds and 22MB in this example) by exploring adaptive clustering in the affinity space. Video courtesy of BBC Motion Gallery (UK). AbstractImage/video editing by strokes has become increasingly popular due to the ease of interaction. Propagating the user inputs to the rest of the image/video, however, is often time and memory consuming especially for large data. We propose here an efficient scheme that allows affinity-based edit propagation to be computed on data containing tens of millions of pixels at interactive rate (in matter of seconds). The key in our scheme is a novel means for approximately solving the optimization problem involved in edit propagation, using adaptive clustering in a high-dimensional, affinity space. Our approximation significantly reduces the cost of existing affinitybased propagation methods while maintaining visual fidelity, and enables interactive stroke-based editing even on high resolution images and long video sequences using commodity computers.

Efficient Edit Propagation Using Hierarchical Data Structure

“…As one concurrent work [22], Xu et al proposed a similar accelerated edit propagation method using adaptive kd-tree, while our method uses an adaptive quadtree for image and octree for video. There are two main differences between these two methods.…”

Efficient antialiased edit propagation for images and videos

“…The key in our method is a new representation called Antialias Map, in which we represent each antialiased edge pixel by a linear interpolation of neighboring pixels around the edge, and instead of considering the original edge pixels in solving edit propagation, we consider those neighboring pixels. We demonstrate that our work is effective in preserving antialiased edges for edit propagation and could be easily integrated with existing edit propagation methods such as [1, 2]. …”

“…4) ( Fig. 5 Resolution 120K 120K 120K 120K 150K 150K 240K 30M Frame Num -------400 K-d tree time 22ms 23ms 17ms 25ms 28ms 24ms 41ms 8s memory 8MB 8MB 8MB 8MB 8MB 8MB 8MB 22MB Improved time 40ms 42ms 32ms 45ms 45ms 47ms 79ms 13s k-d tree memory 9MB 9MB 9MB 9MB 9MB 9MB 9MB 24MB RBF time 16ms 17ms 13ms 20ms 21ms 19ms 26ms 4s memory 1MB 1MB 1MB 1MB 1MB 1MB 1MB 1MB Improved time 32ms 30ms 25ms 38ms 32ms 36ms 51ms 8s RBF memory 1MB 1MB 1MB 1MB 1MB 1MB 1MB 1MB Table 2: Performance comparison between the k-d tree method [1], our method combined with the k-d tree approach, RBF method [2] and our method combined with the RBF method. Both running time and memory cost are reported.…”

“…Figure 1 (b) gives the result generated 23 by a state-of-the-art edit propagation work [1], while it achieves 24 the goal in most parts of the image, however, as shown in the It's not surprising that edit propagation methods would pro-28 duce such aliasing artifacts. This is simply because edit propa- …”

“…All these results and performance are obtained using a con- In Figure 6, we give 2 image results generated by the k-d 411 tree approach [1] and by our method. In Figure 7, we give 2 (Fig.…”