An Improved Geometric Approach for Palette‐based Image Decomposition and Recoloring

Abstract

Palette‐based image decomposition has attracted increasing attention in recent years. A specific class of approaches have been proposed basing on the RGB‐space geometry, which manage to construct convex hulls whose vertices act as palette colors. However, such palettes do not guarantee to have the representative colors which actually appear in the image, thus making it less intuitive and less predictable when editing palette colors to perform recoloring. Hence, we proposed an improved geometric approach to address this issue. We use a polyhedron, but not necessarily a convex hull, in the RGB space to represent the color palette. We then formulate the task of palette extraction as an optimization problem which could be solved in a few seconds. Our palette has a higher degree of representativeness and maintains a relatively similar level of accuracy compared with previous methods. For layer decomposition, we compute layer opacities via simple mean value coordinates, which could achieve instant feedbacks without precomputations. We have demonstrated our method for image recoloring on a variety of examples. In comparison with state‐of‐the‐art works, our approach is generally more intuitive and efficient with fewer artifacts.

PosterChild: Blend‐Aware Artistic Posterization

“…Region colors are represented as linear blends between (at most two) colors in a compact, representative palette. Recoloring is easily accomplished by adjusting the palette colors themselves and updating the apparent region colors by remixing with the same blending weights, as in palette‐based image recoloring approaches [CFL*15b, TLG16, WLX19].…”

“…They proposed to simplify the convex hull until a user‐selected palette size p is reached. Wang et al [WLX19] observed that convex‐hull‐based palettes are sensitive to outliers. To reduce outlier sensitivity, they formulated an optimization problem to refine the simplified convex hull palette to make it more compact.…”

“…Our algorithm to extract palette is built on Tan et al's [TLG16] convex hull simplification and the motivation from Wang et al [WLX19]. Rather than optimizing the output of Tan et al's [TLG16] method, we remove outliers from the input.…”

“…However, K-means is notably slow and does not scale linearly with image size. Although real-time palette extraction is still an open problem, one possible faster solution is to replace our K-means relaxation with the iterative algorithm [WLX19]. Another possible solution to obtain a more compact palette would be to find an affine transformation for the palette that moves its vertices inwards based on color sparsity.…”

“…Tan et al [TLG16,TEG18] proposed to simplify the convex hull of all color samples in RGB‐space to obtain a polyhedron whose vertices better represent the image's color gamut. Wang et al [WLX19] introduced an optimization procedure to improve the compactness and relevance of the polyhedron's vertices. We, too, adopt simplified convex hull palettes, but propose to improve the vertex quality by preprocessing the input.…”