Decomposing Images into Layers via RGB-Space Geometry

Abstract

In digital image editing software, layers organize images. However, layers are often not explicitly represented in the final image, and may never have existed for a scanned physical painting or a photograph. We propose a technique to decompose an image into layers. In our decomposition, each layer represents a single-color coat of paint applied with varying opacity. Our decomposition is based on the image’s RGB-space geometry. In RGB-space, the linear nature of the standard Porter-Duff [1984] “over” pixel compositing operation implies a geometric structure. The vertices of the convex hull of image pixels in RGB-space correspond to a palette of paint colors. These colors may be “hidden” and inaccessible to algorithms based on clustering visible colors. For our layer decomposition, users choose the palette size (degree of simplification to perform on the convex hull), as well as a layer order for the paint colors (vertices). We then solve a constrained optimization problem to find translucent, spatially coherent opacity for each layer, such that the composition of the layers reproduces the original image. We demonstrate the utility of the resulting decompositions for recoloring (global and local) and object insertion. Our layers can be interpreted as generalized barycentric coordinates; we compare to these and other recoloring approaches.

An Improved Geometric Approach for Palette‐based Image Decomposition and Recoloring

“…In Figure 6, we compare our improved palette extraction method (Section 4) and the original convex hull-based palette extraction method [TLG17] for image recoloring. For a fair comparison, both methods use the same layer decomposition scheme, i.e., MVCbased decomposition.…”

“…We use an iterative approach to solve this optimization problem. The polyhedron is initialized using the convex hull produced by Tan et al [TLG17]. In each iteration, we alternatively select a vertex to optimize for its best position.…”

“…The optimization converges quickly after a few iterations. Our extract palette has a higher degree of representativeness compared with [TLG17] and still maintains a high reconstruction accuracy. Details are given in Section 4.…”

“…To address this problem, we propose an improved geometric approach for palette-based image decomposition. We follow Tan et al [TLG17,TEG18] to use a polyhedron to represent the color palette. The motivation of our method is to increase the degree of representativeness for the palette, and still maintains a high level of reconstruction accuracy.…”

Succinct Palette and Color Model Generation and Manipulation Using Hierarchical Representation

“…Geometric and pigment-based approaches are noteworthy even though they are not directly related to our work, since those methods sought the palette for reproducing the image, while our focus is on finding the representative palette. Tan et al defined a palette as a set of colors that are the vertices of the simplified convex hull wrapping colors of an image [TLG16]. They also proposed an automatic technique to select the number of vertices [TEG18].…”