Patch-Based Colour Transfer with Optimal Transport

Abstract

This paper proposes a new colour transfer method with Optimal transport to transfer the colour of a source image to match the colour of a target image of the same scene. We propose to formulate the problem in higher dimensional spaces (than colour spaces) by encoding overlapping neighborhoods of pixels containing colour information as well as spatial information. Since several recoloured candidates are now generated for each pixel in the source image, we define an original procedure to efficiently merge these candidates which allows denoising and artifact removal as well as colour transfer. Experiments show quantitative and qualitative improvements over previous colour transfer methods. Our method can be applied to different contexts of colour transfer such as transferring colour between different camera models, camera settings, illumination conditions and colour retouch styles for photographs.Index Terms-optimal transport, colour transfer, image enhancement, JPEG compression blocks.

Sliced $\mathcal{L}_{2}$ Distance for Colour Grading

“…9. A close up look at some of the results generated using the IDT [5], PMLS [18], GPS/LCP and FGPS/LCP [17], B-PMLS [16], L2 [8], PCT_OT [9], OT_NW [10] , INWDT [11] and our algorithms using correspondences (SL2D corr c and SL2D corr cp ) and without using correspondences (SL2D c and SL2D cp ) -best viewed in colour and zoomed in.…”

“…Grogan & Dahyot L 2 framework [8] relies on a parametric formulation with Thin Plate Splines (TPS) of the mapping function φ : R 3 → R 3 that pushes source image colours toward the target image ones. Our recent research efforts have extended IDT-SWD projection based algorithms using image patches as samples in higher dimensional spaces than the RGB triplets in R 3 [9], taking advantage of correspondences between source and target patches [10] and replacing OT 1D solution by the Nadaraya-Watson estimator [11]. This paper proposes to extend this projective based strategy using the more robust L 2 distance [12], [13] to infer mapping functions suitable with and without correspondences [8], [14].…”

Patch based Colour Transfer using SIFT Flow

“…The spatial information for the target image is calculated using SIFT flow method which estimates dense spatial correspondences by robustly aligning complex scene pairs containing significant spatial differences [4], while in PCT_OT [3] the original pixel positions in the grid coordinate of the image are used. Using correspondences will allow colour transfer between images that contain moving objects and overcome the limitations in PCT_OT.…”

“…Similarly to PCT_OT [3] we encode overlapping neighborhoods of pixels to preserve local topology information. Starting from the origin of the coordinate system of the images (upper left corner), we use a sliding window operation of window size k × k to extract overlapping patches.…”

“…When computing the transfer function, considering colour information only does not take into account the fact that coherent colours should be transferred to neighboring pixels, which can create results with blocky artifacts emphasizing JPEG compression blocks, or increase noise. To tackle this problem, Alghamdi et al [3] proposed the Patch based Colour Transfer (PCT_OT) approach that encodes overlapping neighborhoods of pixels, taking into account both their colour and pixel positions. PCT_OT algorithm shows improvement over the state of the art methods but also shows limitations by creating shadow artifacts when there are large changes between target and source images.…”