Distort-and-Recover: Color Enhancement Using Deep Reinforcement Learning

Abstract

Learning-based color enhancement approaches typically learn to map from input images to retouched images. Most of existing methods require expensive pairs of input-retouched images or produce results in a noninterpretable way. In this paper, we present a deep reinforcement learning (DRL) based method for color enhancement to explicitly model the step-wise nature of human retouching process. We cast a color enhancement process as a Markov Decision Process where actions are defined as global color adjustment operations. Then we train our agent to learn the optimal global enhancement sequence of the actions. In addition, we present a 'distort-and-recover' training scheme which only requires high-quality reference images for training instead of input and retouched image pairs. Given high-quality reference images, we distort the images' color distribution and form distorted-reference image pairs for training. Through extensive experiments, we show that our method produces decent enhancement results and our DRL approach is more suitable for the 'distortand-recover' training scheme than previous supervised approaches. Supplementary material and code are available at https

Beyond the feedforward sweep: feedback computations in the visual cortex

“…Initial work has already demonstrated the benefits of combining reinforcement learning with RNNs to play Atari ® games 145 . Promising results have also been obtained for visual tracking, 146,147 face recognition, 148 action recognition, 149,150 video captioning, 151 color enhancement, 152 and object detection 153,154 …”

“…Initial work has already demonstrated the benefits of combining reinforcement learning with RNNs to play Atari R games. 145 Promising results have also been obtained for visual tracking, 146,147 face recognition, 148 action recognition, 149,150 video captioning, 151 color enhancement, 152 and object detection. 153,154 Another approach to learning structure in the visual world, which does not use explicit labeled examples or a teacher and provides direct rewards/punishment for specific actions, is based on the intuition that predicting what will happen next may be an important principle of computation in the brain.…”

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

“…Using Photoshop, all the images were enhanced by a professional photographer for three different stylistic local effects: Foreground Pop-Out, Local Xpro, and Watercolor. Inspired by the prior work for global color enhancement [4], we decided the action set as shown in Table VII. We simply chose the same action set as [4] except for the "do nothing" action.…”

“…Inspired by the prior work for global color enhancement [4], we decided the action set as shown in Table VII. We simply chose the same action set as [4] except for the "do nothing" action. The motivation for the choice has been discussed in the section 4.1 in their paper.…”

“…Similarly, for fast image processing, Chen et al [42] adopted an FCN to learn an approximate mapping from the input to the desired images. Unlike deep learning-based methods that learn the input for an output mapping, our color enhancement method is interpretable because our method enhances each pixel value iteratively with actions such as [4], [5].…”