PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Abstract

This paper tackles a new problem setting: reinforcement learning with pixel-wise rewards (pixelRL) for image processing. After the introduction of the deep Q-network, deep RL has been achieving great success. However, the applications of deep reinforcement learning (RL) for image processing are still limited. Therefore, we extend deep RL to pixelRL for various image processing applications. In pixelRL, each pixel has an agent, and the agent changes the pixel value by taking an action. We also propose an effective learning method for pixelRL that significantly improves the performance by considering not only the future states of the own pixel but also those of the neighbor pixels. The proposed method can be applied to some image processing tasks that require pixel-wise manipulations, where deep RL has never been applied. Besides, it is possible to visualize what kind of operation is employed for each pixel at each iteration, which would help us understand why and how such an operation is chosen. We also believe that our technology can enhance the explainability and interpretability of the deep neural networks. In addition, because the operations executed at each pixels are visualized, we can change or modify the operations if necessary.We apply the proposed method to a variety of image processing tasks: image denoising, image restoration, local color enhancement, and saliency-driven image editing. Our experimental results demonstrate that the proposed method achieves comparable or better performance, compared with the state-ofthe-art methods based on supervised learning. The source code is available on https://github.com/rfuruta/pixelRL.

DeblurRL: Image Deblurring with Deep Reinforcement Learning

“…On the other hand, Ryosuke et al [3] proposed a deep reinforcement technique in which they have worked at a pixel level and have experimented with various image processing tasks such as image denoising, image restoration, color enhancement, and image editing; and have shown better performance with other state-of-the-art methods. They have used the deep reinforcement and pixel-based rewards technique which certain discrete sets of actions which therefore makes different from other deep learning techniques.…”

“…where (i x , i y ) are the elements of matrices A(a τ , s τ ) and π(a τ |s τ ) respectively. J is ones-vector where every element is one, and denotes element-wise multiplication [3].…”

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

“…Deep Q Networks attracted the attention of many researchers for solving image processing tasks but their usage was limited to simple applications. Introduction of pixelwise rewards in Reinforcement learning (PixelRL) makes it a better choice to solve tasks like image restoration, color enhancement and image denoising [336]. In PixelRL approach, reinforcement learning (RL) is combined with state-of-theart image processing techniques like Convolutional Neural Networks (CNN) to solve real-time complex computer vision problems.…”

“…In PixelRL approach, reinforcement learning (RL) is combined with state-of-theart image processing techniques like Convolutional Neural Networks (CNN) to solve real-time complex computer vision problems. Scheduling of important tasks or finding a shortest path between two points in images are few more examples where CNN extracts the features from images and RL learns the optimized way to proceed and perform scheduled task [337], [336] and [338].…”

Self-Play Reinforcement Learning for Fast Image Retargeting

“…Yu et al [32] proposed an image restoration method by selecting a toolchain from a toolbox. Furuta et al [5,6] proposed a fully convolutional network that allowed agents to perform pixel-wise manipulations for image denoising, image restoration, and color enhancement. Ganin et al [8] used an adversarially trained agent for synthesizing simple images of letters or digits using a non-differentiable renderer.…”