Grayscale Image Colorization Methods: Overview and Evaluation

Abstract

Colorization is a process of transforming grayscale images to color images in a visually acceptable way. The main goal is to convince a viewer in the authenticity of the result. Grayscale images requiring colorization are in most cases images with natural scenes. Over the past 20 years a wide range of colorization methods has been developedfrom algorithmically simple, yet time-and energy-consuming because of unavoidable human intervention to more complicated, but simultaneously more automated methods. Automatic conversion has become a challenging area that links machine learning and deep learning with art. This paper presents an overview and evaluation of the grayscale image colorization methods and techniques applied to natural images. The paper provides classification of the existing colorization methods, explains the principles which they are based on and emphasizes their advantages and disadvantages. Special focus is put on deep learning methods. Relevant methods are compared regarding image quality and processing time. Different metrics for colored image quality assessment are used in this paper. Measuring the perceived quality of a colored image is extremely challenging because of the complexity of the human visual system. Multiple metrics used for colorization method evaluation provide results by determining the difference between the predicted color value and the ground-truth, which in several cases is not in coherence with image plausibility. The results indicate that user-guided colorization neural networks are the most promising colorization category because of the successful merge of human intervention and neural network automation. INDEX TERMSAutomatic methods, black-and-white image, colorfulness, colorization, deep learning methods, example-based methods, grayscale image, image quality assessment, scribble-based methods, userguided methods.

Colorizing the Past: Deep Learning for the Automatic Colorization of Historical Aerial Images

“…Despite a few differences, the existing colorization methods are mainly categorized in the literature [ 9 , 13 ] as user-guided ( Section 2.1 ) and learning-based solutions ( Section 2.2 ), differing in the level of operator intervention.…”

“…Fully automatic deep-learning techniques have supplanted the more demanding traditional guided approaches, and are currently the most promising and explored methods for the image colorization task [ 13 ]. The following sections present an overview of both categories, reviewing more in-depth recent learning-based approaches and some implementations for the aerial-scale image case ( Section 2.3 ).…”

“…The advent and the application, in particular, of deep-learning techniques to the colorization problem is facilitating this image processing activity. Currently, numerous deep-learning models have been proposed for converting grayscale images into color [ 9 , 10 , 11 , 12 , 13 , 14 , 15 ], mainly differing in the learning strategy and neural network architecture.…”

AI Video Editing: a Survey

“…The instance model, image-level models [230] and a fusion module are trained in three steps. More reviews on AI colorization can be found in [203] [232]. Style transfer is extracting a texture from the source image domain and transfer it to the target image domain using a deep neural network.…”

AI Video Editing: a Survey

“…The instance model, image-level models [230] and a fusion module are trained in three steps. More reviews on AI colourization can be found in [203] [232]. Style transfer is extracting a texture from the source image domain and transferring it to the target image domain using a deep neural network.…”