Multitask bilateral learning for real‐time image enhancement

Abstract

Nowadays, deep neural networks (DNNs) for image processing are becoming more complex; thus, reducing computational cost is increasingly important. This study highlights the construction of a DNN for real‐time image processing, training various image processing operators efficiently through multitask learning. For real‐time image processing, the proposed algorithm takes a joint upsampling approach through bilateral guided upsampling. For multitask learning, the overall network is based on an encoder‐decoder architecture, which consists of encoding, processing, and decoding components, in which the encoding and decoding components are shared by all the image processing operators. In the processing component, a semantic guidance map, which contains processing information for each image processing operator, is estimated using simple linear shifts of the shared deep features. Through these components, the proposed algorithm requires an increase of only 5% in the number of parameters to add another image processing operator and achieves faster and higher performance than that of deep‐learning‐based joint upsampling methods in local image processing as well as global image processing.

DeepLPF: Deep Local Parametric Filters for Image Enhancement

“…Chen et al [4] propose a low-light enhancement model that operates directly on raw sensor data and propose a fully-convolutional approach to learn short-exposure, longexposure mappings using their low-light dataset. Recent work makes use of multitask learning [16] for real-time image processing with various image operators. Bilateral guided joint upsampling enables an encoder/decoder architecture for local as well as global image processing.…”

Learning algorithm with noisy labels for video frame classification

“…The use of deep learning frameworks has improved the performance of image classification, 7 optical flow estimation, 8,9 and image enhancement. 10,11 To train deep neural networks for video frame classification, each video frame needs to be labeled, which requires a considerable amount of time and effort. Alternatively, we can assign a label of a video (i.e., genre of movie and tag of TV program) identically to all frames.…”

“…Among them, video frame classification, which performs classification at the frame level, has increased in importance owing to its use in various applications such as 4D effect video classification 6 and genre classification. The use of deep learning frameworks has improved the performance of image classification, 7 optical flow estimation, 8,9 and image enhancement 10,11 . To train deep neural networks for video frame classification, each video frame needs to be labeled, which requires a considerable amount of time and effort.…”