Deep binocular tone mapping

Abstract

Binocular tone mapping is studied in the previous works to generate a fusible pair of LDR images in order to convey more visual content than one single LDR image. However, the existing methods are all based on monocular tone mapping operators. It greatly restricts the preservation of local details and global contrast in a binocular LDR pair. In this paper, we proposed the first binocular tone mapping operator to more effectively distribute visual content to an LDR pair, leveraging the great representability and interpretability of deep convolutional neural network. Based on the existing binocular perception models, novel loss functions are also proposed to optimize the output pairs in terms of local details, global contrast, content distribution, and binocular fusibility. Our method is validated with a qualitative and quantitative evaluation, as well as a user study. Statistics show that our method outperforms the state-of-the-art binocular tone mapping frameworks in terms of both visual quality and time performance.

Deep Tone-Mapping Operator Using Image Quality Assessment Inspired Semi-Supervised Learning

“…Zhang et al [26] applied multi-scale 2-branch CNN similar to [25]. Their 9-layer large-scale branch with dilated convolution [55] is responsible for processing details, 5layer small-scale encoder branch is for global information, and 2-layer "tail" is for merging the output of 2 branches.…”

“…From the 3 rd column we know that some deep TMOs ( [25], [26], [27]) were influenced by traditional Durand02 TMO [18] in that they assigned separate CNN branches to handle different frequency components. Specifically, they use CNN with large receptive filed to handle global/low-frequency component, and CNN with small receptive filed to deal with local/high-frequency component.…”

“…Later work Marnerides et al [39] first applied multi-branch structure by assigning different kernel size in each branch to focus on global/local features. Later, decomposing and multi-branch had become more popular, as they were used by [31], [25], [26], [27], [34], [40], [41], [42], [45], [5] and [47]. Moreover, Wang et al [42] first tackled denoising, while Xu et al [46] first introduced 3D convolution considering temporal information of HDR videos.…”

“…As can be concluded from related works, there are 3 types of multi-branch strategies. First, resizing i.e., processing full-size input image with local/detail/large-scale branch while sending down-sampled image into global/smallscale branch ( [26], [27], [39] and [41]). Second, filtering i.e., decomposing image into base and detail components using filter (usually edge-preserving filter like bilateral filter) and respectively sending them into global and local/detail branches ( [31], [42], [45], [5] and [47]).…”

“…Similar to other methods, global branch in our multi-branch structure is responsible for understanding global luminance distribution, thus is supposed to have a global comprehension. Global or small-scale branch of [26], [39], [41] and [42] contain encoder structures to extract global features. Dilated convolution [55] was further used in [26] and [39] to enlarge their receptive field, thus strengthen the global comprehension of branches other than global branch.…”