Harmonizer: Learning to Perform White-Box Image and Video Harmonization

Ke, Zhanghan; Sun, Chunyi; Zhu, Lei; Xu, Ke; Lau, Rynson W. H.

doi:10.1007/978-3-031-19784-0_40

Cited by 44 publications

(26 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These filters can make the process of image harmonization explicit and easy to explain. Harmonizer [5] extract the information of brightness, contrast, saturation, color temperature, highlight, and shadow (different features of image color) via regressors. Through learning and superimposing the above different features, the model realizes the process of learning information layer by layer.…”

Section: Color Curvementioning

confidence: 99%

The Way to picture Reality: A Review, Evaluation and Recipe of Image Harmonization

Tian

Hou

Zhang

2023

Preprint

View full text Add to dashboard Cite

Image harmonization is an interesting but challenging task in image processing. The task is to adjust the visual information between the foreground and background in the picture, so that the overall picture becomes a harmonious and unified process. Compared to traditional unsupervised methods for image processing, recent developments in deep learning have brought new opportunities to the field. Since 2019, related image harmonization papers have begun to appear on a large scale. In order to better sort out the development of this field, we summarize the field from two aspects of supervised learning and unsupervised learning, and also deeply integrate the existing image harmonization dataset and related index evaluation. We then conduct a detailed comparison and analysis of the existing models. Finally, the current challenges and future research directions of deep learning in image harmonization are discussed.

show abstract

Section: Color Curvementioning

confidence: 99%

The Way to picture Reality: A Review, Evaluation and Recipe of Image Harmonization

Tian

Hou

Zhang

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, more deep learning-based methods have been proposed with notable successes [4,5,10,11,15,18,31]. DoveNet [6] and BargainNet [4] regard image harmonization as a domain translation problem and focus on improving domain consistency between back-and fore-ground.…”

Section: Image Harmonizationmentioning

confidence: 99%

“…A hierarchical dynamic network (HDNet) that enables feature dynamics from both local and global views is proposed, gradually building local dynamics and mask-aware global dynamics. As shown in Figure 2, following [4,5,13,15], we employ the U-Net structure as our Generator G to harmonize the fore-ground.…”

Section: Hierarchical Dynamic Network (Hdnet)mentioning

confidence: 99%

“…We use PyTorch [25] to implement our models with Nvidia Tesla A100 GPUs. including DIH [31], S 2 AM [7], DoveNet [6], RainNet [21], Bargainnet [4], Intrinsic [11], D-HT [10], CDTNet [5], Harmonizer [15], DCCF [34] and S 2 CRNet [18].…”

Section: Experiments Settingmentioning

confidence: 99%

“…However, they could not handle the complex situation where the source image has a large appearance or semantic gap with the target. With the advances in deep learning, more deep neural network-based methods were proposed [4][5][6]15]. Most methods use complex network structures or training strategies [4,6].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Hierarchical Dynamic Image Harmonization

Chen¹,

Gu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image harmonization is a critical task in computer vision, which aims to adjust the fore-ground to make it compatible with the back-ground. Recent works mainly focus on using global transformation (i.e., normalization and color curve rendering) to achieve visual consistency. However, these model ignore local consistency and their model size limit their harmonization ability on edge devices. Inspired by the dynamic deep networks that adapt the model structures or parameters conditioned on the inputs, we propose a hierarchical dynamic network (HDNet) for efficient image harmonization to adapt the model parameters and features from local to global view for better feature transformation. Specifically, local dynamics (LD) and mask-aware global dynamics (MGD) are applied. LD enables features of different channels and positions to change adaptively and improve the representation ability of geometric transformation through structural information learning. MGD learns the representations of fore-and back-ground regions and correlations to global harmonization. Experiments show that the proposed HDNet reduces more than 80% parameters compared with previous methods but still achieves the state-of-the-art performance on the popular iHarmony4 dataset. Our code is avaliable in https://github. com/chenhaoxing/HDNet.

show abstract