Deep Learning-Based Computational Color Constancy With Convoluted Mixture of Deep Experts (CMoDE) Fusion Technique

Abstract

In the human and computer vision, color constancy is the ability to perceive the true color of objects in spite of changing illumination conditions. Color constancy is remarkably benefitting human and computer vision issues such as human tracking, object and human detection and scene understanding. Traditional color constancy approaches based on the gray world assumption fall short of performing a universal predictor, but recent color constancy methods have greatly progressed with the introduction of convolutional neural networks (CNNs). Yet, shallow CNN-based methods face learning capability limitations. Accordingly, this article proposes a novel color constancy method that uses a multi-stream deep neural network (MSDNN)-based convoluted mixture of deep experts (CMoDE) fusion technique in performing deep learning and estimating local illumination. In the proposed method, the CMoDE fusion technique is used to extract and learn spatial and spectral features in an image space. As opposed to previous works where residual networks stack multiple layers linearly and concatenate multiple paths, the proposed method distinctively piles up layers both in series and in parallel, selects and concatenates effective paths in the CMoDE-based DCNN. As a result, the proposed CMoDE-based DCNN brings significant progress to efficiency and accuracy of using computing resources and estimating illuminants, respectively. In the experiments, Shi's Reprocessed, gray-ball and NUS-8 Camera datasets are used to prove illumination and camera invariants. The experimental results illustrate that this new method surpasses its conventional counterparts. INDEX TERMS color constancy, CMoDE fusion technique, multi-stream deep neural network (MSDNN), illumination estimation, residual networks.

Very Deep Learning-Based Illumination Estimation Approach With Cascading Residual Network Architecture (CRNA)

“…This leads the architecture to falsely recognize small objects as noise and eliminate them. To solve the inaccuracy problem, Choi and his colleague [28], [29] set forth novel approaches by bringing the ResNet and the dilated convolution to the architecture. The two new approaches surpass their latest competitors from the estimation accuracy perspective.…”

“…[20] proposed two DCNN architectures and selected the better of the two through comparative studies. Other learning-based approaches include Bayesian learning [21], color moments [22], gamut mapping [23][24][25], spatial localization [26], [27], Choi's illuminant estimation approaches [28][29][30] and others [31], [32]. In short, the learning-based approaches have proven to outperform their statistics-based counterparts in terms of estimation accuracy throughout a lot of literature and studies.…”

Evolving convolution neural network by optimal regularization random vector functional link for computational color constancy

“…35 and 36, they tried to demonstrate that for the algorithm of color constancy, the data itself play a greater role in the algorithm than the pretrained or high-depth network model. Choi et al 37 proposed end-to-end deep learning network for color constancy computation. We listed the relevant color constancy algorithms, including classical and latest methods, and divided them, as shown in Table 1.…”

“…35 and 36, they tried to demonstrate that for the algorithm of color constancy, the data itself play a greater role in the algorithm than the pretrained or high-depth network model. Choi et al 37 . proposed end-to-end deep learning network for color constancy computation.…”

Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem

“…This is because many of the technologies based on deep learning have achieved excellent results in specific domains (semantic segmentation, object identification, emotion analysis, etc.) but also in the field of machine learning (Choi & Yun, 2020; Poterek, Herrault, Skupinski, & Sheeren, 2020; Renò et al, 2020; Sowmyayani & Rani, 2022). However, if you want to use a deep learning model in other contexts (not the one it was trained on), its performance drops rapidly (Huang et al, 2021; Sun & Saenko, 2016; Zhu et al, 2022).…”