How does Color Constancy Affect Target Recognition and Instance Segmentation?

Xue, Siyan; Gao, Shaobing; Tan, Minjie; He, Zhen; He, Liangtian

doi:10.1145/3474085.3475690

Cited by 5 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to RGB and depth data, skeleton data are more robust than RGB and depth data to illumination change and viewpoint occlusion. Advances in depth cameras and pose estimation methods [9][10][11][12] have also made it easier for bionic robots to acquire skeleton data.…”

Section: Introductionmentioning

confidence: 99%

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition

Hu,

Liu

2024

Biomimetics

View full text Add to dashboard Cite

Skeleton-based human interaction recognition is a challenging task in the field of vision and image processing. Graph Convolutional Networks (GCNs) achieved remarkable performance by modeling the human skeleton as a topology. However, existing GCN-based methods have two problems: (1) Existing frameworks cannot effectively take advantage of the complementary features of different skeletal modalities. There is no information transfer channel between various specific modalities. (2) Limited by the structure of the skeleton topology, it is hard to capture and learn the information about two-person interactions. To solve these problems, inspired by the human visual neural network, we propose a multi-modal enhancement transformer (ME-Former) network for skeleton-based human interaction recognition. ME-Former includes a multi-modal enhancement module (ME) and a context progressive fusion block (CPF). More specifically, each ME module consists of a multi-head cross-modal attention block (MH-CA) and a two-person hypergraph self-attention block (TH-SA), which are responsible for enhancing the skeleton features of a specific modality from other skeletal modalities and modeling spatial dependencies between joints using the specific modality, respectively. In addition, we propose a two-person skeleton topology and a two-person hypergraph representation. The TH-SA block can embed their structural information into the self-attention to better learn two-person interaction. The CPF block is capable of progressively transforming the features of different skeletal modalities from low-level features to higher-order global contexts, making the enhancement process more efficient. Extensive experiments on benchmark NTU-RGB+D 60 and NTU-RGB+D 120 datasets consistently verify the effectiveness of our proposed ME-Former by outperforming state-of-the-art methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition

Hu,

Liu

2024

Biomimetics

View full text Add to dashboard Cite

show abstract

“…Color constancy is a fundamental research topic in the image-processing and computervision fields, and it has many applications in photographic technology, object recognition, object detection, image segmentation, and other version systems. Color casts caused by incorrectly applied computational color constancy can negatively impact the performance of image segmentation and classification (Afifi and Brown, 2019;Xue et al, 2021), thus, there is a rich body of work on this topic. Generally, methods for obtaining color constancy with image data are divided into two main categories: lowlevel-feature-based methods (Buchsbaum, 1980;Brainard and Wandell, 1986;Lee, 1986;Wandell and Tominaga, 1989;Nieves et al, 2000;Krasilnikov et al, 2002;Weijer et al, 2007;Gehler et al, 2008;Tan et al, 2008;Toro, 2008;Gijsenij et al, 2011;Finlayson, 2013;Gao et al, 2013;Barron, 2015;Bianco et al, 2015Bianco et al, , 2017Cheng et al, 2015;Shi et al, 2016;Xiao et al, 2020;Yu et al, 2020) and semantic-feature-based methods (Schroeder and Moser, 2001;Spitzer and Semo, 2002;Van De Weijer et al, 2007;Bianco et al, 2008;Lau, 2008;Li et al, 2008;Gao et al, 2015;Afifi, 2018).…”

Section: Introductionmentioning

confidence: 99%

Color Constancy via Multi-Scale Region-Weighed Network Guided by Semantics

Wang

et al. 2022

Front. Neurorobot.

View full text Add to dashboard Cite

In obtaining color constancy, estimating the illumination of a scene is the most important task. However, due to unknown light sources and the influence of the external imaging environment, the estimated illumination is prone to color ambiguity. In this article, a learning-based multi-scale region-weighed network guided by semantic features is proposed to estimate the illuminated color of the light source in a scene. Cued by the human brain's processing of color constancy, we use image semantics and scale information to guide the process of illumination estimation. First, we put the image and its semantics into the network, and then obtain the region weights of the image at different scales. After that, through a special weight-pooling layer (WPL), the illumination on each scale is estimated. The final illumination is calculated by weighting each scale. The results of extensive experiments on Color Checker and NUS 8-Camera datasets show that the proposed approach is superior to the current state-of-the-art methods in both efficiency and effectiveness.

show abstract