CMOS-GAN: Semi-Supervised Generative Adversarial Model for Cross-Modality Face Image Synthesis

Yu, Shikang; Han, Hu; Shan, Shiguang; Chen, Xilin

doi:10.1109/tip.2022.3226413

Cited by 8 publications

(1 citation statement)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conduct experiments on two NIR&VIS facial expression datasets: Oulu-CASIA (Zhao et al 2011) and the proposed Large-HFE. There are rather few publicly available datasets in the field of NIR-VIS heterogeneous FER (Yu et al 2022).…”

Section: Datasetsmentioning

confidence: 99%

Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition

Luo,

Wang,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

With the strong robusticity on illumination variations, near-infrared (NIR) can be an effective and essential complement to visible (VIS) facial expression recognition in low lighting or complete darkness conditions. However, facial expression recognition (FER) from NIR images presents a more challenging problem than traditional FER due to the limitations imposed by the data scale and the difficulty of extracting discriminative features from incomplete visible lighting contents. In this paper, we give the first attempt at deep NIR facial expression recognition and propose a novel method called near-infrared facial expression transformer (NFER-Former). Specifically, to make full use of the abundant label information in the field of VIS, we introduce a Self-Attention Orthogonal Decomposition mechanism that disentangles the expression information and spectrum information from the input image, so that the expression features can be extracted without the interference of spectrum variation. We also propose a Hypergraph-Guided Feature Embedding method that models some key facial behaviors and learns the structure of the complex correlations between them, thereby alleviating the interference of inter-class similarity. Additionally, we construct a large NIR-VIS Facial Expression dataset that includes 360 subjects to better validate the efficiency of NFER-Former. Extensive experiments and ablation studies show that NFER-Former significantly improves the performance of NIR FER and achieves state-of-the-art results on the only two available NIR FER datasets, Oulu-CASIA and Large-HFE.

show abstract

Section: Datasetsmentioning

confidence: 99%

Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition

Luo,

Wang,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

Toward identity preserving in face sketch-photo synthesis using a hybrid CNN-Mamba framework

Tang,

Jiang,

Wang

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The synthesis of facial sketch-photo has important applications in practical life, such as crime investigation. Many convolutional neural networks (CNNs) based methods have been proposed to address this issue. However, due to the substantial modal differences between sketch and photo, the CNN’s insensitivity to global information, and insufficient utilization of hierarchical features, synthesized photos struggle to balance both identity preservation and image quality. Recently, State Space Sequence Models (SSMs) have achieved exciting results in computer vision (CV) tasks. Inspired by SSMs, we design a hybrid CNN–SSM model called FaceMamba for the Face Sketch-Photo Synthesis (FSPS) task. It includes an original Face Vision Mamba Attention for modeling in latent space using SSM. Additionally, it incorporates a general auxiliary method called Attention Feature Injection that combines encoding features, decoding features, and external auxiliary features using attention mechanisms. FaceMamba combines Mamba’s modeling ability for long-range dependencies with CNN’s powerful local feature extraction ability, and utilizes hierarchical features at the appropriate position. Adequate experimental and evaluation results reveal that FaceMamba has strong competitiveness in FSPS task, achieving the best balance between identity preservation and image quality.

show abstract

Unsupervised NIR-VIS Face Recognition via Homogeneous-to-Heterogeneous Learning and Residual-Invariant Enhancement

Yang,

Hu,

2024

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

CMOS-GAN: Semi-Supervised Generative Adversarial Model for Cross-Modality Face Image Synthesis

Cited by 8 publications

References 58 publications

Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition

Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition

Toward identity preserving in face sketch-photo synthesis using a hybrid CNN-Mamba framework

Unsupervised NIR-VIS Face Recognition via Homogeneous-to-Heterogeneous Learning and Residual-Invariant Enhancement

Contact Info

Product

Resources

About