CMAFGAN: A Cross-Modal Attention Fusion based Generative Adversarial Network for attribute word-to-face synthesis

Luo, Xiaodong; Chen, Xiang; He, Xiaohai; Qing, Linbo; Tan, Xiaoli

doi:10.1016/j.knosys.2022.109750

Cited by 13 publications

(2 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CMAF block. Inspired by the methodological framework of CMAFGAN [ 25 ], for simple notation, we denoted

and

as x and y, respectively. As shown in Figure 3 , the CMAF block began with six 1 × 1 convolution layers applied to x and y.…”

Section: Methodsmentioning

confidence: 99%

Intracerebral Hemorrhage Prognosis Classification via Joint-Attention Cross-Modal Network

Xu,

Fu,

Jin

et al. 2024

Brain Sciences

View full text Add to dashboard Cite

Intracerebral hemorrhage (ICH) is a critical condition characterized by a high prevalence, substantial mortality rates, and unpredictable clinical outcomes, which results in a serious threat to human health. Improving the timeliness and accuracy of prognosis assessment is crucial to minimizing mortality and long-term disability associated with ICH. Due to the complexity of ICH, the diagnosis of ICH in clinical practice heavily relies on the professional expertise and clinical experience of physicians. Traditional prognostic methods largely depend on the specialized knowledge and subjective judgment of healthcare professionals. Meanwhile, existing artificial intelligence (AI) methodologies, which predominantly utilize features derived from computed tomography (CT) scans, fall short of capturing the multifaceted nature of ICH. Although existing methods are capable of integrating clinical information and CT images for prognosis, the effectiveness of this fusion process still requires improvement. To surmount these limitations, the present study introduces a novel AI framework, termed the ICH Network (ICH-Net), which employs a joint-attention cross-modal network to synergize clinical textual data with CT imaging features. The architecture of ICH-Net consists of three integral components: the Feature Extraction Module, which processes and abstracts salient characteristics from the clinical and imaging data, the Feature Fusion Module, which amalgamates the diverse data streams, and the Classification Module, which interprets the fused features to deliver prognostic predictions. Our evaluation, conducted through a rigorous five-fold cross-validation process, demonstrates that ICH-Net achieves a commendable accuracy of up to 87.77%, outperforming other state-of-the-art methods detailed within our research. This evidence underscores the potential of ICH-Net as a formidable tool in prognosticating ICH, promising a significant advancement in clinical decision-making and patient care.

show abstract

“…CMAF block. Inspired by the methodological framework of CMAFGAN [ 25 ], for simple notation, we denoted

and

as x and y, respectively. As shown in Figure 3 , the CMAF block began with six 1 × 1 convolution layers applied to x and y.…”

Section: Methodsmentioning

confidence: 99%

Intracerebral Hemorrhage Prognosis Classification via Joint-Attention Cross-Modal Network

Xu,

Fu,

Jin

et al. 2024

Brain Sciences

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the discriminator aims to distinguish between the synthetic image and the matched natural image. The input conditions used by GAN-based image synthesis methods are various, such as sparse sketches [19][20][21] , gaussian noise 22,23 , text descriptions [24][25][26] , natural images 27,28 , and semantic layout [29][30][31][32] . Considering the great success of GANs in image synthesis, we propose a novel GAN-based approach to tackle image synthesis conditioned only on semantic layout.…”

Section: Introductionmentioning

confidence: 99%

UNet-like network fused Swin Transformer and CNN for Semantic Image Synthesis

Ke,

Luo,

Cai

2024

Preprint

View full text Add to dashboard Cite

Semantic image synthesis approaches has been dominated by the modelling of Convolutional Neural Networks (CNN). Due to the limitations of local perception, their performance improvement seems to have plateaued in recent years. To tackle this issue, we propose the TransUNet model, which is a UNet-like network fused Swin Transformer and CNN for semantic image synthesis. Photorealistic image synthesis conditional on the given semantic layout depends on the high-level semantics and the low-level positions. To improve the synthesis performance, we design a novel conditional residual fusion module for the model decoder to efficiently fuse the hierarchical feature maps extracted at different scales. Moreover, this module combines the opposition-based learning mechanism and the weight assignment mechanism for enhancing and attending the semantic information. Compared to pure CNN-based models, our TransUNet combines the local and global perceptions to better extract high- and low-level features and better fuse multi-scale features. We have conducted an extensive amount of comparison experiments, both in quantitative and qualitative terms, to validate the effectiveness of our proposed TransUNet model for semantic image synthesis. The outcomes illustrate that TransUNet distinctively outperforms the state-of-the-art model on three benchmark datasets (Citysacpes, ADE20K, and COCO-Stuff) including numerous real-scene images.

show abstract