VTON-SCFA: A Virtual Try-On Network Based on the Semantic Constraints and Flow Alignment

Du, Chenghu; Yu, Feng; Jiang, Minghua; Hua, Ailing; Xiong, Wei; Peng, Tao; Hu, Xinrong

doi:10.1109/tmm.2022.3152367

Cited by 23 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One model published in 2023 has achieved an impressive SSIM score of 0.918. The highest performing models for the FID metric in each year are ACGPN [195] in 2020, DP-VTON [22] in 2021, and VTON-SCFA [48] in 2022. As for SSIM, the highest performers are ACGPN [195] in 2020, Zflow [33] in 2021, and ST-VTON [32] in 2022.…”

Section: E Comparisonmentioning

confidence: 99%

Deep Learning in Virtual Try-On: A Comprehensive Survey

Islam,

Miron,

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Virtual try-on technology has gained significant importance in the retail industry due to its potential to transform the way customers interact with products and make purchase decisions. It allows users to virtually try on clothing and accessories, providing a realistic representation of how the items would look and fit without the need for physical interaction. The ability to virtually try on products addresses common challenges associated with online shopping, such as uncertainty about fit and style, ultimately enhancing the overall customer experience and satisfaction. As a result, virtual try-on technology has the potential to reduce returns and optimise conversion rates for businesses, making it a valuable tool in the e-commerce landscape. In this paper, we provide a comprehensive review of deep learning based virtual try-on models, focusing on their functionality, technical details, dataset usage, weaknesses, and impact on customer satisfaction. The models are categorised into three main types: image-based, multi-pose, and video virtual try-on models, with detailed examples and technical summaries provided for each category. Additionally, we identify and discuss similarities and differences in these methods. Furthermore, we examine the datasets currently available for building and evaluating virtual try-on models, including the number of images/videos and their resolutions. We present the commonly used methods for both qualitative and quantitative evaluations, comparing synthesised images with previous work and performing quantitative evaluations across various metrics and benchmark datasets. We discuss the weaknesses of current deep learning based virtual try-on models, including challenges in preserving clothing characteristics and textures, the level of accuracy of applying the clothing to the person, and the preservation of facial identities. Additionally, we address dataset bias, particularly the domination of female models, limited diversity in clothing featured, and relatively simple and clean backgrounds in the datasets, which can negatively impact the model's ability to handle challenging situations. Moreover, we explore the impact of virtual try-ons on customer satisfaction, highlighting the benefits that customers can enjoy, which also reduces returns and optimises conversion rates for businesses.INDEX TERMS Virtual try-on (VTO), deep learning, image synthesis, generative adversarial networks (GANs), diffusion models (DMs).

show abstract

Section: E Comparisonmentioning

confidence: 99%

Deep Learning in Virtual Try-On: A Comprehensive Survey

Islam,

Miron,

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Virtual Try-on. Existing deep learning-based methods on virtual try-on can be mainly categorized as 3D model based approaches [13]- [16] and 2D image-based ones [1], [2], [7], [9], [17]- [19]. As the former methods require 3D measurements, which bring extra computation resources.…”

Section: Related Workmentioning

confidence: 99%

OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup

Yang

Chen

Shi

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities. Prior methods successfully preserve the character of clothing images, however, occlusion remains a pernicious effect for realistic virtual try-on. In this work, we first present a comprehensive analysis of the occlusions and categorize them into two aspects: i) Inherent-Occlusion: the ghost of the former cloth still exists in the try-on image; ii) Acquired-Occlusion: the target cloth warps to the unreasonable body part. Based on the in-depth analysis, we find that the occlusions can be simulated by a novel semantically-guided mixup module, which can generate semantic-specific occluded images that work together with the try-on images to facilitate training a de-occlusion try-on (DOC-VTON) framework. Specifically, DOC-VTON first conducts a sharpened semantic parsing on the tryon person. Aided by semantics guidance and pose prior, various complexities of texture are selectively blending with human parts in a copy-and-paste manner. Then, the Generative Module (GM) is utilized to take charge of synthesizing the final try-on image and learning to de-occlusion jointly. In comparison to the state-ofthe-art methods, DOC-VTON achieves better perceptual quality by reducing occlusion effects.

show abstract

“…In response to this challenge, virtual try-on technology optimizes simulations that resemble real-world try-on scenarios. Existing research [1][2][3][4][5][6]predominantly focuses on virtual try-on solutions utilizing two-dimensional images to provide feedback on the chosen garment's appearance. Nevertheless, persistent challenges exist in accurately adjusting and aligning garments when the character's pose varies significantly and is affected by occlusion.…”

Section: Introductionmentioning

confidence: 99%

GIC-Flow: Appearance Flow Estimation Via Global Information Correlation for Virtual Try-On under large deformation

Zhang,

Zhan,

Sun

et al. 2024

Preprint

View full text Add to dashboard Cite

The primary aim of image-based virtual try-on is to seamlessly deform the target garment image to align with the human body. Due to the inherent non-rigid nature of garments, current methods prioritize flexible deformation through appearance flow with high degrees of freedom. However, existing appearance flow estimation methods focus solely on the correlation of local feature information. While this strategy successfully avoids the extensive computational effort associated with directly computing the global information correlation of feature maps, it leads to challenges in garments adapting to large deformation scenarios. To address these limitations, we propose the GIC-Flow framework, which obtains appearance flow by calculating the global information correlation while reducing computational regression. Specifically, our proposed Global Streak Information Matching Module is designed to decompose the appearance flow into horizontal and vertical vectors, effectively propagating global information in both directions. This innovative approach significantly diminishes 1 computational requirements, contributing to an enhanced and efficient process. Additionally, to ensure the accurate deformation of local texture in garments, we propose the Global Aggregate Information Matching Module to aggregate information from nearest neighbors before computing the global correlation, and to enhance weak semantic information. Comprehensive experiments conducted with our method on the VITON and VITON-HD datasets demonstrate GIC-Flow outperforms existing state-of-the-art algorithms, particularly in cases involving complex clothing deformation.

show abstract

VTON-SCFA: A Virtual Try-On Network Based on the Semantic Constraints and Flow Alignment

Cited by 23 publications

References 46 publications

Deep Learning in Virtual Try-On: A Comprehensive Survey

Deep Learning in Virtual Try-On: A Comprehensive Survey

OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup

GIC-Flow: Appearance Flow Estimation Via Global Information Correlation for Virtual Try-On under large deformation

Contact Info

Product

Resources

About