FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversion

Pernuš, Martin; Fookes, Clinton; Dobrišek, Simon

doi:10.48550/arxiv.2301.02110

Cited by 5 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Limitations include constraints on text description length and processing time, particularly for detailed object/logo descriptions. Nevertheless, FICE emerges as a powerful tool for fashion image editing, seamlessly integrating natural language understanding and image synthesis capabilities, with opportunities for further enhancements [4]. In this study, DiffCloth is introduced, a diffusion-based pipeline that has revolutionized cross-modal garment synthesis and manipulation.…”

Section: Literature Reviewmentioning

confidence: 99%

FashionMorph: Contextually Adaptive Clothing Replacement with CLIP, Segmentation, and Stable Diffusion

Goel,

Kadgi,

Gorka

et al. 2024

IJRASET

View full text Add to dashboard Cite

FashionMorph represents a groundbreaking advancement in digital image manipulation, seamlessly blending advanced segmentation techniques with the language comprehension capabilities of CLIP. This innovative approach simplifies the process of precise garment replacements while ensuring the preservation of realistic textures and boundaries. By incorporating stable diffusion, FashionMorph eliminates the need for extensive training on specific clothing styles, offering users a streamlined experience in expressing their preferences through natural language prompts. This empowers users in photo editing, granting them unparalleled creative freedom. The system comprises a comprehensive pipeline designed for the identification and inpainting of clothing elements. Leveraging CLIP and CLIPSeg models for robust image analysis, FashionMorph accurately identifies clothing elements and generates corresponding masks. The inpainting process utilizes a stable diffusion pipeline, resulting in impressive image quality that surpasses the performance of notable models like DCTransformer, StyleGAN, and ADM. Evaluations based on FID and SFID scores, along with Inception Score assessments, further validate FashionMorph's effectiveness in context-aware garment editing. Visual representations of original images, inpainted results, and masks vividly demonstrate the successful outcomes achieved by FashionMorph, establishing it as an efficient and user-friendly solution for professionals and enthusiasts alike. Among the evaluated models, Stable Diffusion emerges as the standout performer, boasting remarkable FID and SFID scores of 0.35 and 0.20, respectively, indicative of its outstanding image fidelity and diversity. Conversely, DCTransformer exhibits the highest FID and SFID scores, while StyleGAN excels in diversity but falls short in FID. ADM (dropout) strikes a balance between fidelity and diversity. These results underscore the exceptional performance of Stable Diffusion, solidifying its position as the leading approach with the lowest FID and SFID values

show abstract

Section: Literature Reviewmentioning

confidence: 99%

FashionMorph: Contextually Adaptive Clothing Replacement with CLIP, Segmentation, and Stable Diffusion

Goel,

Kadgi,

Gorka

et al. 2024

IJRASET

View full text Add to dashboard Cite

show abstract

“…DiffEdit (Couairon et al 2022) and Prompt-to-Prompt (Hertz et al 2022) achieved the goal of text-only editing by automatically predicting a mask before image editing. In the fashion domain, FICE (Pernuš et al 2023) employed latent code regularization to enhance the GAN inversion process by leveraging CLIP textual embeddings to guide the fashion image editing process. Multimodal Garment Designer (Baldrati et al 2023) proposed a multimodal-conditioned fashion image editing solution based on latent diffusion models.…”

Section: Related Workmentioning

confidence: 99%

“…The purpose of fashion image editing is to manipulate the clothes and accessories in the given fashion image to show the desired outfit combination for users. Successful applications of fashion image editing algorithms enable users and designers to visualize a personalized clothing in a realistic and visually convincing manner (Jiang and Fu 2017;Pernuš et al 2023;Baldrati et al 2023;Lin et al 2023). This topic has inspiring potential different fields, such as online apparel sales and social media.…”

Section: Introductionmentioning

confidence: 99%

“…This topic has inspiring potential different fields, such as online apparel sales and social media. With the development of generative models, significant research efforts have been devoted towards fashion image manipulation (Jiang and Fu 2017;Patashnik et al 2021;Xia et al 2021;Huang et al 2022;Pernuš et al 2023;Baldrati et al 2023;Lin et al 2023).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TexFit: Text-Driven Fashion Image Editing with Diffusion Models

Wang,

2024

AAAI

View full text Add to dashboard Cite

Fashion image editing aims to edit an input image to obtain richer or distinct visual clothing matching effects. Existing global fashion image editing methods are difficult to achieve rich outfit combination effects while local fashion image editing is more in line with the needs of diverse and personalized outfit matching. The local editing techniques typically depend on text and auxiliary modalities (e.g., human poses, human keypoints, garment sketches, etc.) for image manipulation, where the auxiliary modalities essentially assist in locating the editing region. Since these auxiliary modalities usually involve additional efforts in practical application scenarios, text-driven fashion image editing shows high flexibility. In this paper, we propose TexFit, a Text-driven Fashion image Editing method using diffusion models, which performs the local image editing only with the easily accessible text. Our approach employs a text-based editing region location module to predict precise editing region in the fashion image. Then, we take the predicted region as the generation condition of diffusion models together with the text prompt to achieve precise local editing of fashion images while keeping the rest part intact. In addition, previous fashion datasets usually focus on global description, lacking local descriptive information that can guide the precise local editing. Therefore, we develop a new DFMM-Spotlight dataset by using region extraction and attribute combination strategies. It focuses locally on clothes and accessories, enabling local editing with text input. Experimental results on the DFMM-Spotlight dataset demonstrate the effectiveness of our model. Code and Datasets are available at https://texfit.github.io/.

show abstract

“…Although tasks associated with fashion object detection may share similarities with those found in general computer vision benchmarks, the fashion industry's unique intricacies and domain-specific terminology warrant a tailored approach. Object detection is paramount in various applications, such as clothing product search [1] and virtual try-on [2,3]. Consequently, it is crucial to develop a specialized object detector explicitly designed for the fashion domain, which can contribute to advancing the field and improving real-world applications.…”

Section: Introductionmentioning

confidence: 99%

Efficient Fine Tuning for Fashion Object Detection

2023

Sensors

View full text Add to dashboard Cite

Pre-trained models have achieved success in object detection. However, challenges remain due to dataset noise and lack of domain-specific data, resulting in weaker zero-shot capabilities in specialized fields such as fashion imaging. We addressed this by constructing a novel clothing object detection benchmark, Garment40K, which includes more than 140,000 human images with bounding boxes and over 40,000 clothing images. Each clothing item within this dataset is accompanied by its corresponding category and textual description. The dataset covers 2 major categories, pants and tops, which are further divided into 15 fine-grained subclasses, providing a rich and high-quality clothing resource. Leveraging this dataset, we propose an efficient fine-tuning method based on the Grounding DINO framework to tackle the issue of missed and false detections of clothing targets. This method incorporates additional similarity loss constraints and adapter modules, leading to a significantly enhanced model named Improved Grounding DINO. By fine-tuning only a small number of additional adapter module parameters, we considerably reduced computational costs while achieving performance comparable to full parameter fine tuning. This allows our model to be conveniently deployed on a variety of low-cost visual sensors. Our Improved Grounding DINO demonstrates considerable performance improvements in computer vision applications in the clothing domain.

show abstract

FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversion

Cited by 5 publications

References 0 publications

FashionMorph: Contextually Adaptive Clothing Replacement with CLIP, Segmentation, and Stable Diffusion

FashionMorph: Contextually Adaptive Clothing Replacement with CLIP, Segmentation, and Stable Diffusion

TexFit: Text-Driven Fashion Image Editing with Diffusion Models

Efficient Fine Tuning for Fashion Object Detection

Contact Info

Product

Resources

About