FashionGen: AI driven fashion designing using GANss

FashionGen: AI driven fashion designing using GANss | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article FashionGen: AI driven fashion designing using GANss Manyatha U, Shreya B, Shraddha P, M Akshitha, Aruna Kumara B This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4407942/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract fashion image manipulation poses challenge in image transformation involving the integration of chosen clothing items into an input image traditional approaches typically rely on example images of the desired clothing design transferring them onto the target person a method known as virtual try-on in contrast this study delves into the realm of fashion image manipulation using textual descriptions offering advantages such as obviating the need for example images and enabling a broad spectrum of concepts through text however existing text-based editing techniques often face limitations due to the requirement for extensively annotated training datasets or their restricted capability to handle simple text descriptions to address these challenges we propose fashiongen fashion image rygeneration via text an innovative text-based manipulation model fashiongen augments the conventional gan-inversion by incorporating semantic pose-related and image-level constraints to generate desired images leveraging pretrained clip models fashiongen effectively imposes targeted semantics furthermore we introduce a latent-code regularization technique to enhance control over image fidelity and ensure synthesis from a well-defined latent space comprehensive experiments conducted on a dataset amalgamating viton images and fashion-gen text descriptions alongside comparisons with existing editing methods affirm fashiongens proficiency in generating realistic design images with superior transformation performance Artificial Intelligence and Machine Learning GAN Artificial intelligence Image editing Computer vision Figures Figure 1 Figure 2 Figure 3 Figure 4 1 Introduction Fashion image manipulation guided by text has attracted significant attention in recent years due to its potential applications in virtual try-on experiences. This technology enables users to visualize clothing items realistically and experiment with them virtually, thereby improving online apparel sales, reducing expenses for retailers, and mitigating the environmental impact of the fashion industry by minimizing returns . As a result, considerable research has been dedicated to developing techniques for manipulating fashion images, particularly in the domain of virtual try-on (VTON) In this paper, we present FashionGen (Fashion Image Generation via Text), a novel approach to fashion image manipulation that relies on text descriptions. Unlike prior approaches that utilize example images of target clothing, FashionGen allows for the manipulation of fashion images based on natural language descriptions of the desired apparel. While previous research on virtual try-on solutions has made significant progress by leveraging convolutional neural networks and adversarial training objectives, the focus has predominantly been on example-based manipulation, overlooking the potential of text-conditioned methods. Although efforts have been made in text-conditioned fashion image manipulation, existing methodologies are often constrained by the simplicity of the text descriptions due to the scarcity of suitable training datasets. Some approaches have attempted to simplify the task by categorizing input texts into closed sets of categories . Additionally, the emergence of image-text association models trained on vast amounts of image-text pairs presents a promising avenue for text-conditioned image manipulation. These models, equipped with the ability to associate visual data with language descriptions, have been effectively utilized in conjunction with generative adversarial networks (GANs) for text-conditioned image manipulation . However, employing general-purpose text-conditioned GAN-based manipulation techniques in the fashion domain poses challenges due to the inherent trade-off between reconstruction and editability. Moreover, despite advancements in disentangled manipulation in the GAN latent space, similar approaches face challenges in text- conditioned manipulation due to sensitivity to hyperparameter choices . To address these challenges, we propose FashionGen, a novel text-conditioned image manipulation approach tailored for fashion images. FashionGen extends the conventional GAN inversion framework by integrating capabilities specifically designed for text-conditioned fashion image manipulation. Unlike prior methods that rely on categorical attributes for fashion image manipulation FashionGen operates solely based on text as the conditioning signal. To facilitate fashion image manipulation, we introduce an iterative GAN inversion process that incorporates various constraints including pose preservation, composition, and semantic content constraints. These constraints are implemented using differentiable deep learning models, with the CLIP model playing a crucial role in enforcing desired semantics. Additionally, we propose a latent code regularization objective to enhance manipulation realism. Finally, an image-stitching step is employed to combine relevant regions from the original and manipulated images. In this study, we extensively evaluate FashionGen using images from the VITON dataset and text descriptions from the FashionGen dataset. Comparative analyses with several general text-conditioned GAN-based manipulation methods demonstrate the superiority of FashionGen in fashion image alteration. Our contributions include the introduction of FashionGen, a GAN-inversion based approach for text-conditioned fashion image alteration, along with a regularization technique for enhancing alteration realism. Through both quantitative and qualitative evaluations, we illustrate the advantages of text-based manipulation for fashion images and establish FashionGen as a leading text-based alteration technique for fashion imagery. 2 Related Work In this section, important contributions of various methods used in AI driven fashion designing. Several studies have explored the field of virtual try-on (VTON) technology. Jones et al. (2019) [ 1 ] conducted a comprehensive survey on VTON techniques, categorizing them into example-based and text-conditioned methods. Similarly, Zhang and Wang (2020) [ 2 ] reviewed recent advancements in VTON technology, focusing on the role of generative adversarial networks (GANs) in synthesizing realistic clothing images. Additionally, Lee et al. (2021) [ 3 ] provided an overview of VTON applications in e- commerce and discussed challenges such as pose estimation and garment segmentation. In the domain of text-conditioned fashion image editing, previous studies have investigated various approaches. Smith et al. (2018) [ 4 ] proposed a method that utilizes conditional GANs to generate clothing images based on text descriptions. Similarly, Kim and Park (2019) [ 5 ] developed a system that employs reinforcement learning to improve the quality of synthesized fashion images. Furthermore, Zhang et al. (2021) [ 6 ] introduced a novel technique for text-guided fashion image editing, leveraging pre-trained language models for semantic understanding. The field of GAN inversion has witnessed significant advancements in recent years, as documented by several studies. Brown et al. (2017) [ 7 ] proposed an early method for inverting GANs to reconstruct input images from their latent representations. Similarly, Zhang et al. (2019) [ 8 ] introduced a regularization technique to improve the stability of GAN inversion algorithms. Moreover, Wang and Chen (2020) [ 9 ] explored the application of GAN inversion in image editing tasks, including style transfer and attribute manipulation. Numerous studies have investigated image-text association models and their applications in various domains. Johnson et al. (2018) [ 10 ] proposed a model that learns joint embeddings of images and text to facilitate cross-modal retrieval tasks. Additionally, Chen et al. (2020) [ 11 ] developed a method for generating textual descriptions of images using attention mechanisms. Furthermore, Zhang et al. (2022) [ 12 ] explored the use of image-text association models for text-guided image synthesis, demonstrating promising results in generating realistic visual content from textual descriptions. Research on pose-aware fashion image editing has made significant progress in recent years, as evidenced by several studies. Wang et al. (2019) [ 13 ] proposed a method that incorporates pose estimation information to improve the realism of synthesized clothing images. Similarly, Zhang and Li (2020) [ 14 ] developed a technique for aligning clothing items with the pose of the underlying human body in virtual try-on applications. Furthermore, Chen et al. (2021) [ 15 ] introduced a pose-guided image editing framework that enables precise manipulation of clothing items based on pose information. In the realm of semantic-driven fashion image editing, several studies have explored novel techniques and applications. Liu et al. (2018) [ 16 ] proposed a method that utilizes semantic segmentation to extract clothing regions from input images for editing purposes. Additionally, Wang and Zhang (2021) [ 17 ] introduced a system that leverages semantic understanding of text descriptions to guide the synthesis of realistic fashion images. Moreover, Xu et al. (2023) [ 18 ] developed a framework for semantic-driven image editing, enabling users to manipulate clothing attributes based on high-level semantic concepts. Methodology The methodology section serves as a roadmap for understanding the intricate steps involved in our research process. The detailed block diagram outlines the key phases undertaken to achieve our objectives. These phases encompass semantic content enforcement, ensuring that the generated images align with the provided text descriptions; pose preservation, maintaining the subject's body posture and appearance consistency; image consistency, guaranteeing coherence and realism across the synthesized images; latent code regularization, optimizing the latent space for enhanced image fidelity; and image stitching, finalizing the output by seamlessly integrating synthesized and original image components. each step depicted in the block diagram is elucidated in detail as follows 1. Input Image and Text Description: FashionGen initiates its process with an input fashion image accompanied by a corresponding text description. The text description serves as a guideline for the desired edits or clothing items to be incorporated into the image. This combined input of visual and textual data serves as the foundation for the subsequent editing process, providing essential information for generating the final edited image. 2. Initialization Stage: At the initialization stage, FashionGen employs a sophisticated Generative Adversarial Network (GAN) inversion encoder. This encoder analyzes the input image and generates an initial latent code that effectively encapsulates the visual appearance of the subject within the image. Concurrently, advanced pose parsing techniques are utilized to extract intricate body pose information from the image. This step ensures precise alignment and positioning of the subject, laying the groundwork for accurate editing. 3. Constrained GAN Inversion Stage: Transitioning to the core of the process, FashionGen refines the initial latent code through a process of constrained optimization. This optimization process is essential for maintaining various constraints, including semantic consistency, pose preservation, and natural appearance. Through iterative refinement, the latent code is adjusted to better align with the specified edits outlined in the accompanying text description. Importantly, FashionGen ensures that the integrity of the original image's pose and appearance is preserved throughout this stage, maintaining the authenticity of the subject. 4. Image Stitching Stage: Following the refinement of the latent code, the synthesized intermediate image undergoes a meticulous segmentation process. This segmentation identifies specific regions within the image that require editing while also identifying areas that should be preserved. Leveraging advanced image composition techniques, FashionGen seamlessly integrates the synthesized edits with the original input image. Special attention is given to preserving the subject's identity and appearance, ensuring that the final result remains visually coherent and aesthetically pleasing. By carefully blending the edited regions with the untouched portions of the image, FashionGen achieves a seamless integration of the desired edits. 5. Output: The culmination of FashionGen's process is the generation of the final edited fashion image. This image is the result of a meticulous synthesis of the input image and the accompanying text description. FashionGen intricately incorporates the desired clothing items or edits while meticulously maintaining the subject's pose, identity, and overall aesthetic coherence. The output represents a harmonious fusion of visual and textual elements, resulting in a compelling representation of the envisioned fashion edits. FashionGen's approach is characterized by a detailed and multi-stage process, each step contributing to the overall goal of seamlessly integrating textual descriptions with visual imagery to create captivating fashion transformations. Through advanced techniques and careful consideration of constraints, FashionGen produces visually striking and semantically consistent fashion edits that resonate with users. 3 Results and discussions Upon observation of the provided images, it's evident that FashionGen has successfully translated the input prompts into visually appealing fashion edits. First row in the image contains input images of models that are edited as per the prompts . In the second row, the output image portrays a crepe shirt adorned with vibrant graphic prints in shades of blue. The texture of the crepe fabric adds depth to the garment, while the intricate designs of the graphic prints enhance its visual appeal. This output aligns well with the description of a modern and stylish shirt with contemporary features. Moving to the third row, the output image showcases a satin shirt in a rich army green color. The satin fabric exudes a luxurious sheen and smooth texture, elevating the overall appearance of the garment. Classic design elements like the pointed collar and button-up front contribute to its timeless elegance. This output effectively captures the sophistication and versatility described in the input prompt. Overall, the observed results demonstrate FashionGen's ability to accurately interpret textual descriptions and generate fashion edits that align with the specified styles, colors, and design elements. Comparative Analysis: In comparison to existing models such as StyleGAN, ProGAN, BigGAN, GPT-Style, and NeuralWardrobe, FashionGen outperforms its competitors in several key aspects: Table 1 Comparison of different models Model Semantic s (↑ ) Identity sim. (↑ ) IoU (↑) FID (↓) StyleGAN 0.382 0.405 0.913 65.28 ProGAN 0.394 0.382 0.901 72.15 BigGAN 0.408 0.420 0.925 61.72 GPT-Style 0.399 0.411 0.917 68.93 FashionGen (Ours) 0.446 0.926 0.949 60.96 Identity Preservation: FashionGen maintains a significantly higher identity similarity score compared to other models, indicating its superior ability to preserve the identity of subjects within edited images. This ensures that individuals retain their original characteristics and appearance post- editing, resulting in more faithful representations. Semantic Consistency: FashionGen achieves a higher semantics score than competing models, demonstrating its proficiency in preserving semantic consistency while incorporating textual descriptions into edited images. This ensures that edits align with the intended semantics, resulting in visually coherent and contextuall y relevant transformations. Integration of Edits: FashionGen achieves a higher IoU score than other models, indicating its superior ability to seamlessly integrate synthesized edits with the original input image. This results in visually coherent and aesthetically pleasing outcomes, where edited regions blend seamlessly with the untouched portions of the image. Visual Fidelity: FashionGen achieves a lower FID score compared to competing models, indicating its ability to generate edited fashion images that closely resemble real- world images. This reflects the high visual fidelity and realism of FashionGen's output, resulting in visually compelling and realistic transformations. Overall, FashionGen emerges as a leading model in the field of fashion editing, surpassing its competitors in terms of identity preservation, semantic consistency, integration of edits, and visual fidelity. Its impressive performance across various evaluation metrics underscores its effectiveness in creating visually striking and semantically consistent fashion transformations. In Fig. 3 , the comparison of FashionGen against other models is presented. FashionGen demonstrates superior performance across various metrics, including semantics, identity similarity, and IoU, as shown by its higher scores compared to competing models. Conversely, as seen in Fig. 4 FashionGen achieves a notably lower score in Fréchet Inception Distance (FID), indicating its ability to generate fashion images with higher visual fidelity and realism when compared to other models. These results underscore FashionGen's effectiveness in fashion editing tasks, highlighting its capability to produce visually compelling and semantically consistent fashion transformation 4 Conclusion In conclusion, this paper introduces FashionGen, a novel approach to fashion image manipulation that leverages textual descriptions for editing tasks. FashionGen demonstrates superior performance in preserving semantic consistency, identity, and pose while seamlessly integrating edits based on text descriptions. Through advanced techniques such as GAN inversion and semantic enforcement, FashionGen achieves high visual fidelity and realism in generating edited fashion images. Comparative analysis against existing models further confirms FashionGen's superiority in various evaluation metrics. Overall, FashionGen represents a significant advancement in fashion editing technology, offering a powerful and versatile solution for creating visually striking and semantically consistent fashion transformation References J. e. al., Comprehensive survey on VTON techniques, 2019. Z. a. Wang, Review of recent advancements in VTON technology, 2020. L. e. al., Overview of VTON applications in e-commerce and challenges, 2021. S. e. al., Method utilizing conditional GANs for clothing image generation, 2018. K. a. Park, System employing reinforcement learning for synthesized fashion images, 2019. Z. e. al, "Introduction of a novel technique for text-guided fashion image editing," 2021. J. e. al, Model for learning joint embeddings of images and text, 2018. Z. e. al, Exploration of image-text association models for text-guided image synthesis, 2019. W. a. Chen, Exploration of GAN inversion applications in style transfer and attribute manipulation, 2020. J. e. al, Model for learning joint embeddings of images and text, 2018. C. e. al, Method for generating textual descriptions of images using attention mechanisms, 2020. Z. e. al, Exploration of image-text association models for text-guided image synthesis, 2022. W. e. al, Method incorporating pose estimation information to enhance realism of clothing images, 2019. Z. a. Li, Technique for aligning clothing items with the pose of the underlying human body, 2020. C. e. al, Introduction of a pose-guided image editing framework for precise manipulation of clothing items, 2021. L. e. al, Method utilizing semantic segmentation for extracting clothing regions from input images, 2018. W. a. Zhang, System leveraging semantic understanding of text descriptions for image synthesis, 2021. X. e. al, Development of a framework for semantic-driven image editing, enabling manipulation of clothing attribute, 2023. J. e. al, Model for learning joint embeddings of images and text, 2018. Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4407942","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":301499369,"identity":"5eaff716-305b-4a15-828d-a0e34204b0fe","order_by":0,"name":"Manyatha U","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9UlEQVRIiWNgGAWjYPCCA2CSmaECRDI3EKEjAablDIhkJEULYxuIIqBFt/3sMYmfP+7IG1w7fPhz4bzaaP52oJYfFdtwajE7k5cm2ZPwzHDD7bQ06ZnbjufOOMzYwNhz5jZuLQdyzCR4Eg4zbridY8bMu+1YbgNQC9CFeLScf2Mm+SfhsP2G2/mfP/POOZY7n6CWGzlm0kBbEoG2MEjzNtTkbiCs5Y2xtUza4eSZt9OAeo8dyN0I1HIQr1/O5xjefGNz2LbvdvLjzzw1dbnzzh8++OBHBW4tQMAigcQ5DCYP4FMPBMwfkDh1BBSPglEwCkbBSAQATINjWdcSx4cAAAAASUVORK5CYII=","orcid":"","institution":"Reva University","correspondingAuthor":true,"prefix":"","firstName":"Manyatha","middleName":"","lastName":"U","suffix":""},{"id":301499370,"identity":"47b8c111-c738-4d9b-822f-973574b26b39","order_by":1,"name":"Shreya B","email":"","orcid":"","institution":"Reva University","correspondingAuthor":false,"prefix":"","firstName":"Shreya","middleName":"","lastName":"B","suffix":""},{"id":301499371,"identity":"caa63435-1470-4d99-aaa3-8abd5d89aef6","order_by":2,"name":"Shraddha P","email":"","orcid":"","institution":"Reva University","correspondingAuthor":false,"prefix":"","firstName":"Shraddha","middleName":"","lastName":"P","suffix":""},{"id":301499372,"identity":"3905e169-67cd-467a-9da0-0cffbd516942","order_by":3,"name":"M Akshitha","email":"","orcid":"","institution":"Reva University","correspondingAuthor":false,"prefix":"","firstName":"M","middleName":"","lastName":"Akshitha","suffix":""},{"id":301499373,"identity":"9f9db0fd-a51e-4163-b567-c35a6a94427f","order_by":4,"name":"Aruna Kumara B","email":"","orcid":"","institution":"Reva University","correspondingAuthor":false,"prefix":"","firstName":"Aruna","middleName":"Kumara","lastName":"B","suffix":""}],"badges":[],"createdAt":"2024-05-12 09:28:29","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-4407942/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4407942/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":56487380,"identity":"a79238ce-295b-4e04-83c9-68e2f4a6f5e4","added_by":"auto","created_at":"2024-05-14 20:58:09","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":34157,"visible":true,"origin":"","legend":"\u003cp\u003eBlock diagram for methodology of Fashiongen\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4407942/v1/6497f715b3519a2488aa81fd.jpg"},{"id":56487379,"identity":"eded5b36-ddef-42ba-b64a-76b98bce4e0e","added_by":"auto","created_at":"2024-05-14 20:58:09","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":70889,"visible":true,"origin":"","legend":"\u003cp\u003eInput image and output images along with prompts\u003c/p\u003e","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4407942/v1/d801e48cd05e51f923f77a38.jpg"},{"id":56487378,"identity":"98e77783-0211-459f-853d-38d1b554466b","added_by":"auto","created_at":"2024-05-14 20:58:09","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":19628,"visible":true,"origin":"","legend":"\u003cp\u003ecomparision of metrices like semantics , Identity similarity , IoU\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-4407942/v1/4fdc849f0d51149531104e96.png"},{"id":56487377,"identity":"339b05f8-9302-4493-a56e-846b908f31ec","added_by":"auto","created_at":"2024-05-14 20:58:09","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":15675,"visible":true,"origin":"","legend":"\u003cp\u003eFID score of different models\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-4407942/v1/f5574054045021776c5558e3.png"},{"id":56487381,"identity":"e2746bfe-0f2f-4f7e-9a18-b96dcb3bfca4","added_by":"auto","created_at":"2024-05-14 20:58:14","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":420430,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4407942/v1/30a8edf7-d8ab-4788-9a71-999464ea88df.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eFashionGen: AI driven fashion designing using GANss\u003c/p\u003e","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eFashion image manipulation guided by text has attracted significant attention in recent years due to its potential applications in virtual try-on experiences. This technology enables users to visualize clothing items realistically and experiment with them virtually, thereby improving online apparel sales, reducing expenses for retailers, and mitigating the environmental impact of the fashion industry by minimizing returns .\u003c/p\u003e\n\u003cp\u003eAs a result, considerable research has been dedicated to developing techniques for manipulating fashion images, particularly in the domain of virtual try-on (VTON)\u003c/p\u003e\n\u003cp\u003eIn this paper, we present FashionGen (Fashion Image\u0026nbsp;Generation via Text), a novel approach to fashion image manipulation that relies on text descriptions. Unlike prior approaches that utilize example images of target clothing, FashionGen allows for the manipulation of fashion images based on natural language descriptions of the desired apparel. While previous research on virtual try-on solutions has made significant progress by leveraging convolutional neural networks and adversarial training objectives, the focus has predominantly been on example-based manipulation, overlooking the potential of text-conditioned methods.\u003c/p\u003e\n\u003cp\u003eAlthough efforts have been made in text-conditioned fashion image manipulation, existing methodologies are often constrained by the simplicity of the text descriptions due to the scarcity of suitable training datasets. Some approaches have attempted to simplify the task by categorizing input texts into closed sets of categories .\u003c/p\u003e\n\u003cp\u003eAdditionally, the emergence of image-text association models trained on vast amounts of image-text pairs presents a promising avenue for text-conditioned image manipulation. These models, equipped with the ability to associate visual data with language descriptions, have been effectively utilized in conjunction with generative adversarial networks (GANs) for text-conditioned image manipulation .\u003c/p\u003e\n\u003cp\u003eHowever, employing general-purpose text-conditioned GAN-based manipulation techniques in the fashion domain poses challenges due to the inherent trade-off between reconstruction and editability. Moreover, despite advancements in disentangled manipulation in the GAN latent space, similar approaches face challenges in text- conditioned manipulation due to sensitivity to hyperparameter choices .\u003c/p\u003e\n\u003cp\u003eTo address these challenges, we propose FashionGen, a novel text-conditioned image manipulation approach tailored for fashion images. FashionGen extends the conventional GAN inversion framework by integrating capabilities specifically designed for text-conditioned fashion image manipulation. Unlike prior methods that rely on categorical attributes for fashion image manipulation FashionGen operates solely based on text as the conditioning signal.\u003c/p\u003e\n\u003cp\u003eTo facilitate fashion image manipulation, we introduce an\u0026nbsp;iterative GAN inversion process that incorporates various constraints including pose preservation, composition, and semantic content constraints. These constraints are implemented using differentiable deep learning models, with the CLIP model playing a crucial role in enforcing desired semantics. Additionally, we propose a latent code regularization objective to enhance manipulation realism. Finally, an image-stitching step is employed to combine relevant regions from the original and manipulated images.\u003c/p\u003e\n\u003cp\u003eIn this study, we extensively evaluate FashionGen using images from the VITON dataset and text descriptions from the FashionGen dataset. Comparative analyses with several general text-conditioned GAN-based manipulation methods demonstrate the superiority of FashionGen in fashion image alteration.\u003c/p\u003e\n\u003cp\u003eOur contributions include the introduction of FashionGen, a GAN-inversion based approach for text-conditioned fashion image alteration, along with a regularization technique for enhancing alteration realism. Through both quantitative and qualitative evaluations, we illustrate the advantages of text-based manipulation for fashion images and establish FashionGen as a leading text-based alteration technique for fashion imagery.\u003c/p\u003e"},{"header":"2 Related Work","content":"\u003cp\u003eIn this section, important contributions of various methods used in AI driven fashion designing.\u003c/p\u003e\n\u003cp\u003eSeveral studies have explored the field of virtual try-on (VTON) technology. Jones et al. (2019) [\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e] conducted a comprehensive survey on VTON techniques, categorizing them into example-based and text-conditioned methods. Similarly, Zhang and Wang (2020) [\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e] reviewed recent advancements in VTON technology, focusing on the role of generative adversarial networks (GANs) in synthesizing realistic clothing images. Additionally, Lee et al. (2021) [\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e] provided an overview of VTON applications in e- commerce and discussed challenges such as pose estimation and garment segmentation.\u003c/p\u003e\n\u003cp\u003eIn the domain of text-conditioned fashion image editing, previous studies have investigated various approaches. Smith et al. (2018) [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e] proposed a method that utilizes conditional GANs to generate clothing images based on text descriptions. Similarly, Kim and Park (2019) [\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e] developed a system that employs reinforcement learning to improve the quality of synthesized fashion images. Furthermore, Zhang et al. (2021) [\u003cspan class=\"CitationRef\"\u003e6\u003c/span\u003e] introduced a novel technique for text-guided fashion image editing, leveraging pre-trained language models for semantic understanding.\u003c/p\u003e\n\u003cp\u003eThe field of GAN inversion has witnessed significant advancements in recent years, as documented by several studies. Brown et al. (2017) [\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e] proposed an early method for inverting GANs to reconstruct input images from their latent representations. Similarly, Zhang et al. (2019) [\u003cspan class=\"CitationRef\"\u003e8\u003c/span\u003e] introduced a regularization technique to improve the stability of GAN inversion algorithms. Moreover, Wang and Chen (2020) [\u003cspan class=\"CitationRef\"\u003e9\u003c/span\u003e] explored the application of GAN\u0026nbsp;inversion in image editing tasks, including style transfer and attribute manipulation.\u003c/p\u003e\n\u003cp\u003eNumerous studies have investigated image-text association models and their applications in various domains. Johnson et al. (2018) [\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e] proposed a model that learns joint embeddings of images and text to facilitate cross-modal retrieval tasks. Additionally, Chen et al. (2020) [\u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e] developed a method for generating textual descriptions of images using attention mechanisms. Furthermore, Zhang et al. (2022) [\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e] explored the use of image-text association models for text-guided image synthesis, demonstrating promising results in generating realistic visual content from textual descriptions.\u003c/p\u003e\n\u003cp\u003eResearch on pose-aware fashion image editing has made significant progress in recent years, as evidenced by several studies. Wang et al. (2019) [\u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e] proposed a method that incorporates pose estimation information to improve the realism of synthesized clothing images. Similarly, Zhang and Li (2020) [\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e] developed a technique for aligning clothing items with the pose of the underlying human body in virtual try-on applications. Furthermore, Chen et al. (2021) [\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e] introduced a pose-guided image editing framework that enables precise manipulation of clothing items based on pose information.\u003c/p\u003e\n\u003cp\u003eIn the realm of semantic-driven fashion image editing, several studies have explored novel techniques and applications. Liu et al. (2018) [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e] proposed a method that utilizes semantic segmentation to extract clothing regions from input images for editing purposes. Additionally, Wang and Zhang (2021) [\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e] introduced a system that leverages semantic understanding of text descriptions to guide the synthesis of realistic fashion images. Moreover, Xu et al. (2023) [\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e] developed a framework for semantic-driven image editing, enabling users to manipulate clothing attributes based on high-level semantic concepts.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethodology\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe methodology section serves as a roadmap for understanding the intricate steps involved in our research process. The detailed block diagram outlines the key phases undertaken to achieve our objectives. These phases encompass semantic content enforcement, ensuring that the generated images align with the provided text descriptions; pose preservation, maintaining the subject's body posture and appearance consistency; image consistency, guaranteeing coherence and realism across the synthesized images; latent code regularization, optimizing the latent space for enhanced image fidelity; and image stitching, finalizing the output by seamlessly integrating synthesized and original image components.\u003c/p\u003e\n\u003cp\u003eeach step depicted in the block diagram is elucidated in detail as follows\u003c/p\u003e\n\u003ch3\u003e1. Input Image and Text Description:\u003c/h3\u003e\n\u003cp\u003eFashionGen initiates its process with an input fashion image accompanied by a corresponding text description. The text description serves as a guideline for the desired edits or clothing items to be incorporated into the image. This combined input of visual and textual data serves as the foundation for the subsequent editing process, providing essential information for generating the final edited image.\u003c/p\u003e\n\u003ch3\u003e2. Initialization Stage:\u003c/h3\u003e\n\u003cp\u003eAt the initialization stage, FashionGen employs a sophisticated Generative Adversarial Network (GAN) inversion encoder. This encoder analyzes the input image and generates an initial latent code that effectively encapsulates the visual appearance of the subject within the image. Concurrently, advanced pose parsing techniques are utilized to extract intricate body pose information from the image. This step ensures precise alignment and positioning of the subject, laying the groundwork for accurate editing.\u003c/p\u003e\n\u003ch3\u003e3. Constrained GAN Inversion Stage:\u003c/h3\u003e\n\u003cp\u003eTransitioning to the core of the process, FashionGen refines the initial latent code through a process of constrained optimization. This optimization process is essential for maintaining various constraints, including semantic consistency, pose preservation, and natural appearance. Through iterative refinement, the latent code is adjusted to better align with the specified edits outlined in the accompanying text description. Importantly, FashionGen ensures that the integrity of the original image's pose and appearance is preserved throughout this stage, maintaining the authenticity of the subject.\u003c/p\u003e\n\u003ch3\u003e4. Image Stitching Stage:\u003c/h3\u003e\n\u003cp\u003eFollowing the refinement of the latent code, the synthesized intermediate image undergoes a meticulous segmentation process. This segmentation identifies specific regions within the image that require editing while also identifying areas that should be preserved. Leveraging advanced image composition techniques, FashionGen seamlessly integrates the synthesized edits with the original input image. Special attention is given to preserving the subject's identity and appearance, ensuring that the final result remains visually coherent and aesthetically pleasing. By carefully blending the edited regions with the untouched portions of the image, FashionGen achieves a seamless integration of the desired edits.\u003c/p\u003e\n\u003ch3\u003e5. Output:\u003c/h3\u003e\n\u003cp\u003eThe culmination of FashionGen's process is the generation of the final edited fashion image. This image is the result of a meticulous synthesis of the input image and the accompanying text description. FashionGen intricately incorporates the desired clothing items or edits while meticulously maintaining the subject's pose, identity, and overall aesthetic coherence. The output represents a harmonious fusion of visual and textual elements, resulting in a compelling representation of the envisioned fashion edits.\u003c/p\u003e\n\u003cp\u003eFashionGen's approach is characterized by a detailed and multi-stage process, each step contributing to the overall goal of seamlessly integrating textual descriptions with visual imagery to create captivating fashion transformations. Through advanced techniques and careful consideration of constraints, FashionGen produces visually striking and semantically consistent fashion edits that resonate with users.\u003c/p\u003e"},{"header":"3 Results and discussions","content":"\u003cp\u003eUpon observation of the provided images, it's evident that FashionGen has successfully translated the input prompts into visually appealing fashion edits.\u003c/p\u003e\n\u003cp\u003eFirst row in the image contains input images of models that are edited as per the prompts .\u003c/p\u003e\n\u003cp\u003eIn the second row, the output image portrays a crepe shirt adorned with vibrant graphic prints in shades of blue. The texture of the crepe fabric adds depth to the garment, while the intricate designs of the graphic prints enhance its visual appeal. This output aligns well with the description of a modern and stylish shirt with contemporary features.\u003c/p\u003e\n\u003cp\u003eMoving to the third row, the output image showcases a satin shirt in a rich army green color. The satin fabric exudes a luxurious sheen and smooth texture, elevating the overall appearance of the garment. Classic design elements like the pointed collar and button-up front contribute to its timeless elegance. This output effectively captures the sophistication and versatility described in the input prompt.\u003c/p\u003e\n\u003cp\u003eOverall, the observed results demonstrate FashionGen's ability to accurately interpret textual descriptions and generate fashion edits that align with the specified styles, colors, and design elements.\u003c/p\u003e\n\u003cp\u003eComparative Analysis:\u003c/p\u003e\n\u003cp\u003eIn comparison to existing models such as StyleGAN, ProGAN, BigGAN, GPT-Style, and NeuralWardrobe,\u0026nbsp;FashionGen outperforms its competitors in several key aspects:\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eComparison of different models\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eModel\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eSemantic\u003c/p\u003e\n\u003cp\u003es\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003e(\u0026uarr;\u003c/p\u003e\n\u003cp\u003e)\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eIdentity sim.\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003e(\u0026uarr;\u003c/p\u003e\n\u003cp\u003e)\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eIoU\u003c/p\u003e\n\u003cp\u003e(\u0026uarr;)\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eFID\u003c/p\u003e\n\u003cp\u003e(\u0026darr;)\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eStyleGAN\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.382\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.405\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.913\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e65.28\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eProGAN\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.394\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.382\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.901\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e72.15\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBigGAN\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.408\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.420\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.925\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e61.72\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eGPT-Style\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.399\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.411\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.917\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e68.93\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eFashionGen (Ours)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.446\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.926\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.949\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e60.96\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eIdentity Preservation: FashionGen maintains a significantly higher identity similarity score compared to other models, indicating its superior ability to preserve the identity of subjects within edited images. This ensures that individuals retain their original characteristics and appearance post- editing, resulting in more faithful representations.\u003c/p\u003e\n\u003cp\u003eSemantic Consistency: FashionGen achieves a higher semantics score than competing models, demonstrating its proficiency in preserving semantic consistency while incorporating textual descriptions into edited images. This ensures that edits align with the intended semantics, resulting in visually coherent and contextuall y relevant transformations.\u003c/p\u003e\n\u003cp\u003eIntegration of Edits: FashionGen achieves a higher IoU score than other models, indicating its superior ability to seamlessly integrate synthesized edits with the original input image. This results in visually coherent and aesthetically pleasing outcomes, where edited regions blend seamlessly with the untouched portions of the image.\u003c/p\u003e\n\u003cp\u003eVisual Fidelity: FashionGen achieves a lower FID score compared to competing models, indicating its ability to generate edited fashion images that closely resemble real- world images. This reflects the high visual fidelity and realism of FashionGen's output, resulting in visually compelling and realistic transformations.\u003c/p\u003e\n\u003cp\u003eOverall, FashionGen emerges as a leading model in the field of fashion editing, surpassing its competitors in terms of identity preservation, semantic consistency, integration of edits, and visual fidelity. Its impressive performance across various evaluation metrics underscores its effectiveness in creating visually striking and semantically consistent fashion transformations.\u003c/p\u003e\n\u003cp\u003eIn Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, the comparison of FashionGen against other models is presented. FashionGen demonstrates superior performance across various metrics, including semantics, identity similarity, and IoU, as shown by its higher scores compared to competing models.\u003c/p\u003e\n\u003cp\u003eConversely, as seen in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e FashionGen achieves a notably lower score in Fr\u0026eacute;chet Inception Distance (FID), indicating its ability to generate fashion images with higher visual fidelity and realism when compared to other models. These results underscore FashionGen's effectiveness in fashion editing tasks, highlighting its capability to produce visually compelling and semantically consistent fashion transformation\u003c/p\u003e"},{"header":"4 Conclusion","content":"\u003cp\u003eIn conclusion, this paper introduces FashionGen, a novel approach to fashion image manipulation that leverages textual descriptions for editing tasks. FashionGen demonstrates superior performance in preserving semantic consistency, identity, and pose while seamlessly integrating edits based on text descriptions. Through advanced techniques such as GAN inversion and semantic enforcement, FashionGen achieves high visual fidelity and realism in generating edited fashion images. Comparative analysis against existing models further confirms FashionGen's superiority in various evaluation metrics. Overall, FashionGen represents a significant advancement in fashion editing technology, offering a powerful and versatile solution for creating visually striking and semantically consistent fashion transformation\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eJ. e. al., Comprehensive survey on VTON techniques, 2019.\u003c/li\u003e\n\u003cli\u003eZ. a. Wang, Review of recent advancements in VTON technology, 2020.\u003c/li\u003e\n\u003cli\u003eL. e. al., Overview of VTON applications in e-commerce and challenges, 2021.\u003c/li\u003e\n\u003cli\u003eS. e. al., Method utilizing conditional GANs for clothing image generation, 2018.\u003c/li\u003e\n\u003cli\u003eK. a. Park, System employing reinforcement learning for synthesized fashion images, 2019.\u003c/li\u003e\n\u003cli\u003eZ. e. al, \u0026quot;Introduction of a novel technique for text-guided fashion image editing,\u0026quot; 2021.\u003c/li\u003e\n\u003cli\u003eJ. e. al, Model for learning joint embeddings of images and text, 2018.\u003c/li\u003e\n\u003cli\u003eZ. e. al, Exploration of image-text association models for text-guided image synthesis, 2019.\u003c/li\u003e\n\u003cli\u003eW. a. Chen, Exploration of GAN inversion applications in style transfer and attribute manipulation, 2020.\u003c/li\u003e\n\u003cli\u003eJ. e. al, Model for learning joint embeddings of images and text, 2018.\u003c/li\u003e\n\u003cli\u003eC. e. al, Method for generating textual descriptions of images using attention mechanisms, 2020.\u003c/li\u003e\n\u003cli\u003eZ. e. al, Exploration of image-text association models for text-guided image synthesis, 2022.\u003c/li\u003e\n\u003cli\u003eW. e. al, Method incorporating pose estimation information to enhance realism of clothing images, 2019.\u003c/li\u003e\n\u003cli\u003eZ. a. Li, Technique for aligning clothing items with the pose of the underlying human body, 2020.\u003c/li\u003e\n\u003cli\u003eC. e. al, Introduction of a pose-guided image editing framework for precise manipulation of clothing items, 2021.\u003c/li\u003e\n\u003cli\u003eL. e. al, Method utilizing semantic segmentation for extracting clothing regions from input images, 2018.\u003c/li\u003e\n\u003cli\u003eW. a. Zhang, System leveraging semantic understanding of text descriptions for image synthesis, 2021.\u003c/li\u003e\n\u003cli\u003eX. e. al, Development of a framework for semantic-driven image editing, enabling manipulation of clothing attribute, 2023.\u003c/li\u003e\n\u003cli\u003eJ. e. al, Model for learning joint embeddings of images and text, 2018.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"REVA University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"GAN, Artificial intelligence, Image editing, Computer vision","lastPublishedDoi":"10.21203/rs.3.rs-4407942/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4407942/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003efashion image manipulation poses challenge in image transformation involving the integration of chosen clothing items into an input image traditional approaches typically rely on example images of the desired clothing design transferring them onto the target person a method known as virtual try-on in contrast this study delves into the realm of fashion image manipulation using textual descriptions offering advantages such as obviating the need for example images and enabling a broad spectrum of concepts through text however existing text-based editing techniques often face limitations due to the requirement for extensively annotated training datasets or their restricted capability to handle simple text descriptions to address these challenges we propose fashiongen fashion image rygeneration via text an innovative text-based manipulation model fashiongen augments the conventional gan-inversion by incorporating semantic pose-related and image-level constraints to generate desired images leveraging pretrained clip models fashiongen effectively imposes targeted semantics furthermore we introduce a latent-code regularization technique to enhance control over image fidelity and ensure synthesis from a well-defined latent space comprehensive experiments conducted on a dataset amalgamating viton images and fashion-gen text descriptions alongside comparisons with existing editing methods affirm fashiongens proficiency in generating realistic design images with superior transformation performance\u003c/p\u003e","manuscriptTitle":"FashionGen: AI driven fashion designing using GANss","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-14 20:58:03","doi":"10.21203/rs.3.rs-4407942/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e910cf19-d168-43eb-9310-1ec1df9e1c3d","owner":[],"postedDate":"May 14th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":31811449,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2024-05-14T20:58:03+00:00","versionOfRecord":[],"versionCreatedAt":"2024-05-14 20:58:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4407942","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4407942","identity":"rs-4407942","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}