A Multimodal Transformer-Based Generative Adversarial Network for Age-Aware Sketch-to-Photo Synthesis.

A Multimodal Transformer-Based Generative Adversarial Network for Age-Aware Sketch-to-Photo Synthesis. | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Multimodal Transformer-Based Generative Adversarial Network for Age-Aware Sketch-to-Photo Synthesis. Vedant Maheshwari, Mithun Kumar Kar, Ayswarya R Kurup, Debanga Raj Neog, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9370432/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 9 You are reading this latest preprint version Abstract The synthesis of photorealistic facial images from sparse hand-drawn sketches, while preserving demographic attributes such as age, remains challenging in computer vision. This work presents a multi-task Generative Adversarial Network (GAN) that jointly addresses sketch-to-image translation and age prediction within a unified end-to-end architecture. The generator of the GAN model makes use of a pretrained ResNet-based content encoder with a 4-block Transformer bottleneck followed by a synthesis decoder network. The model utilizes this setup to comprehend dependencies that occur over a significant spatial distance. To align its demographic characteristics with the synthesis process, an age encoder network predicts a normalized age scalar directly from the sketch. The synthesis decoder is dynamically guided by age conditioning with a scheduled sampling curriculum through adaptive instance normalization (AdaIN) to mitigate the discrepancy between training and inference. To maintain consistent age and texture at the patch level, the proposed multi-scale CNN discriminator examines 4-channel inputs (the RGB image plus a uniform age map) at distinct resolutions. The model was tested on different datasets and obtained a learned perceptual image patch similarity of 0.1982 with a PSNR of around 20.31 dB. The MAE for age prediction is 9.29 years on unconstrained images and 2.09 years on aligned faces. The architecture is computationally efficient, which makes it an effective baseline for forensics and identity-preserving generation. Adaptive Instance Normalization (AdaIN) age prediction face sketch synthesis Generative Adversarial Networks (GANs) image-to-image translation multi-task learning Transformer bottleneck Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 12 May, 2026 Reviews received at journal 09 May, 2026 Reviews received at journal 05 May, 2026 Reviewers agreed at journal 05 May, 2026 Reviewers agreed at journal 23 Apr, 2026 Reviewers invited by journal 23 Apr, 2026 Editor assigned by journal 22 Apr, 2026 Submission checks completed at journal 21 Apr, 2026 First submitted to journal 21 Apr, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9370432","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":630170111,"identity":"ca73089d-0a92-4fdf-93e8-c9529c46caaf","order_by":0,"name":"Vedant Maheshwari","email":"","orcid":"","institution":"Amrita Vishwa Vidyapeetham University","correspondingAuthor":false,"prefix":"","firstName":"Vedant","middleName":"","lastName":"Maheshwari","suffix":""},{"id":630170113,"identity":"57f71c9c-82d9-4397-b2ed-2e08ca48f7b9","order_by":1,"name":"Mithun Kumar Kar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABE0lEQVRIiWNgGAWjYBACCSA+wNgAYTAwVDAkgGlkEquWg3AtZ2CKE/BrYYBrYWyDKcOjRbKBO/Hwxx2H8yRnZCd+rpxnl8fAfjrtw8Mfdgz87DkG2LRIM/BuOHDwzOFiaYnczZJntyUXM/Dkbp6RkJDMINnzBqsWObCWtsOJ8yRyN0g2bjuQ2MCQuxnoF2YGgxvYbUHWsvln4xygFv63IC31DPY4tEjDtMyWyN0m2dgA1CIBtuUwg4EEdi2SDUAtZ9vSE2f2vN1m2XAsuZhNAmRL2nEeiTPPCrBpkTjAu/lDZZt14ozjuZtvNtTY5fHz525m/GFTLcffnrwBayjLP0ATYIPSPFiVj4JRMApGwSggCgAA/R9prVWhCpgAAAAASUVORK5CYII=","orcid":"","institution":"Amrita Vishwa Vidyapeetham University","correspondingAuthor":true,"prefix":"","firstName":"Mithun","middleName":"Kumar","lastName":"Kar","suffix":""},{"id":630170114,"identity":"50d2a98c-946a-4f78-a833-5469e17b5d86","order_by":2,"name":"Ayswarya R Kurup","email":"","orcid":"","institution":"Amrita Vishwa Vidyapeetham University","correspondingAuthor":false,"prefix":"","firstName":"Ayswarya","middleName":"R","lastName":"Kurup","suffix":""},{"id":630170115,"identity":"a5dff949-3ab6-4047-8777-27f6555ad1d3","order_by":3,"name":"Debanga Raj Neog","email":"","orcid":"","institution":"Indian Institute of Technology Guwahati","correspondingAuthor":false,"prefix":"","firstName":"Debanga","middleName":"Raj","lastName":"Neog","suffix":""},{"id":630170116,"identity":"b9260634-12eb-4b4c-bba8-aff600fd5106","order_by":4,"name":"Malaya Kumar Nath","email":"","orcid":"","institution":"National Institute of Technology Puducherry","correspondingAuthor":false,"prefix":"","firstName":"Malaya","middleName":"Kumar","lastName":"Nath","suffix":""}],"badges":[],"createdAt":"2026-04-09 15:24:26","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9370432/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9370432/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108007347,"identity":"c48e130f-f5ad-41db-9fc1-db4acf5bc646","added_by":"auto","created_at":"2026-04-28 12:59:35","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1530364,"visible":true,"origin":"","legend":"","description":"","filename":"VedantAIDiscovery.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9370432/v1_covered_68e39ea3-e182-49e8-bf90-858b71881f7c.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Multimodal Transformer-Based Generative Adversarial Network for Age-Aware Sketch-to-Photo Synthesis.","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"discover-artificial-intelligence","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"diai","sideBox":"Learn more about [Discover Artificial Intelligence](https://www.springer.com/44163)","snPcode":"","submissionUrl":"","title":"Discover Artificial Intelligence","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Discover Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Adaptive Instance Normalization (AdaIN), age prediction, face sketch synthesis, Generative Adversarial Networks (GANs), image-to-image translation, multi-task learning, Transformer bottleneck ","lastPublishedDoi":"10.21203/rs.3.rs-9370432/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9370432/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The synthesis of photorealistic facial images from sparse hand-drawn sketches, while preserving demographic attributes such as age, remains challenging in computer vision. This work presents a multi-task Generative Adversarial Network (GAN) that jointly addresses sketch-to-image translation and age prediction within a unified end-to-end architecture. The generator of the GAN model makes use of a pretrained ResNet-based content encoder with a 4-block Transformer bottleneck followed by a synthesis decoder network. The model utilizes this setup to comprehend dependencies that occur over a significant spatial distance. To align its demographic characteristics with the synthesis process, an age encoder network predicts a normalized age scalar directly from the sketch. The synthesis decoder is dynamically guided by age conditioning with a scheduled sampling curriculum through adaptive instance normalization (AdaIN) to mitigate the discrepancy between training and inference. To maintain consistent age and texture at the patch level, the proposed multi-scale CNN discriminator examines 4-channel inputs (the RGB image plus a uniform age map) at distinct resolutions. The model was tested on different datasets and obtained a learned perceptual image patch similarity of 0.1982 with a PSNR of around 20.31 dB. The MAE for age prediction is 9.29 years on unconstrained images and 2.09 years on aligned faces. The architecture is computationally efficient, which makes it an effective baseline for forensics and identity-preserving generation.","manuscriptTitle":"A Multimodal Transformer-Based Generative Adversarial Network for Age-Aware Sketch-to-Photo Synthesis.","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-28 10:48:02","doi":"10.21203/rs.3.rs-9370432/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-05-12T09:13:59+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-09T05:12:55+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-05T13:03:26+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"47493111739155900201768865026982301038","date":"2026-05-05T05:36:56+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"141524738834126994637241951932621464702","date":"2026-04-23T06:25:40+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-23T05:03:13+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-22T09:57:01+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-04-21T16:46:46+00:00","index":"","fulltext":""},{"type":"submitted","content":"Discover Artificial Intelligence","date":"2026-04-21T15:52:31+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"discover-artificial-intelligence","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"diai","sideBox":"Learn more about [Discover Artificial Intelligence](https://www.springer.com/44163)","snPcode":"","submissionUrl":"","title":"Discover Artificial Intelligence","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Discover Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"0a87bd8b-eec3-4ace-80b4-b947980c919f","owner":[],"postedDate":"April 28th, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Revision requested","date":"2026-05-12T09:13:59+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-09T05:12:55+00:00","index":38,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-05T13:03:26+00:00","index":37,"fulltext":""},{"type":"reviewerAgreed","content":"47493111739155900201768865026982301038","date":"2026-05-05T05:36:56+00:00","index":36,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-05-12T09:25:02+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-28 10:48:02","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9370432","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9370432","identity":"rs-9370432","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}