A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Survey paper A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks Vignesh Sampath, Iñaki Maurtua, Juan José Aguilar Martín, Aitor Gutierrez This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-45616/v4 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 29 Jan, 2021 Read the published version in Journal of Big Data → Version 4 posted 7 You are reading this latest preprint version Show more versions Abstract Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets. In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms. Computer Architecture and Engineering Generative Adversarial Neural Networks Imbalanced data Object detection Segmentation Classification Deep learning Deep generative model Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 Figure 13 Figure 14 Figure 15 Figure 16 Figure 17 Figure 18 Figure 19 Figure 20 Figure 21 Figure 22 Figure 23 Figure 24 Figure 25 Figure 26 Figure 27 Figure 28 Figure 29 Figure 30 Figure 31 Figure 32 Figure 33 Full Text Cite Share Download PDF Status: Published Journal Publication published 29 Jan, 2021 Read the published version in Journal of Big Data → Version 4 posted Editorial decision: Accept 13 Jan, 2021 Reviewer # 1 agreed at journal 06 Jan, 2021 Review # 1 received at journal 06 Jan, 2021 Editor assigned by journal 05 Jan, 2021 Reviewers invited by journal 05 Jan, 2021 Submission checks completed at journal 05 Jan, 2021 Editor invited by journal 05 Jan, 2021 You are reading this latest preprint version Show more versions Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-45616","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Survey paper","associatedPublications":[],"authors":[{"id":7762865,"identity":"ef13a00e-b2dd-4e3a-94d7-114bb6eb8eb7","order_by":0,"name":"Vignesh Sampath","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2UlEQVRIiWNgGAWjYFACxgYDBgMJOX4QO6GAKC3MDQUMFRbGkg0gLQZEaWFv+MBwpiLR4ACIQ4wW/mkHGzfdbJNIMD6/OvHDAwMGeX6xA/i1SNxObDbObZPIM7vxdrME0GGGM2cnELDmdmIbSEux2Y2zG0BaEgxuE9Aifzux/TdQS+LmGWc3/yBKi8HtxAbjnDMSiRv4e7cRZ4shWEuFhLHEDd5tFgkGEoT9Inc7/YFxjkGdHH//2c03f1TYyPNLE9CCABJglRLEKgcB/gOkqB4Fo2AUjIKRBABdxkhsS7OgPwAAAABJRU5ErkJggg==","orcid":"","institution":"Universidad de Zaragoza Escuela de Ingenieria y Arquitectura","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Vignesh","middleName":"","lastName":"Sampath","suffix":""},{"id":7762866,"identity":"3cdfc03b-ac6e-431f-9b9b-1c90c8cc5de2","order_by":1,"name":"Iñaki Maurtua","email":"","orcid":"","institution":"Tekniker","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Iñaki","middleName":"","lastName":"Maurtua","suffix":""},{"id":7762867,"identity":"8203a72c-ac9c-4aec-947a-73b2d97410fb","order_by":2,"name":"Juan José Aguilar Martín","email":"","orcid":"","institution":"Universidad de Zaragoza Escuela de Ingenieria y Arquitectura","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Juan","middleName":"José Aguilar","lastName":"Martín","suffix":""},{"id":7762868,"identity":"125a8f79-a129-40a6-84f5-f3d778fd651c","order_by":3,"name":"Aitor Gutierrez","email":"","orcid":"","institution":"Tekniker","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Aitor","middleName":"","lastName":"Gutierrez","suffix":""}],"badges":[],"createdAt":"2020-07-19 10:34:07","currentVersionCode":4,"declarations":"","doi":"10.21203/rs.3.rs-45616/v4","doiUrl":"https://doi.org/10.21203/rs.3.rs-45616/v4","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s40537-021-00414-0","type":"published","date":"2021-01-29T15:02:28+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":4993205,"identity":"b4e07d1e-8400-4bc3-95f0-6cc7a34b4782","added_by":"auto","created_at":"2021-01-15 14:55:04","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":72541,"visible":true,"origin":"","legend":"Distribution of different type of datasets (a) Dataset with adequate sample (b) Dataset with inadequate sample","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/a2e2fdfffd3f6d894baea824.jpg"},{"id":4993203,"identity":"13d8939f-7bd4-406f-a66f-2b302e156491","added_by":"auto","created_at":"2021-01-15 14:55:04","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":66396,"visible":true,"origin":"","legend":"Proposed taxonomy for the review of imbalanced problem in computer vision tasks","description":"","filename":"Figure2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/a95e8d649947ac0432a11809.jpg"},{"id":4993204,"identity":"4b2445d9-e62e-414b-97f8-fb6a9f09d791","added_by":"auto","created_at":"2021-01-15 14:55:04","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":25442,"visible":true,"origin":"","legend":"Autoregressive models train a network that models conditional distribution of each pixel given all previous pixels. The image is processed pixel-by-pixel in (a) Raster scan order and (b) Sequentially predicts pixels.","description":"","filename":"Figure3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/7ab1fd63806eb47fed620ecf.jpg"},{"id":4993207,"identity":"d37a59bc-924a-47c6-8aa3-2f6bbd225e67","added_by":"auto","created_at":"2021-01-15 14:55:04","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":91394,"visible":true,"origin":"","legend":"An illustration of Masked Autoencoder Density Estimator (MADE)[99]. A set of connections in an autoencoder is removed using multiplicative binary masks, such that each output unit is connected only to relevant input units. ","description":"","filename":"Figure4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/07e4750de01952fc3cc8ba90.jpg"},{"id":4993265,"identity":"db84ff43-52bb-4bc1-bebf-cbc6d832c3a2","added_by":"auto","created_at":"2021-01-15 14:58:05","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":86429,"visible":true,"origin":"","legend":"The architecture of (a) Autoencoders; (b) Variational Auto Encoders; (c) Conditional Variational Auto Encoders ","description":"","filename":"Figure5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/2eee06d6420968cbd33981d5.jpg"},{"id":4993157,"identity":"15b832d0-2933-4d90-aef2-713eb662b225","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":31677,"visible":true,"origin":"","legend":"An illustration of the importance of an adversarial loss [111].","description":"","filename":"Figure6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/dd830374ea07cd5c21e39c2f.jpg"},{"id":4993209,"identity":"e7b023c1-d1f4-4513-a2bf-73471f186148","added_by":"auto","created_at":"2021-01-15 14:55:05","extension":"jpg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":141992,"visible":true,"origin":"","legend":"Comparative summary of Deep generative models discussed in section 2","description":"","filename":"Figure7.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/a5b716c0017149574d490508.jpg"},{"id":4993148,"identity":"3edd0169-5f56-45de-866f-f12a7d3b02a5","added_by":"auto","created_at":"2021-01-15 14:52:04","extension":"jpg","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":134367,"visible":true,"origin":"","legend":"A schematic view of (a) the vanilla GAN and (b)-(f) variants of Conditional GANs","description":"","filename":"Figure8.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/a5818c1fc66a9231bc0a4c99.jpg"},{"id":4993264,"identity":"194d23af-4c91-4629-8d04-5d039df35d5c","added_by":"auto","created_at":"2021-01-15 14:58:05","extension":"jpg","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":80906,"visible":true,"origin":"","legend":"A schematic view of Variants of GANs with multiple discriminators and generators: (a) LAPGAN, (b) MADGAN and (c) D2GAN","description":"","filename":"Figure9.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/713fa61b9ec91cd009380db9.jpg"},{"id":4993164,"identity":"37643319-ca39-409b-902c-e3bf45677129","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":95562,"visible":true,"origin":"","legend":"A schematic view of Variants of GANs based on Encoder and decoder architecture: (a) AAE, (b) VAEGAN, (c) DEGAN and (d) BIGAN","description":"","filename":"Figure10.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/c7e1187dcafcc5ef48a046ac.jpg"},{"id":4993149,"identity":"7b0f5b4c-8317-4cd7-bda5-10f5b187fb8f","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":48903,"visible":true,"origin":"","legend":"flowchart of GAN-based oversampling technique.","description":"","filename":"Figure11.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/32ceef440b742d436c593eff.jpg"},{"id":4993266,"identity":"a7659cca-ae3d-4fbf-8681-44faf9c9231a","added_by":"auto","created_at":"2021-01-15 14:58:05","extension":"jpg","order_by":12,"title":"Figure 12","display":"","copyAsset":false,"role":"figure","size":43394,"visible":true,"origin":"","legend":"Architecture diagram of clustering based GAN for solving intra-class imbalance presented by N. Hase et al. [163]","description":"","filename":"Figure12.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/09c498498ae5f4f9cbd2683c.jpg"},{"id":4993211,"identity":"1c7c3fb8-0cb6-4e4b-a751-a69b616627e3","added_by":"auto","created_at":"2021-01-15 14:55:05","extension":"jpg","order_by":13,"title":"Figure 13","display":"","copyAsset":false,"role":"figure","size":92645,"visible":true,"origin":"","legend":"Imbalanced distribution of 40 binary face attributes (positive and negative) on CelebA dataset [169]","description":"","filename":"Figure13.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/2cee4020694439ec56890244.jpg"},{"id":4993170,"identity":"43a22d27-1398-42d6-8bd5-219b20fbfbbe","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":14,"title":"Figure 14","display":"","copyAsset":false,"role":"figure","size":104349,"visible":true,"origin":"","legend":"Face attribute editing examples created by AttGAN [64]","description":"","filename":"Figure14.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/a5120647af872f8527012b32.jpg"},{"id":4993155,"identity":"da035d35-f4df-4b60-9c4e-2c6e7c015847","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":15,"title":"Figure 15","display":"","copyAsset":false,"role":"figure","size":43470,"visible":true,"origin":"","legend":"illustration of invertible conditional GAN presented by Perarnau et al. [65]","description":"","filename":"Figure15.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/2922ccf98db629396b46c729.jpg"},{"id":4993147,"identity":"6749fdaf-6669-4f05-90a1-960ae6b2f26f","added_by":"auto","created_at":"2021-01-15 14:52:04","extension":"jpg","order_by":16,"title":"Figure 16","display":"","copyAsset":false,"role":"figure","size":56088,"visible":true,"origin":"","legend":"Example of Person reidentification task. Person reidentification is a key element in video surveillance that deals with matching images of same person over many non-overlapping camera views.","description":"","filename":"Figure16.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/8fb16f33438bf5796e783bf9.jpg"},{"id":4993167,"identity":"266044fc-e48d-456c-b065-d564cf0c974f","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":17,"title":"Figure 17","display":"","copyAsset":false,"role":"figure","size":67812,"visible":true,"origin":"","legend":"Architecture diagram of pose-normalized GAN presented by Qian et al. [176]","description":"","filename":"Figure17.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/9d578e0fbfdf3ef15e7e69dd.jpg"},{"id":4993153,"identity":"f3ccead1-318b-4f9b-b844-9a30c0a9e57d","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":18,"title":"Figure 18","display":"","copyAsset":false,"role":"figure","size":36951,"visible":true,"origin":"","legend":"Illustration of challenges in vehicle Re-identification provided by Zheng et al [181]","description":"","filename":"Figure18.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/2483307a4968a81855f8845f.jpg"},{"id":4993141,"identity":"79a7c5a3-a1f3-4d63-9de7-7f5f9c800942","added_by":"auto","created_at":"2021-01-15 14:52:04","extension":"jpg","order_by":19,"title":"Figure 19","display":"","copyAsset":false,"role":"figure","size":61745,"visible":true,"origin":"","legend":"Sample images from the Stanford Dogs dataset [186] and the Caltech-UCSD Birds dataset [187], which exhibits minor inter-class variations and major intra-class variations.","description":"","filename":"Figure19.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/3905a16002ffb0dcc5176d64.jpg"},{"id":4993213,"identity":"c21388a4-b700-4826-b3d2-a69fe4aad7a9","added_by":"auto","created_at":"2021-01-15 14:55:05","extension":"jpg","order_by":20,"title":"Figure 20","display":"","copyAsset":false,"role":"figure","size":51563,"visible":true,"origin":"","legend":"Complete fine-grained Plankton classifier architecture used by C. Wang et al. [188]","description":"","filename":"Figure20.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/b995120f9d35e4e378f2ca47.jpg"},{"id":4993151,"identity":"433b2b3d-37b9-4cde-8de8-f1a5d33664ff","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":21,"title":"Figure 21","display":"","copyAsset":false,"role":"figure","size":100592,"visible":true,"origin":"","legend":"On emotion classification task [197], the images on the left are original data and the rest are images generated by cycle-GAN. ","description":"","filename":"Figure21.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/0d3c990e345f8ffa520d30e9.jpg"},{"id":4993172,"identity":"837b2d43-3833-4e4d-9b9f-2f172403f221","added_by":"auto","created_at":"2021-01-15 14:52:07","extension":"jpg","order_by":22,"title":"Figure 22","display":"","copyAsset":false,"role":"figure","size":74577,"visible":true,"origin":"","legend":"Illustration from Y. Huang et al. [199] showing how the Ensemble learning is integrated with GAN Framework.","description":"","filename":"Figure22.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/bbadbf8814511e4098a20e1a.jpg"},{"id":4993166,"identity":"339464b6-187d-4c6f-b423-117dc3bc2e86","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":23,"title":"Figure 23","display":"","copyAsset":false,"role":"figure","size":39749,"visible":true,"origin":"","legend":"(a) Distribution of the seven skin lesion class labels of the ISIC 2018 dataset [202]. (b) Sample images from each class.","description":"","filename":"Figure23.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/27b09b3d4bee258466719180.jpg"},{"id":4993158,"identity":"10d8dc39-236e-4b82-8383-e12ac1d0a0c2","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":24,"title":"Figure 24","display":"","copyAsset":false,"role":"figure","size":54014,"visible":true,"origin":"","legend":"The distributions of (a) Chest X-ray image dataset [193] and (b) tomato plant disease dataset [192], before (left) and after class balancing using GANs (right).","description":"","filename":"Figure24.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/9d369cd45e236ad1d6d70840.jpg"},{"id":4993262,"identity":"42fd9803-9d85-4a92-9c9f-50a1be6c8eea","added_by":"auto","created_at":"2021-01-15 14:58:04","extension":"jpg","order_by":25,"title":"Figure 25","display":"","copyAsset":false,"role":"figure","size":91947,"visible":true,"origin":"","legend":"Illustration of the class expert generative adversarial network architecture [206]","description":"","filename":"Figure25.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/5701f51750be231928443c2a.jpg"},{"id":4993263,"identity":"ac914666-1c94-4469-834e-bc3abd51f523","added_by":"auto","created_at":"2021-01-15 14:58:04","extension":"jpg","order_by":26,"title":"Figure 26","display":"","copyAsset":false,"role":"figure","size":53164,"visible":true,"origin":"","legend":"Example of scale variation and the scale (object size) distribution of the VisDrone2019 dataset objects in pixels [208].","description":"","filename":"Figure26.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/0cb4b8183ca0b5cea8630d52.jpg"},{"id":4993169,"identity":"5dbc90eb-2d21-4631-94ff-dc90eb74fd9f","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":27,"title":"Figure 27","display":"","copyAsset":false,"role":"figure","size":93122,"visible":true,"origin":"","legend":"Architecture diagram of (a) SOD-MTGAN [214] (b) Perceptual GAN [209] and (c) Detector GAN [210]. ","description":"","filename":"Figure27.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/e2c56f1b61e929948f03a83a.jpg"},{"id":4993215,"identity":"fad3306f-3daa-4bb5-b6f4-715019c7acff","added_by":"auto","created_at":"2021-01-15 14:55:07","extension":"jpg","order_by":28,"title":"Figure 28","display":"","copyAsset":false,"role":"figure","size":52962,"visible":true,"origin":"","legend":"Imbalanced distribution of occluded, partially occluded and heavily occluded objects in VisDrone-DET2018 dataset [216]","description":"","filename":"Figure28.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/18ebbb308b9a3dfcb75aefa1.jpg"},{"id":4993161,"identity":"ebe45495-50d7-4300-95f1-d3de87b4343e","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":29,"title":"Figure 29","display":"","copyAsset":false,"role":"figure","size":60313,"visible":true,"origin":"","legend":"Illustration of real world occlusions and deformations provided by Wang et al. [217]","description":"","filename":"Figure29.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/0f55eadb9fb4787885410c28.jpg"},{"id":4993154,"identity":"13d96137-dafe-4bb1-b005-a98ecacd8522","added_by":"auto","created_at":"2021-01-15 14:52:05","extension":"jpg","order_by":30,"title":"Figure 30","display":"","copyAsset":false,"role":"figure","size":44492,"visible":true,"origin":"","legend":"Architecture diagram to generate hard examples for training [217]","description":"","filename":"Figure30.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/f8d67862d68469c005aa4d7a.jpg"},{"id":4993162,"identity":"f4873ebc-57dc-4eaf-b26c-7a6eff595c7a","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":31,"title":"Figure 31","display":"","copyAsset":false,"role":"figure","size":44511,"visible":true,"origin":"","legend":"Pipeline of task aware image synthesis used by Tripathi et.al [220]","description":"","filename":"Figure31.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/70d25421853504d8d44c134a.jpg"},{"id":4993168,"identity":"9556621d-3ec3-4c1f-8fed-d89a3b25659f","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":32,"title":"Figure 32","display":"","copyAsset":false,"role":"figure","size":46069,"visible":true,"origin":"","legend":"Illustration of Sensitivity conditional GAN[224]","description":"","filename":"Figure32.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/e82c7774fe320aefd2120270.jpg"},{"id":4993165,"identity":"65b07fd6-052d-4689-9b84-ccca7ed9fa74","added_by":"auto","created_at":"2021-01-15 14:52:06","extension":"jpg","order_by":33,"title":"Figure 33","display":"","copyAsset":false,"role":"figure","size":65039,"visible":true,"origin":"","legend":"Illustration of SeGAN [232] (left) and Occlusion-Aware GAN [233] (right)","description":"","filename":"Figure33.jpg","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4/e231506bea100b99e7b1c8f8.jpg"},{"id":13575903,"identity":"2d80c6d4-59e4-4250-9f38-ccd78a51e24c","added_by":"auto","created_at":"2021-09-17 04:04:02","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4506784,"visible":true,"origin":"","legend":"","description":"","filename":"ManuscriptRevisedV6.pdf","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4_covered.pdf"},{"id":4993342,"identity":"a286ae2c-07f6-4f0a-88fc-18ab7a6d8532","added_by":"auto","created_at":"2021-01-15 15:01:08","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3522770,"visible":true,"origin":"","legend":"","description":"","filename":"ManuscriptRevisedV6.pdf","url":"https://assets-eu.researchsquare.com/files/rs-45616/v4_stamped.pdf"}],"financialInterests":"","formattedTitle":"A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks","fulltext":[{"header":"Full Text","content":"\u003cp\u003eThis preprint is available for \u003ca href='/article/rs-45616/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Generative Adversarial Neural Networks, Imbalanced data, Object detection, Segmentation, Classification, Deep learning, Deep generative model","lastPublishedDoi":"10.21203/rs.3.rs-45616/v4","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-45616/v4","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAny computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.\u003c/p\u003e\u003cp\u003eIn this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.\u003c/p\u003e","manuscriptTitle":"A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks","msid":"","msnumber":"","nonDraftVersions":[{"code":4,"date":"2021-01-15 14:52:02","doi":"10.21203/rs.3.rs-45616/v4","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Accept","date":"2021-01-14T00:00:00+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2021-01-07T00:00:00+00:00","index":1,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2021-01-07T00:00:00+00:00","index":1,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorAssigned","content":"","date":"2021-01-06T00:00:00+00:00","index":"","fulltext":""},{"type":"reviewersInvited","content":"","date":"2021-01-06T00:00:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2021-01-05T23:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2021-01-05T23:00:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}},{"code":"","date":"2020-12-28 00:00:00","doi":"","editorialEvents":[{"type":"reviewerAgreed","content":"","date":"2020-12-28T00:00:00+00:00","index":1,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-12-28T00:00:00+00:00","index":1,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"reviewersInvited","content":"","date":"2020-12-27T00:00:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2020-12-26T00:00:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2020-12-25T23:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2020-12-25T23:00:00+00:00","index":"","fulltext":""},{"type":"notPreprinted","content":""}],"status":"timeline","journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}},{"code":3,"date":"2020-12-02 19:58:42","doi":"10.21203/rs.3.rs-45616/v3","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Minor Revision","date":"2020-11-26T00:00:00+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-11-22T00:00:00+00:00","index":1,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"reviewerAgreed","content":"","date":"2020-11-20T00:00:00+00:00","index":1,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2020-11-19T00:00:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2020-11-15T00:00:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2020-11-14T23:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2020-11-14T23:00:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}},{"code":2,"date":"2020-10-24 02:50:00","doi":"10.21203/rs.3.rs-45616/v2","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Minor Revision","date":"2020-10-30T00:00:00+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-10-26T12:00:00+00:00","index":4,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-10-26T12:00:00+00:00","index":4,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-10-26T00:00:00+00:00","index":3,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-10-24T12:00:00+00:00","index":1,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-10-24T12:00:00+00:00","index":2,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"reviewerAgreed","content":"","date":"2020-10-23T12:00:00+00:00","index":1,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-10-23T12:00:00+00:00","index":2,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-10-23T12:00:00+00:00","index":3,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2020-10-22T12:00:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2020-10-19T12:00:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2020-10-18T12:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2020-10-18T12:00:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}},{"code":1,"date":"2020-08-17 20:42:30","doi":"10.21203/rs.3.rs-45616/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major Revision","date":"2020-09-12T12:00:00+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-09-10T12:00:00+00:00","index":2,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-09-08T12:00:00+00:00","index":3,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-09-08T12:00:00+00:00","index":1,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"reviewerAgreed","content":"","date":"2020-09-07T12:00:00+00:00","index":12,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-08-31T12:00:00+00:00","index":11,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"reviewerAgreed","content":"","date":"2020-08-31T12:00:00+00:00","index":11,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-28T12:00:00+00:00","index":10,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":4,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":10,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":7,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":9,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":5,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":8,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"editorInvitedReview","content":"","date":"2020-08-28T12:00:00+00:00","index":6,"fulltext":"Recommendation: Reviewer's comments unavailable due to the journal's policy.\n"},{"type":"reviewerAgreed","content":"","date":"2020-08-28T12:00:00+00:00","index":9,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-28T12:00:00+00:00","index":8,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-28T12:00:00+00:00","index":7,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-28T12:00:00+00:00","index":6,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-28T12:00:00+00:00","index":5,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-27T12:00:00+00:00","index":4,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-27T12:00:00+00:00","index":3,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-27T12:00:00+00:00","index":2,"fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-08-27T12:00:00+00:00","index":1,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2020-08-27T12:00:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2020-08-14T12:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2020-08-13T12:00:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2020-08-13T12:00:00+00:00","index":"","fulltext":""},{"type":"submitted","content":"","date":"2020-07-30T12:00:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8c18f9de-b8ab-4c8b-b866-cb39e7ab1c02","owner":[],"postedDate":"January 15th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":1781855,"name":"Computer Architecture and Engineering"}],"tags":[],"updatedAt":"2021-01-31T15:02:44+00:00","versionOfRecord":{"articleIdentity":"rs-45616","link":"https://doi.org/10.1186/s40537-021-00414-0","journal":{"identity":"journal-of-big-data","isVorOnly":false,"title":"Journal of Big Data"},"publishedOn":"2021-01-29 15:02:28","publishedOnDateReadable":"January 29th, 2021"},"versionCreatedAt":"2021-01-15 14:52:02","video":"","vorDoi":"10.1186/s40537-021-00414-0","vorDoiUrl":"https://doi.org/10.1186/s40537-021-00414-0","workflowStages":[]},"version":"v4","identity":"rs-45616","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-45616","identity":"rs-45616","version":["v4"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}