A lightweight optimization framework for real-time object detector on the embedded GPU platform

A lightweight optimization framework for real-time object detector on the embedded GPU platform | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A lightweight optimization framework for real-time object detector on the embedded GPU platform Jiawei Zhu, Haogang Feng, Shida Zhong, Zhi Quan, Tao Yuan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4537009/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract At present, embedded GPU devices play a central role in carrying edge intelligence applications, especially object detection. However, efficiently deploying and inferring convolutional neural network (CNN)-based object detection models on embedded edge GPU devices with small size, low power consumption, and limited computing resources remains a significant challenge. In this paper, we proposed a lightweight optimization framework for YOLOv3 to improve its inference efficiency onto a small GPU-based edge device, in this case the Jetson Nano, which is a low-cost entry-level artificial intelligence (AI) computer for the embedded and edge intelligence markets. First, we replace the Backbone of YOLOv3 by Efficient-RepVGG, which is more suitable for GPU architecture. Secondly, a lightweight and efficient bidirectional feature pyramid network is designed to further compress the model. Finally, the Optimized Couple Head is proposed to compensate for the loss of accuracy without affecting the effect of model lightweight. After the improvement, the model saves 92.26% and 92.10% of the parameters and computation, increasing the inference speed by 9.95 times, while maintaining high object detection accuracy. Compared with other lightweight object detectors, the inference speed of Optimized YOLOv3 is about 1.6 to 2 times faster while maintaining similar detection accuracy; and the detection accuracy is 3 to 6 percentage higher while maintaining similar inference speed. Edge intelligence Object detection Embedded GPU devices YOLOv3 Model lightweight Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4537009","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":316022264,"identity":"b7c32e2e-350e-416b-8799-abff0b9fbc23","order_by":0,"name":"Jiawei Zhu","email":"","orcid":"","institution":"College of Electronics and Information Engineering, Shenzhen University","correspondingAuthor":false,"prefix":"","firstName":"Jiawei","middleName":"","lastName":"Zhu","suffix":""},{"id":316022265,"identity":"59a42772-8735-406b-be0d-f6d7e7c9e044","order_by":1,"name":"Haogang Feng","email":"","orcid":"","institution":"College of Electronics and Information Engineering, Shenzhen University","correspondingAuthor":false,"prefix":"","firstName":"Haogang","middleName":"","lastName":"Feng","suffix":""},{"id":316022266,"identity":"b34d3979-fdb8-41fd-ade4-c6f13f5a1726","order_by":2,"name":"Shida Zhong","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVRIiWNgGAWjYDACCcYGCSCVAMSMDyBCCcRrYTYgUgsYgZWxSRClRX52c+ONHwx1efzS7dcqf/w5zMDPnmPA8HMHbi0Gdw42W/YwsBVLzjlTdpuH5zCDZM8bA8beM3i0SCS2SfAw8CRuuJGTdptB4jCDwY0cA2bGNjwOm5HYJvmHQQKspfCHwWEGe0JaGG4ktknzMBgAtaQfY+BJANoiQUCLwY3EZmsZg4TEmTNymKV5DqTzSJx5VnCwF6/D0h/efFNRl9gvkf7w448/1nL87ckbH/zE5zCIXSCCB0KCiAOENEAB+wMiFY6CUTAKRsFIAwCZ0E+BWKvIQQAAAABJRU5ErkJggg==","orcid":"","institution":"College of Electronics and Information Engineering, Shenzhen University","correspondingAuthor":true,"prefix":"","firstName":"Shida","middleName":"","lastName":"Zhong","suffix":""},{"id":316022267,"identity":"9e55a800-a84c-4d93-8153-fd4d5dcdce88","order_by":3,"name":"Zhi Quan","email":"","orcid":"","institution":"College of Electronics and Information Engineering, Shenzhen University","correspondingAuthor":false,"prefix":"","firstName":"Zhi","middleName":"","lastName":"Quan","suffix":""},{"id":316022268,"identity":"50426e4f-7f43-428b-8d2b-cc5d51de8a9d","order_by":4,"name":"Tao Yuan","email":"","orcid":"","institution":"State Key Laboratory of Radio Frequency Heterogeneous Integration (Shenzhen University)","correspondingAuthor":false,"prefix":"","firstName":"Tao","middleName":"","lastName":"Yuan","suffix":""}],"badges":[],"createdAt":"2024-06-06 03:24:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4537009/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4537009/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":69847975,"identity":"212a2d44-ecef-40ca-9edd-dfd338367824","added_by":"auto","created_at":"2024-11-25 21:01:54","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1982083,"visible":true,"origin":"","legend":"","description":"","filename":"AlightweightoptimizationframeworkforrealtimeobjectdetectorontheembeddedGPUplatform.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4537009/v1_covered_ab2a0f63-4504-452d-892b-de576f8f48d9.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A lightweight optimization framework for real-time object detector on the embedded GPU platform","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Edge intelligence, Object detection, Embedded GPU devices, YOLOv3, Model lightweight","lastPublishedDoi":"10.21203/rs.3.rs-4537009/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4537009/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"At present, embedded GPU devices play a central role in carrying edge intelligence applications, especially object detection. However, efficiently deploying and inferring convolutional neural network (CNN)-based object detection models on embedded edge GPU devices with small size, low power consumption, and limited computing resources remains a significant challenge. In this paper, we proposed a lightweight optimization framework for YOLOv3 to improve its inference efficiency onto a small GPU-based edge device, in this case the Jetson Nano, which is a low-cost entry-level artificial intelligence (AI) computer for the embedded and edge intelligence markets. First, we replace the Backbone of YOLOv3 by Efficient-RepVGG, which is more suitable for GPU architecture. Secondly, a lightweight and efficient bidirectional feature pyramid network is designed to further compress the model. Finally, the Optimized Couple Head is proposed to compensate for the loss of accuracy without affecting the effect of model lightweight. After the improvement, the model saves 92.26% and 92.10% of the parameters and computation, increasing the inference speed by 9.95 times, while maintaining high object detection accuracy. Compared with other lightweight object detectors, the inference speed of Optimized YOLOv3 is about 1.6 to 2 times faster while maintaining similar detection accuracy; and the detection accuracy is 3 to 6 percentage higher while maintaining similar inference speed.","manuscriptTitle":"A lightweight optimization framework for real-time object detector on the embedded GPU platform","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-20 02:52:59","doi":"10.21203/rs.3.rs-4537009/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d85389c8-dce2-425e-afff-41eab0dd33df","owner":[],"postedDate":"June 20th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-11-25T20:53:37+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-20 02:52:59","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4537009","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4537009","identity":"rs-4537009","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}