The Energy Efficiency Paradox: Lightweight CNNs Consume More Power than ResNets on Consumer GPUs

The Energy Efficiency Paradox: Lightweight CNNs Consume More Power than ResNets on Consumer GPUs | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article The Energy Efficiency Paradox: Lightweight CNNs Consume More Power than ResNets on Consumer GPUs Someyo Kamal Utsho This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9384241/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Deploying deep neural networks in energyconstrained environments requires that inference optimisation strategies — runtime backends and reduced numerical precision — reliably deliver their promised gains on the target hardware. Yet how consistently these gains transfer across GPU tiers remains poorly characterised, and practitioners routinely apply optimisations benchmarked on high-end hardware to lower-tier devices without accounting for architectural differences. This paper presents a systematic, energy-aware benchmarking study of 15 image classification architectures evaluated across four runtime configurations on two consumer-grade GPUs: an NVIDIA GeForce GTX 1650 (Turing TU117, no Tensor Cores) and an NVIDIA GeForce RTX 3060 (Ampere GA106, Tensor Core-equipped). Each configuration is measured over ten independent runs at batch sizes 1, 8, and 32, recording perinference latency, throughput, and energy consumption. Runtime configurations include native PyTorch FP32, ONNX Runtime with CUDA Execution Provider (ORT-CUDA FP32), and FP16 variants in both PyTorch and ORT; all pairwise runtime differences are assessed viaWilcoxon signed-rank tests with Bonferroni correction. We also evaluate INT8 quantization (CPU) with accuracy agreement checks and roofline analysis to explain the observed energy paradox. Results show that ORT-CUDA consistently outperforms native PyTorch across both platforms, with a mean speedup of 2.01× on the GTX 1650 at batch size 1. The benefit is stratified by architecture family: lightweight depthwise-separable CNNs gain 2.79 ± 0.35×, standard CNNs 1.16 ± 0.18×, and LayerNormbased models 1.05±0.29×. FP16 behaviour, however, is strongly hardware-dependent. On the GTX 1650, which lacks Tensor Cores, FP16 systematically regresses latency across all 15 models: lightweight CNNs slow by a mean of 16%, standard CNNs by 115%, and LayerNorm-based architectures by 188% (up to 279% for ViT-B/16). A numerical stability failure renders EfficientNet-B3 unusable at batch size 32. On the RTX 3060, Tensor Core acceleration yields FP16 speedups only for compute-intensive models at moderate batch sizes (mean 1.95× at batch 32), while at batch size 1 only attention-based architectures benefit modestly (4–5%). Additionally, FP16 ONNX models for ViT-B/16 and Swin-T fail ONNX Runtime’s type checks on both GPUs due to a toolchain limitation in transformer attention layers, highlighting a separate portability constraint. Cross-platform, the GTX 1650 outperforms the RTX 3060 under PyTorch FP32 for 12 of 15 models at batch size 1 (FPS ratios 1.59–1.84×), revealing a batch-1 throughput paradox. INT8 quantization on CPU achieves 54–98% energy savings for lightweight models with 98–100% accuracy agreement. Roofline analysis confirms that memory bound lightweights (arithmetic intensity ¡35 FLOP/byte on RTX) are responsible for the energy paradox, explaining why models with fewer FLOPs can consume more energy than computebound ResNets. Together, these findings expose a hardware- and toolchaindependent portability gap that is invisible to single-platform benchmarks and carry direct implications for hardware-aware model selection in energy-sensitive deployment scenarios. All experimental scripts, raw results, and intermediate data are made publicly available to support reproducibility. Index Terms—Energy efficiency, deep learning Artificial Intelligence and Machine Learning Computer Architecture and Engineering Energy efficiency deep learning CNN GPU benchmark edge computing sustainable AI Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9384241","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":621258989,"identity":"006fd177-a642-44f4-aa70-88745d074f67","order_by":0,"name":"Someyo Kamal Utsho","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIiWNgGAWjYHACNgaGChCdQLwGoJYzUC0HgBQPUVoY20jRYnC//dmDj/Nq8/nbcww/f6hgSNwvdoB1Mz59Bsd4zA1nbjtuOePMG2OJA2cYEnukE9huE9DCJs277ZgBw40cA4mDbURpYX8mzTvnmIH8jRzjHwf/EaWFwUyat6HGwOBGjpnEwQYitEgeyzGTnHHsgIHhmWdlFmeOSRj33E5suzkHjxa+w8efSXyoqTOQO568+UZFjY1s++zkYzfe4NECBYdhDAkgZmxgIhyfDHWoXMYfhLWMglEwCkbByAEAPyFQ6hBKyoMAAAAASUVORK5CYII=","orcid":"https://orcid.org/0009-0006-0869-2031","institution":"University of technology malaysia","correspondingAuthor":true,"prefix":"","firstName":"Someyo","middleName":"Kamal","lastName":"Utsho","suffix":""}],"badges":[],"createdAt":"2026-04-11 03:51:07","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":true,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":true},"doi":"10.21203/rs.3.rs-9384241/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9384241/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106960890,"identity":"f1455d4c-48d6-4012-af60-8d17a5d9ae03","added_by":"auto","created_at":"2026-04-15 09:23:31","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":323194,"visible":true,"origin":"","legend":"","description":"","filename":"TheUAVpaper5.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9384241/v1_covered_40c5c6f6-d808-4b50-862a-1bd139820aa9.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eThe Energy Efficiency Paradox: Lightweight CNNs Consume More Power than ResNets on Consumer GPUs\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of Technology Malaysia","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Energy efficiency, deep learning, CNN, GPU benchmark, edge computing, sustainable AI","lastPublishedDoi":"10.21203/rs.3.rs-9384241/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9384241/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDeploying deep neural networks in energyconstrained environments requires that inference optimisation strategies — runtime backends and reduced numerical precision — reliably deliver their promised gains on the target hardware. Yet how consistently these gains transfer across GPU tiers remains poorly characterised, and practitioners routinely apply optimisations benchmarked on high-end hardware to lower-tier devices without accounting for architectural differences.\u003c/p\u003e\n\u003cp\u003eThis paper presents a systematic, energy-aware benchmarking study of 15 image classification architectures evaluated across four runtime configurations on two consumer-grade GPUs: an NVIDIA GeForce GTX 1650 (Turing TU117, no Tensor Cores) and an NVIDIA GeForce RTX 3060 (Ampere GA106, Tensor Core-equipped). Each configuration is measured over ten independent runs at batch sizes 1, 8, and 32, recording perinference latency, throughput, and energy consumption. Runtime configurations include native PyTorch FP32, ONNX Runtime with CUDA Execution Provider (ORT-CUDA FP32), and FP16 variants in both PyTorch and ORT; all pairwise runtime differences are assessed viaWilcoxon signed-rank tests with Bonferroni correction. We also evaluate INT8 quantization (CPU) with accuracy agreement checks and roofline analysis to explain the observed energy paradox.\u003c/p\u003e\n\u003cp\u003eResults show that ORT-CUDA consistently outperforms native PyTorch across both platforms, with a mean speedup of 2.01× on the GTX 1650 at batch size 1. The benefit is stratified by architecture family: lightweight depthwise-separable CNNs gain 2.79 ± 0.35×, standard CNNs 1.16 ± 0.18×, and LayerNormbased models 1.05±0.29×. FP16 behaviour, however, is strongly hardware-dependent. On the GTX 1650, which lacks Tensor Cores, FP16 systematically regresses latency across all 15 models: lightweight CNNs slow by a mean of 16%, standard CNNs by 115%, and LayerNorm-based architectures by 188% (up to 279% for ViT-B/16). A numerical stability failure renders EfficientNet-B3 unusable at batch size 32. On the RTX 3060, Tensor Core acceleration yields FP16 speedups only for compute-intensive models at moderate batch sizes (mean 1.95× at batch 32), while at batch size 1 only attention-based architectures benefit modestly (4–5%). Additionally, FP16 ONNX models for ViT-B/16 and Swin-T fail ONNX Runtime’s type checks on both GPUs due to a toolchain limitation in transformer attention layers, highlighting a separate portability constraint. Cross-platform, the GTX 1650 outperforms the RTX 3060 under PyTorch FP32 for 12 of 15 models at batch size 1 (FPS ratios 1.59–1.84×), revealing a batch-1 throughput paradox. INT8 quantization on CPU achieves 54–98% energy savings for lightweight models with 98–100% accuracy agreement. Roofline analysis confirms that memory bound lightweights (arithmetic intensity ¡35 FLOP/byte on RTX) are responsible for the energy paradox, explaining why models with fewer FLOPs can consume more energy than computebound ResNets.\u003c/p\u003e\n\u003cp\u003eTogether, these findings expose a hardware- and toolchaindependent portability gap that is invisible to single-platform benchmarks and carry direct implications for hardware-aware model selection in energy-sensitive deployment scenarios. All experimental scripts, raw results, and intermediate data are made publicly available to support reproducibility. Index Terms—Energy efficiency, deep learning\u003c/p\u003e","manuscriptTitle":"The Energy Efficiency Paradox: Lightweight CNNs Consume More Power than ResNets on Consumer GPUs","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-14 04:45:41","doi":"10.21203/rs.3.rs-9384241/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"20855f21-154f-4eb1-8f69-f38783793fe4","owner":[],"postedDate":"April 14th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":66261904,"name":"Artificial Intelligence and Machine Learning"},{"id":66261905,"name":"Computer Architecture and Engineering"}],"tags":[],"updatedAt":"2026-04-14T04:45:41+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-14 04:45:41","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9384241","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9384241","identity":"rs-9384241","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}