Toward Simplicity in Dynamic Inference: A Critical Study and Redesign of Early-Exit Networks

Toward Simplicity in Dynamic Inference: A Critical Study and Redesign of Early-Exit Networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Toward Simplicity in Dynamic Inference: A Critical Study and Redesign of Early-Exit Networks Youva Addad, Alexis Lechervy, Frederic Jurie This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6608894/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 11 Oct, 2025 Read the published version in Machine Vision and Applications → Version 1 posted 12 You are reading this latest preprint version Abstract In recent years, dynamic early-exiting neural networks have emerged as an efficient solution for balancing classification performance and inference cost—typically measured in FLOPs—in image classification tasks. Given the significance of this domain, numerous extensions to early-exiting networks have been proposed over the past five years, focusing on improving this tradeoff. This paper critically analyzes these advancements and, through extensive experimentation, demonstrates that a simple yet carefully designed architecture—eschewing unnecessary complexities—can achieve results that surpass the current state-of-the-art models when paired with appropriate algorithmic strategies. We introduce SEEDNet (Simple Early-Exiting Dynamic Image Network), a streamlined early-exiting dynamic image network that leverages a combination of mechanisms to enhance both efficiency and efficacy. By integrating early-exiting strategies within a dynamic framework, SEEDNet achieves a top-1 accuracy of 81.22% on ImageNet while requiring only 2 × 109 FLOPs during inference. This demonstrates that the right existing mechanisms have been selected and integrated to obtain a more efficient architecture. To promote transparency and encourage further advancements in the field, we will release our code to the community. Deep Learning Computer Vision Efficient Neural Network Early-Exit Neural Network Knowledge distillation Efficient Architecture Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 11 Oct, 2025 Read the published version in Machine Vision and Applications → Version 1 posted Editorial decision: Revision requested 14 Jul, 2025 Reviews received at journal 13 Jul, 2025 Reviews received at journal 13 Jul, 2025 Reviews received at journal 07 Jul, 2025 Reviewers agreed at journal 07 Jul, 2025 Reviewers agreed at journal 07 Jul, 2025 Reviewers agreed at journal 07 Jul, 2025 Reviewers agreed at journal 26 Jun, 2025 Reviewers invited by journal 24 Jun, 2025 Editor assigned by journal 07 May, 2025 Submission checks completed at journal 07 May, 2025 First submitted to journal 07 May, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6608894","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":475605660,"identity":"0d69aeed-b7ca-4401-a6bd-9f58713ac8d2","order_by":0,"name":"Youva Addad","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA80lEQVRIiWNgGAWjYBADxoYDDIYPGBgkINwKIrUYG8C1nCFSi5kEnItPi8HxHjOJHwx2sn3HD2+r5t1hIc/f3nuA4eAePFrOnDGT7GFINp55Jq3sNu8ZCcMZZ84lMBx4hkfLjRxjAx4G5sQNB3LMbvO2SSQw3MgxYP5wAL8Wwz8M9Ykbzr8xKwZpkb//xoDhAH4tho95GA4nbriRY8YM0mJwgwe/FskzxwofyxgcN55541mx5Nw2CcONZ3IMDuDTwne8ecPBNxXVsn3nkzd+eNtWJy93/IzhA3xaFMByBmiieDQwMMg34JMdBaNgFIyCUQACAGktWrm0cXa7AAAAAElFTkSuQmCC","orcid":"","institution":"Université de Caen Normandie","correspondingAuthor":true,"prefix":"","firstName":"Youva","middleName":"","lastName":"Addad","suffix":""},{"id":475605661,"identity":"7208f8f3-85ac-4748-85af-d016efba6678","order_by":1,"name":"Alexis Lechervy","email":"","orcid":"","institution":"Université de Caen Normandie","correspondingAuthor":false,"prefix":"","firstName":"Alexis","middleName":"","lastName":"Lechervy","suffix":""},{"id":475605662,"identity":"02294928-032b-49bb-a551-84690ce5b93a","order_by":2,"name":"Frederic Jurie","email":"","orcid":"","institution":"Université de Caen Normandie","correspondingAuthor":false,"prefix":"","firstName":"Frederic","middleName":"","lastName":"Jurie","suffix":""}],"badges":[],"createdAt":"2025-05-07 06:53:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6608894/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6608894/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s00138-025-01748-y","type":"published","date":"2025-10-11T15:58:13+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":93421300,"identity":"a8ae84a9-7d62-4c36-b4e2-c492b944273f","added_by":"auto","created_at":"2025-10-13 16:10:53","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":687760,"visible":true,"origin":"","legend":"","description":"","filename":"TowardSimplicityinDynamicInferenceACriticalStudyandRedesignofEarlyExitNetworks.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6608894/v1_covered_daa1bbcc-bdd7-476c-ae0b-1ad1c788f2d7.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Toward Simplicity in Dynamic Inference: A Critical Study and Redesign of Early-Exit Networks","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"machine-vision-and-applications","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"mvap","sideBox":"Learn more about [Machine Vision and Applications](https://www.springer.com/journal/138)","snPcode":"138","submissionUrl":"https://submission.springernature.com/new-submission/138/3","title":"Machine Vision and Applications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Deep Learning, Computer Vision, Efficient Neural Network, Early-Exit Neural Network, Knowledge distillation, Efficient Architecture","lastPublishedDoi":"10.21203/rs.3.rs-6608894/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6608894/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn recent years, dynamic early-exiting neural networks have emerged as an efficient solution for balancing classification performance and inference cost—typically measured in FLOPs—in image classification tasks. Given the significance of this domain, numerous extensions to early-exiting networks have been proposed over the past five years, focusing on improving this tradeoff. This paper critically analyzes these advancements and, through extensive experimentation, demonstrates that a simple yet carefully designed architecture—eschewing unnecessary complexities—can achieve results that surpass the current state-of-the-art models when paired with appropriate algorithmic strategies. We introduce SEEDNet (Simple Early-Exiting Dynamic Image Network), a streamlined early-exiting dynamic image network that leverages a combination of mechanisms to enhance both efficiency and efficacy. By integrating early-exiting strategies within a dynamic framework, SEEDNet achieves a top-1 accuracy of 81.22% on ImageNet while requiring only 2 × 109 FLOPs during inference. This demonstrates that the right existing mechanisms have been selected and integrated to obtain a more efficient architecture. To promote transparency and encourage further advancements in the field, we will release our code to the community.\u003c/p\u003e","manuscriptTitle":"Toward Simplicity in Dynamic Inference: A Critical Study and Redesign of Early-Exit Networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-26 02:36:31","doi":"10.21203/rs.3.rs-6608894/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-07-14T14:31:06+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-14T03:57:21+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-13T13:31:42+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-08T01:48:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"240760991459415563950676827165996316132","date":"2025-07-08T01:29:48+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"128188828162489701955884563312834482696","date":"2025-07-07T13:43:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"167609341276733015261761515301401386481","date":"2025-07-07T13:23:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"177443112165788136999073931911577007770","date":"2025-06-26T07:15:45+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-06-24T05:05:25+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-05-07T10:58:17+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-05-07T10:58:13+00:00","index":"","fulltext":""},{"type":"submitted","content":"Machine Vision and Applications","date":"2025-05-07T06:48:34+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"machine-vision-and-applications","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"mvap","sideBox":"Learn more about [Machine Vision and Applications](https://www.springer.com/journal/138)","snPcode":"138","submissionUrl":"https://submission.springernature.com/new-submission/138/3","title":"Machine Vision and Applications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"03c744ed-7910-4764-9bb8-4310389717b8","owner":[],"postedDate":"June 26th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-10-13T16:09:25+00:00","versionOfRecord":{"articleIdentity":"rs-6608894","link":"https://doi.org/10.1007/s00138-025-01748-y","journal":{"identity":"machine-vision-and-applications","isVorOnly":false,"title":"Machine Vision and Applications"},"publishedOn":"2025-10-11 15:58:13","publishedOnDateReadable":"October 11th, 2025"},"versionCreatedAt":"2025-06-26 02:36:31","video":"","vorDoi":"10.1007/s00138-025-01748-y","vorDoiUrl":"https://doi.org/10.1007/s00138-025-01748-y","workflowStages":[]},"version":"v1","identity":"rs-6608894","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6608894","identity":"rs-6608894","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}