Quantum-Enhanced Hybrid-Model Compressionusing Knowledge Distillation

Quantum-Enhanced Hybrid-Model Compressionusing Knowledge Distillation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Quantum-Enhanced Hybrid-Model Compressionusing Knowledge Distillation Luigi Barbato, Massimo Esposito, Francesco Gargiulo This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8919757/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Quantum computing has emerged as a promising paradigm capable of addressing computational tasks that are intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore vast solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while maintaining the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via knowledge distillation with feature space alignment, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller student model. We introduce a hybrid quantum-classical student architecture that incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, using a multi-Pauli feature map followed by an EfficientSU2 ansatz to embed and process classical features in a high-dimensional Hilbert space. The hybrid student is directly compared to a purely classical student of comparable size to evaluate the representational benefits of the quantum component. We describe the knowledge-distillation framework and examine alternative gradient-estimation techniques for optimizing the quantum circuit, analyzing their influence on performance and convergence. The results indicate that the hybrid quantum student achieves superior parameter efficiency and expressive capacity compared to a classical counterpart of similar scale. While the hybrid model demonstrates improved compression capabilities and competitive accuracy, it incurs higher computational overhead and exhibits increased training instability relative to purely classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and highlight directions for future work aimed at reducing computational costs and improving robustness. Overall, our findings suggest that parameterized quantum circuits can serve as compact and expressive components within neural compression schemes, providing a promising avenue for further exploration as quantum hardware and optimization techniques mature. Quantum Machine Learning Knowledge Distillation Model Compression Hybrid Quantum-Classical Models Quantum Neural Networks Qiskit Parameter-Shift Rule Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 10 May, 2026 Reviewers invited by journal 27 Feb, 2026 Editor assigned by journal 20 Feb, 2026 Submission checks completed at journal 19 Feb, 2026 First submitted to journal 19 Feb, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8919757","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":598292809,"identity":"9b74565a-aeee-4b0a-8172-51a778cb3221","order_by":0,"name":"Luigi Barbato","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2UlEQVRIiWNgGAWjYBACPgbGBobEBgYGAxDvA4hgJqCFDaGFmYFxBlgLAT1sIIIRqoWZh4EIa9gkkptfPNxhl2/Ofv7gY5uaO/kM7PwHCGhJbLNIPJNsubMnmdk459gzywaCDgNqMUhsYzYwOJDMJp3bcNiAsF8gWuoNDM4/ZpO2JFJL84PEtsMGBjeAtjASpYXnYRtD4pnjBpYzHhsb9hw7bMDGzGyAVws/e/rjjz93VBuY8yc+fPCj5rABP//BB/itAbkNlUtIPRAwfyBC0SgYBaNgFIxkAADq3T6EF4s+QAAAAABJRU5ErkJggg==","orcid":"","institution":"Università Campus Bio-Medico","correspondingAuthor":true,"prefix":"","firstName":"Luigi","middleName":"","lastName":"Barbato","suffix":""},{"id":598292810,"identity":"a2596343-32d8-4659-82d5-98dcd23406c9","order_by":1,"name":"Massimo Esposito","email":"","orcid":"","institution":"National Research Council","correspondingAuthor":false,"prefix":"","firstName":"Massimo","middleName":"","lastName":"Esposito","suffix":""},{"id":598292811,"identity":"0103984f-338d-46c4-8dd2-1035067cc263","order_by":2,"name":"Francesco Gargiulo","email":"","orcid":"","institution":"National Research Council","correspondingAuthor":false,"prefix":"","firstName":"Francesco","middleName":"","lastName":"Gargiulo","suffix":""}],"badges":[],"createdAt":"2026-02-19 17:23:57","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8919757/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8919757/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":103889238,"identity":"dba58ed3-6723-4f3b-bed1-f6811f4679db","added_by":"auto","created_at":"2026-03-04 07:48:33","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1286216,"visible":true,"origin":"","legend":"","description":"","filename":"PaperBarbatoeall.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8919757/v1_covered_cdfe9175-9998-4aee-a883-2e2f3188182a.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Quantum-Enhanced Hybrid-Model Compressionusing Knowledge Distillation","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"quantum-machine-intelligence","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"qumi","sideBox":"Learn more about [Quantum Machine Intelligence](http://link.springer.com/journal/42484)","snPcode":"42484","submissionUrl":"https://submission.nature.com/new-submission/42484/3","title":"Quantum Machine Intelligence","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Quantum Machine Learning, Knowledge Distillation, Model Compression, Hybrid Quantum-Classical Models, Quantum Neural Networks, Qiskit, Parameter-Shift Rule","lastPublishedDoi":"10.21203/rs.3.rs-8919757/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8919757/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Quantum computing has emerged as a promising paradigm capable of addressing computational tasks that are intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore vast solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while maintaining the practicality of classical machine learning frameworks.\nThis paper presents a study on quantum-enhanced model compression via knowledge distillation with feature space alignment, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller student model. We introduce a hybrid quantum-classical student architecture that incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, using a multi-Pauli feature map followed by an EfficientSU2 ansatz to embed and process classical features in a high-dimensional Hilbert space. The hybrid student is directly compared to a purely classical student of comparable size to evaluate the representational benefits of the quantum component.\nWe describe the knowledge-distillation framework and examine alternative gradient-estimation techniques for optimizing the quantum circuit, analyzing their influence on performance and convergence. The results indicate that the hybrid quantum student achieves superior parameter efficiency and expressive capacity compared to a classical counterpart of similar scale.\nWhile the hybrid model demonstrates improved compression capabilities and competitive accuracy, it incurs higher computational overhead and exhibits increased training instability relative to purely classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and highlight directions for future work aimed at reducing computational costs and improving robustness. Overall, our findings suggest that parameterized quantum circuits can serve as compact and expressive components within neural compression schemes, providing a promising avenue for further exploration as quantum hardware and optimization techniques mature.","manuscriptTitle":"Quantum-Enhanced Hybrid-Model Compressionusing Knowledge Distillation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-04 07:48:22","doi":"10.21203/rs.3.rs-8919757/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"143980467215058105174810254961758211290","date":"2026-05-11T02:53:32+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-02-27T16:42:45+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-02-20T11:29:43+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-02-20T00:01:11+00:00","index":"","fulltext":""},{"type":"submitted","content":"Quantum Machine Intelligence","date":"2026-02-19T17:20:55+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"quantum-machine-intelligence","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"qumi","sideBox":"Learn more about [Quantum Machine Intelligence](http://link.springer.com/journal/42484)","snPcode":"42484","submissionUrl":"https://submission.nature.com/new-submission/42484/3","title":"Quantum Machine Intelligence","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"dbe2b28f-1850-44a4-80b3-ea7994ed216c","owner":[],"postedDate":"March 4th, 2026","published":true,"recentEditorialEvents":[{"type":"reviewerAgreed","content":"143980467215058105174810254961758211290","date":"2026-05-11T02:53:32+00:00","index":20,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-03-04T07:48:22+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-04 07:48:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8919757","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8919757","identity":"rs-8919757","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}