Improving Large Language Models with Concept-Aware Fine-Tuning

Improving Large Language Models with Concept-Aware Fine-Tuning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Physical Sciences - Article Improving Large Language Models with Concept-Aware Fine-Tuning Dacheng Tao, Michael Chen, Xikun ZHANG, Jiaxing Huang, Yingjie Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7391246/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract Large language models (LLMs) have become the cornerstone of modern AI. However, the current paradigm of next-token prediction fundamentally limits their ability to form coherent, high-level concepts, making it a critical barrier to human-like understanding and reasoning. Specifically, an LLM will first decompose text into tokens, i.e., artificial text fragments. These tokens are then learned sequentially, rather than as part of a unified, coherent phrase or semantic entity 1 . This fragmented representation hinders deeper conceptual understanding and, ultimately, the development of truly intelligent systems 2–4 . In response, we introduce Concept-Aware Fine-Tuning (CAFT), a multi-token training method that reshapes how LLMs are fine-tuned. By enabling the learning of sequences that span multiple tokens, this method fosters stronger concept-aware learning. Our experiments demonstrate significant improvements compared to conventional next-token fine-tuning. CAFT can be applied to diverse tasks, from traditional LLM tasks like coding to challenging scientific tasks involving domain-specific modalities like de novo protein design. CAFT successfully leverages the multi-token setting for fine-tuning, an approach previously considered impossible 2,5,6 , by introducing several technical innovations to address the inherent challenges of fine-tuning. Our results challenge the machine learning research community to rethink the ubiquitous next-token prediction paradigm and enable the broader scientific community to develop more powerful scientific LLMs involving domain-specific modalities 7 . Physical sciences/Mathematics and computing/Computational science Physical sciences/Mathematics and computing/Computer science Physical sciences/Mathematics and computing/Information technology Full Text Additional Declarations There is NO Competing Interest. Supplementary Files NatureCAFTsupplementaryInformationtosubmit.docx Supplementary Information Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7391246","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Physical Sciences - Article","associatedPublications":[],"authors":[{"id":509228590,"identity":"eae8901c-b77b-467a-b002-c7ad77ee5cd0","order_by":0,"name":"Dacheng Tao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+klEQVRIiWNgGAWjYDCCAwxsMCbjAwidQKQWHgYGZgOIahK0sEkQpYXv+OFnDz7uqGWwZz98rLrwx2EGfvYcA8Yfv3BrkTyTZm4488xxBh6etLTbMxIOM0j2vDFg5u3DrcXgQA6bNG/bMaDDcsxu8wC1GNzIMWBm7MGj5fwbNum/IC38b8yKQVrsgVoYf+LTcgNoC2NbDQOPRI4ZM9gWiRwDBp4fePxy45mZZG/bAR6eG8+SpXnS0nkkzjwrOMzbgFsL3/nkZxI/2+rk2PuTD37msbGW429P3vjwxx/cWqDgMA+MBWYcYGwjqKUOXYCwLaNgFIyCUTByAAAF+075AKbXsQAAAABJRU5ErkJggg==","orcid":"","institution":"Nanyang Technological University","correspondingAuthor":true,"prefix":"","firstName":"Dacheng","middleName":"","lastName":"Tao","suffix":""},{"id":509228591,"identity":"e35720f9-ad0b-450e-b022-56faa12cdff6","order_by":1,"name":"Michael Chen","email":"","orcid":"","institution":"Nanyang Technological University","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Chen","suffix":""},{"id":509228592,"identity":"2e00f143-d1b5-4257-b22f-9f410cdb962e","order_by":2,"name":"Xikun ZHANG","email":"","orcid":"","institution":"Nanyang Technological University","correspondingAuthor":false,"prefix":"","firstName":"Xikun","middleName":"","lastName":"ZHANG","suffix":""},{"id":509228593,"identity":"9c9d0679-c13e-4e90-9b88-8e0886397813","order_by":3,"name":"Jiaxing Huang","email":"","orcid":"","institution":"Nanyang Technological University","correspondingAuthor":false,"prefix":"","firstName":"Jiaxing","middleName":"","lastName":"Huang","suffix":""},{"id":509228594,"identity":"2e57dfcb-56df-45e1-b5d5-bf8fe8e9652a","order_by":4,"name":"Yingjie Wang","email":"","orcid":"","institution":"Nanyang Technological University","correspondingAuthor":false,"prefix":"","firstName":"Yingjie","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2025-08-17 08:25:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7391246/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7391246/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92567848,"identity":"6983af9f-3884-4b61-bd71-a93b8491fa97","added_by":"auto","created_at":"2025-10-01 07:00:04","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":956530,"visible":true,"origin":"","legend":"","description":"","filename":"NatureCAFTmainarticletosubmit.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7391246/v1_covered_c3b47ece-3e23-4527-8426-387b44fc0542.pdf"},{"id":92567335,"identity":"f276b339-5414-472e-9f70-610ba3a84e21","added_by":"auto","created_at":"2025-10-01 06:52:01","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":446589,"visible":true,"origin":"","legend":"Supplementary Information","description":"","filename":"NatureCAFTsupplementaryInformationtosubmit.docx","url":"https://assets-eu.researchsquare.com/files/rs-7391246/v1/630e27e7b63116d0475e058c.docx"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Improving Large Language Models with Concept-Aware Fine-Tuning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7391246/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7391246/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eLarge language models (LLMs) have become the cornerstone of modern AI. However, the current paradigm of next-token prediction fundamentally limits their ability to form coherent, high-level concepts, making it a critical barrier to human-like understanding and reasoning. Specifically, an LLM will first decompose text into tokens, i.e., artificial text fragments. These tokens are then learned sequentially, rather than as part of a unified, coherent phrase or semantic entity\u003csup\u003e1\u003c/sup\u003e. This fragmented representation hinders deeper conceptual understanding and, ultimately, the development of truly intelligent systems\u003csup\u003e2–4\u003c/sup\u003e. In response, we introduce Concept-Aware Fine-Tuning (CAFT), a multi-token training method that reshapes how LLMs are fine-tuned. By enabling the learning of sequences that span multiple tokens, this method fosters stronger concept-aware learning. Our experiments demonstrate significant improvements compared to conventional next-token fine-tuning. CAFT can be applied to diverse tasks, from traditional LLM tasks like coding to challenging scientific tasks involving domain-specific modalities like de novo protein design. CAFT successfully leverages the multi-token setting for fine-tuning, an approach previously considered impossible\u003csup\u003e2,5,6\u003c/sup\u003e, by introducing several technical innovations to address the inherent challenges of fine-tuning. Our results challenge the machine learning research community to rethink the ubiquitous next-token prediction paradigm and enable the broader scientific community to develop more powerful scientific LLMs involving domain-specific modalities\u003csup\u003e7\u003c/sup\u003e.\u003c/p\u003e","manuscriptTitle":"Improving Large Language Models with Concept-Aware Fine-Tuning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-01 06:51:56","doi":"10.21203/rs.3.rs-7391246/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"a278f92d-2ee3-407d-bbd9-74f9437489ff","owner":[],"postedDate":"October 1st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":54084120,"name":"Physical sciences/Mathematics and computing/Computational science"},{"id":54084121,"name":"Physical sciences/Mathematics and computing/Computer science"},{"id":54084122,"name":"Physical sciences/Mathematics and computing/Information technology"}],"tags":[],"updatedAt":"2025-10-01T06:51:56+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-01 06:51:56","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7391246","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7391246","identity":"rs-7391246","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}