GPTVD: Vulnerability Detection and Analysis Method Based on LLM's Chain of Thoughts.

GPTVD: Vulnerability Detection and Analysis Method Based on LLM's Chain of Thoughts. | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article GPTVD: Vulnerability Detection and Analysis Method Based on LLM's Chain of Thoughts. Yinan Chen, Yuan Huang, Xiangping Chen, Pengfei Shen, Lei Yun This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6275325/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 09 Sep, 2025 Read the published version in Automated Software Engineering → Version 1 posted 10 You are reading this latest preprint version Abstract Purpose: Traditional vulnerability detection methods based on machine learning (ML) and deep learning (DL) primarily focus on coarse-grained predictions, often lacking precise localization and interpretability regarding the root causes of vulnerabilities. The growing availability of open-source vulnerability databases calls for advanced methods that can reason about vulnerabilities at a finer slice-level granularity. GPTVD, which leverages large language models’ (LLMs) in-context learning (ICL) and chain-of-thought (COT) reasoning capabilities. The goal is to enhance both detection performance and explainability. Methods: GPTVD extracts threat code slices through static code analysis, focusing on data and control dependencies. Positive and negative samples are clustered based on heuristic features, and representative samples are manually annotated with reasoning processes to build COT prompts. These prompts are combined with target samples to form LLM input queries, enabling slice-level vulnerability inference and explanation using ChatGPT. The method was evaluated on 3,512 threat code slices from a public dataset. Results: GPTVD achieved superior performance compared to state-of-the-art methods, with 90.72% accuracy, 86.71% precision, and 96.39% recall. Ablation studies confirm that clustering-based prompt selection, explicit threat code slices, and human expert reasoning significantly improve detection effectiveness and interpretability. Conclusion: GPTVD demonstrates that combining static code analysis with LLM-based COT reasoning can effectively detect vulnerabilities at the slice level with high accuracy and interpretability. Vulnerability detection Large language model ICL COT Prompt engineering Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 09 Sep, 2025 Read the published version in Automated Software Engineering → Version 1 posted Editorial decision: Revision requested 28 May, 2025 Reviews received at journal 18 May, 2025 Reviews received at journal 15 May, 2025 Reviewers agreed at journal 12 May, 2025 Reviewers agreed at journal 12 May, 2025 Reviewers agreed at journal 02 Apr, 2025 Reviewers invited by journal 30 Mar, 2025 Editor assigned by journal 29 Mar, 2025 Submission checks completed at journal 22 Mar, 2025 First submitted to journal 21 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6275325","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":440877601,"identity":"ad1f1540-7724-4142-9d97-fd78d4e81a57","order_by":0,"name":"Yinan Chen","email":"","orcid":"","institution":"School of Software Engineering, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Yinan","middleName":"","lastName":"Chen","suffix":""},{"id":440877603,"identity":"f55113c8-bbdc-4e73-a8a8-234d8ebb83b7","order_by":1,"name":"Yuan Huang","email":"","orcid":"","institution":"School of Software Engineering, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Yuan","middleName":"","lastName":"Huang","suffix":""},{"id":440877604,"identity":"19cb45e6-6740-4a86-bee7-089a4d8855b3","order_by":2,"name":"Xiangping Chen","email":"","orcid":"","institution":"School of Journalism and Communication, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Xiangping","middleName":"","lastName":"Chen","suffix":""},{"id":440877605,"identity":"b687180b-d287-43a1-a563-748172bcdd34","order_by":3,"name":"Pengfei Shen","email":"","orcid":"","institution":"School of Software Engineering, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Pengfei","middleName":"","lastName":"Shen","suffix":""},{"id":440877606,"identity":"3f6752dd-ac37-4d56-8ce3-b817507f7bac","order_by":4,"name":"Lei Yun","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAn0lEQVRIiWNgGAWjYDACduYDYJqNeC3MbAkka+ExIF4xGBgc5vn44eOOe3J8DMwPH90gTgvvZsmZZ4qN2RjYjI1ziNSyjZm3LSGxjYGHTZpILTzPmP+2JdSTpIWNmbEtIYGNaC2Sh9mMJXvPJBi2MRPrF77jzQ8//NyRIC/f3vzwMVFaFA4ACcYGIMFMjHIQkG+AaRkFo2AUjIJRgAsAACf2KpcckesmAAAAAElFTkSuQmCC","orcid":"","institution":"School of Software Engineering, Sun Yat-sen University","correspondingAuthor":true,"prefix":"","firstName":"Lei","middleName":"","lastName":"Yun","suffix":""}],"badges":[],"createdAt":"2025-03-21 07:53:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6275325/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6275325/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10515-025-00550-4","type":"published","date":"2025-09-09T15:57:03+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":91359169,"identity":"e0fb7009-031e-4f27-9838-4ff7db7a4b29","added_by":"auto","created_at":"2025-09-15 16:05:25","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":917288,"visible":true,"origin":"","legend":"","description":"","filename":"GPTVDASE.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6275325/v1_covered_1b4ee60a-2bf6-41cc-810c-d01bd5309b08.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"GPTVD: Vulnerability Detection and Analysis Method Based on LLM's Chain of Thoughts.","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"automated-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ause","sideBox":"Learn more about [Automated Software Engineering](http://link.springer.com/journal/10515)","snPcode":"10515","submissionUrl":"https://submission.nature.com/new-submission/10515/3","title":"Automated Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Vulnerability detection, Large language model, ICL, COT, Prompt engineering","lastPublishedDoi":"10.21203/rs.3.rs-6275325/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6275325/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003ePurpose:\u003c/h2\u003e\u003cp\u003eTraditional vulnerability detection methods based on machine learning (ML) and deep learning (DL) primarily focus on coarse-grained predictions, often lacking precise localization and interpretability regarding the root causes of vulnerabilities. The growing availability of open-source vulnerability databases calls for advanced methods that can reason about vulnerabilities at a finer slice-level granularity. GPTVD, which leverages large language models\u0026rsquo; (LLMs) in-context learning (ICL) and chain-of-thought (COT) reasoning capabilities. The goal is to enhance both detection performance and explainability.\u003c/p\u003e\u003ch2\u003eMethods:\u003c/h2\u003e\u003cp\u003eGPTVD extracts threat code slices through static code analysis, focusing on data and control dependencies. Positive and negative samples are clustered based on heuristic features, and representative samples are manually annotated with reasoning processes to build COT prompts. These prompts are combined with target samples to form LLM input queries, enabling slice-level vulnerability inference and explanation using ChatGPT. The method was evaluated on 3,512 threat code slices from a public dataset.\u003c/p\u003e\u003ch2\u003eResults:\u003c/h2\u003e\u003cp\u003eGPTVD achieved superior performance compared to state-of-the-art methods, with 90.72% accuracy, 86.71% precision, and 96.39% recall. Ablation studies confirm that clustering-based prompt selection, explicit threat code slices, and human expert reasoning significantly improve detection effectiveness and interpretability.\u003c/p\u003e\u003ch2\u003eConclusion:\u003c/h2\u003e\u003cp\u003eGPTVD demonstrates that combining static code analysis with LLM-based COT reasoning can effectively detect vulnerabilities at the slice level with high accuracy and interpretability.\u003c/p\u003e","manuscriptTitle":"GPTVD: Vulnerability Detection and Analysis Method Based on LLM's Chain of Thoughts.","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-21 15:46:32","doi":"10.21203/rs.3.rs-6275325/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-05-28T14:40:08+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-18T23:24:14+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-15T05:57:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"25200624951321875502209766586423170608","date":"2025-05-12T22:23:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"19670489804494374604415408842412278711","date":"2025-05-12T12:31:17+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"274790385804597249306140167982976763135","date":"2025-04-02T21:08:09+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-31T02:00:40+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-03-29T19:26:04+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-03-22T12:51:51+00:00","index":"","fulltext":""},{"type":"submitted","content":"Automated Software Engineering","date":"2025-03-21T07:45:33+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"automated-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ause","sideBox":"Learn more about [Automated Software Engineering](http://link.springer.com/journal/10515)","snPcode":"10515","submissionUrl":"https://submission.nature.com/new-submission/10515/3","title":"Automated Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"240d80c7-c773-47bb-8648-7c2c6313f2e8","owner":[],"postedDate":"April 21st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-09-15T16:04:20+00:00","versionOfRecord":{"articleIdentity":"rs-6275325","link":"https://doi.org/10.1007/s10515-025-00550-4","journal":{"identity":"automated-software-engineering","isVorOnly":false,"title":"Automated Software Engineering"},"publishedOn":"2025-09-09 15:57:03","publishedOnDateReadable":"September 9th, 2025"},"versionCreatedAt":"2025-04-21 15:46:32","video":"","vorDoi":"10.1007/s10515-025-00550-4","vorDoiUrl":"https://doi.org/10.1007/s10515-025-00550-4","workflowStages":[]},"version":"v1","identity":"rs-6275325","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6275325","identity":"rs-6275325","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}