Adversarial Generation Method for Smart Contract Fuzz Testing Seeds Guided by Chain-Based LLM

Adversarial Generation Method for Smart Contract Fuzz Testing Seeds Guided by Chain-Based LLM | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Adversarial Generation Method for Smart Contract Fuzz Testing Seeds Guided by Chain-Based LLM Jiaze Sun, Zhiqiang Yin, Hengshan Zhang, Xiang Chen, Wei Zheng This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4666159/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 31 Dec, 2024 Read the published version in Automated Software Engineering → Version 1 posted 9 You are reading this latest preprint version Abstract With the rapid development of smart contract technology and the continuous expansion of blockchain application scenarios, the security issues of smart contracts have garnered significant attention. However, traditional fuzz testing typically relies on randomly generated initial seed sets. This random generation method fails to understand the semantics of smart contracts, resulting in insufficient seed coverage. Additionally, traditional fuzz testing often ignores the syntax and semantic constraints within smart contracts, leading to the generation of seeds that may not conform to the syntactic rules of the contracts and may even include logic that violates contract semantics, thereby reducing the efficiency of fuzz testing. To address these challenges, we propose a method for adversarial generation for smart contract fuzz testing seeds guided by Chain-Based LLM, leveraging the deep semantic understanding capabilities of LLM to assist in seed set generation. Firstly, we propose a method that utilizes Chain-Based prompts to request LLM to generate fuzz testing seeds, breaking down the LLM tasks into multiple steps to gradually guide the LLM in generating high-coverage seed sets. Secondly, by establishing adversarial roles for the LLM, we guide the LLM to autonomously generate and optimize seed sets, producing high-coverage initial seed sets for the program under test. To evaluate the effectiveness of the proposed method, 160 smart contracts were crawled from Etherscan for experimental purposes. Results indicate that using Chain-Based prompts to request LLM to generate fuzz testing seed sets improved instruction coverage by 2.63% compared to single-step requests. The method of generating seed sets by establishing adversarial roles for the LLM reduced the time to reach maximum instruction coverage from 60 seconds to approximately 30 seconds compared to single-role methods. Additionally, the seed sets generated by the proposed method can directly trigger simple types of vulnerabilities (e.g., timestamp dependency and block number dependency vulnerabilities), with instruction coverage improvements of 3.3% and 3.8%, respectively. Fuzz Testing Smart Contracts LLM Seed Set Generation Prompts Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 31 Dec, 2024 Read the published version in Automated Software Engineering → Version 1 posted Editorial decision: Revision requested 22 Aug, 2024 Reviews received at journal 22 Aug, 2024 Reviews received at journal 13 Aug, 2024 Reviewers agreed at journal 15 Jul, 2024 Reviewers agreed at journal 14 Jul, 2024 Reviewers invited by journal 14 Jul, 2024 Editor assigned by journal 03 Jul, 2024 Submission checks completed at journal 03 Jul, 2024 First submitted to journal 01 Jul, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4666159","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":329337433,"identity":"c3ff0332-9082-4a19-b881-ca6d4a7c6072","order_by":0,"name":"Jiaze Sun","email":"","orcid":"","institution":"Xi’an University of Posts and Telecommunications","correspondingAuthor":false,"prefix":"","firstName":"Jiaze","middleName":"","lastName":"Sun","suffix":""},{"id":329337434,"identity":"c0a4217d-fc38-4e68-975f-7b11e276799a","order_by":1,"name":"Zhiqiang Yin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIie3RoQ7CMBCA4S5Nqjpmu4zAK3RmBoLhRdaQDAMJsgLBEtIJwCN4iEnkSJOpAyyO4RHgkCQgIdtwiH76/uSuRcgw/hBp7+LiJrs9Bx0vRSin1UmDYe2vIRq4MWBeQF6dtBiJPFtpK9VA3Msc11jMW3KPgsau2udSzAhykkVYnjSB+xs5bDj0EJ3EtokY7NPyBI14eIUOdtdZcBJAEGfj6iSzFbbScxFMhMI1EhaJ2FZ9K80gQPUSqrX1euRZPmAh5LTylnYSJ4/3V+rd/SGnLSdZlScf6G/jhmEYxldPGYNQ51G6paIAAAAASUVORK5CYII=","orcid":"","institution":"Xi’an University of Posts and Telecommunications","correspondingAuthor":true,"prefix":"","firstName":"Zhiqiang","middleName":"","lastName":"Yin","suffix":""},{"id":329337435,"identity":"c86bd9f2-ad1b-4c86-8b94-3cc6f378729f","order_by":2,"name":"Hengshan Zhang","email":"","orcid":"","institution":"Xi’an University of Posts and Telecommunications","correspondingAuthor":false,"prefix":"","firstName":"Hengshan","middleName":"","lastName":"Zhang","suffix":""},{"id":329337436,"identity":"b6114db5-3166-4cb4-9d70-e51ccf41b534","order_by":3,"name":"Xiang Chen","email":"","orcid":"","institution":"Nantong University","correspondingAuthor":false,"prefix":"","firstName":"Xiang","middleName":"","lastName":"Chen","suffix":""},{"id":329337437,"identity":"c1840c1a-b4ef-4201-bed1-d4c78a3f955a","order_by":4,"name":"Wei Zheng","email":"","orcid":"","institution":"Northwestern Polytechnical University","correspondingAuthor":false,"prefix":"","firstName":"Wei","middleName":"","lastName":"Zheng","suffix":""}],"badges":[],"createdAt":"2024-07-01 07:37:39","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4666159/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4666159/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10515-024-00483-4","type":"published","date":"2024-12-31T15:57:06+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":73095125,"identity":"88e2f998-0f6a-4d6f-abd1-53ba97cf6fc8","added_by":"auto","created_at":"2025-01-06 16:24:42","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":822182,"visible":true,"origin":"","legend":"","description":"","filename":"AdversarialGenerationMethodforSmartContractFuzzTestingSeedsGuidedbyChainBasedLLM.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4666159/v1_covered_5b46094b-7bc5-40cb-8244-4c27c750343f.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Adversarial Generation Method for Smart Contract Fuzz Testing Seeds Guided by Chain-Based LLM","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"automated-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ause","sideBox":"Learn more about [Automated Software Engineering](http://link.springer.com/journal/10515)","snPcode":"10515","submissionUrl":"https://submission.nature.com/new-submission/10515/3","title":"Automated Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Fuzz Testing, Smart Contracts, LLM, Seed Set Generation, Prompts","lastPublishedDoi":"10.21203/rs.3.rs-4666159/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4666159/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"With the rapid development of smart contract technology and the continuous expansion of blockchain application scenarios, the security issues of smart contracts have garnered significant attention. However, traditional fuzz testing typically relies on randomly generated initial seed sets. This random generation method fails to understand the semantics of smart contracts, resulting in insufficient seed coverage. Additionally, traditional fuzz testing often ignores the syntax and semantic constraints within smart contracts, leading to the generation of seeds that may not conform to the syntactic rules of the contracts and may even include logic that violates contract semantics, thereby reducing the efficiency of fuzz testing. To address these challenges, we propose a method for adversarial generation for smart contract fuzz testing seeds guided by Chain-Based LLM, leveraging the deep semantic understanding capabilities of LLM to assist in seed set generation. Firstly, we propose a method that utilizes Chain-Based prompts to request LLM to generate fuzz testing seeds, breaking down the LLM tasks into multiple steps to gradually guide the LLM in generating high-coverage seed sets. Secondly, by establishing adversarial roles for the LLM, we guide the LLM to autonomously generate and optimize seed sets, producing high-coverage initial seed sets for the program under test. To evaluate the effectiveness of the proposed method, 160 smart contracts were crawled from Etherscan for experimental purposes. Results indicate that using Chain-Based prompts to request LLM to generate fuzz testing seed sets improved instruction coverage by 2.63\\% compared to single-step requests. The method of generating seed sets by establishing adversarial roles for the LLM reduced the time to reach maximum instruction coverage from 60 seconds to approximately 30 seconds compared to single-role methods. Additionally, the seed sets generated by the proposed method can directly trigger simple types of vulnerabilities (e.g., timestamp dependency and block number dependency vulnerabilities), with instruction coverage improvements of 3.3\\% and 3.8\\%, respectively.","manuscriptTitle":"Adversarial Generation Method for Smart Contract Fuzz Testing Seeds Guided by Chain-Based LLM","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-26 14:34:15","doi":"10.21203/rs.3.rs-4666159/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-08-22T09:17:57+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-22T08:42:18+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-13T08:08:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"114365721273154323121197627944843319750","date":"2024-07-15T08:24:58+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"256639866134459908725481742350980551047","date":"2024-07-15T01:18:29+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-07-14T23:08:56+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-07-03T16:25:23+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-07-03T16:24:57+00:00","index":"","fulltext":""},{"type":"submitted","content":"Automated Software Engineering","date":"2024-07-01T07:36:21+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"automated-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ause","sideBox":"Learn more about [Automated Software Engineering](http://link.springer.com/journal/10515)","snPcode":"10515","submissionUrl":"https://submission.nature.com/new-submission/10515/3","title":"Automated Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"4d9dd0b1-b803-4ced-8a6c-e66fe1d34369","owner":[],"postedDate":"July 26th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-01-06T16:23:09+00:00","versionOfRecord":{"articleIdentity":"rs-4666159","link":"https://doi.org/10.1007/s10515-024-00483-4","journal":{"identity":"automated-software-engineering","isVorOnly":false,"title":"Automated Software Engineering"},"publishedOn":"2024-12-31 15:57:06","publishedOnDateReadable":"December 31st, 2024"},"versionCreatedAt":"2024-07-26 14:34:15","video":"","vorDoi":"10.1007/s10515-024-00483-4","vorDoiUrl":"https://doi.org/10.1007/s10515-024-00483-4","workflowStages":[]},"version":"v1","identity":"rs-4666159","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4666159","identity":"rs-4666159","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}