Event Extraction Based on Self-Data Augmentation with Large Language Models

Event Extraction Based on Self-Data Augmentation with Large Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Event Extraction Based on Self-Data Augmentation with Large Language Models Lishan Yang, Xi Fan, Xiangyu Wang, Xin Wang, Qiuju Chen This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4444035/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 31 Jan, 2025 Read the published version in Memetic Computing → Version 1 posted 9 You are reading this latest preprint version Abstract Event extraction plays a crucial role in natural language processing (NLP), facilitating the transformation of unstructured text into structured representations. This conversion significantly enhances the performance of various applications, such as automated question answering and information retrieval systems. However, traditional event extraction methodologies often encounter challenges stemming from limited datasets, imbalanced sample distributions, the necessity for extra resources to annotate large datasets, and the potential for data quality degradation during the augmentation process. To surmount these obstacles, this study introduces an innovative self-data augmentation strategy that leverages a single large language model (LLM) to concurrently perform data augmentation and event extraction. By dynamically assessing and refining the quality of generated samples, this approach mitigates the inclusion of noisy data, ultimately bolstering the model's performance. Demonstrable enhancements in precision, recall, and F1 scores across various model configurations underscore the efficacy of this strategy in managing small and imbalanced datasets. Furthermore, the incorporation of Logical Thoughts for Self-Data Augmentation (LoTSA) ensures the superior quality of augmented data, culminating in more accurate and reliable extraction outcomes. Event extraction Data augmentation Large language models Logical Thoughts for Self-Data Augmentation (LoTSA) Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 31 Jan, 2025 Read the published version in Memetic Computing → Version 1 posted Editorial decision: Revision requested 17 Jun, 2024 Reviews received at journal 06 Jun, 2024 Reviews received at journal 03 Jun, 2024 Reviewers agreed at journal 03 Jun, 2024 Reviewers agreed at journal 03 Jun, 2024 Reviewers invited by journal 03 Jun, 2024 Editor assigned by journal 21 May, 2024 Submission checks completed at journal 20 May, 2024 First submitted to journal 19 May, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4444035","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":308336892,"identity":"fc83fe14-14c1-48b8-8c3d-0624eeccc8b6","order_by":0,"name":"Lishan Yang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+0lEQVRIiWNgGAWjYDACdsYGZiAlx8DA2HgAJiiBVwszRIsxUEvDAZhqAlrAiCGxAUgQp8XgMHPz54KaO+lr2w83HGBsq6vjb2A+eJuHwS4PtxbGNukZx57lbjuTCNJyWELiAFuyNQ9DcjE+Lcw8bIdztx0AazkgYcDAYybNw3AA7FQcWpo/8/w7nG52/iHYYUAt/N8IaWmQ5m07nGB2A2wLM8gWNrxaJEF+4e07bLjtBtCWhHOHJWccZjO2nGOQjFML3/H2x595vh2WNzuf/vDBh7I6fv725oc33lTY4dSicACZlwAimMEOxqEeCORxmTUKRsEoGAWjAA4ATlRZl98gA4AAAAAASUVORK5CYII=","orcid":"","institution":"University of Science and Technology of China","correspondingAuthor":true,"prefix":"","firstName":"Lishan","middleName":"","lastName":"Yang","suffix":""},{"id":308336893,"identity":"78b2c50b-76de-4ff5-bdc2-c8b23fbb8d1a","order_by":1,"name":"Xi Fan","email":"","orcid":"","institution":"University of Science and Technology of China","correspondingAuthor":false,"prefix":"","firstName":"Xi","middleName":"","lastName":"Fan","suffix":""},{"id":308336894,"identity":"13a23ef3-e46a-4e3c-8d56-3f1f22f05c9e","order_by":2,"name":"Xiangyu Wang","email":"","orcid":"","institution":"University of Science and Technology of China","correspondingAuthor":false,"prefix":"","firstName":"Xiangyu","middleName":"","lastName":"Wang","suffix":""},{"id":308336896,"identity":"418defc9-7539-4bc0-b4cf-a2f6486035de","order_by":3,"name":"Xin Wang","email":"","orcid":"","institution":"University of Science and Technology of China","correspondingAuthor":false,"prefix":"","firstName":"Xin","middleName":"","lastName":"Wang","suffix":""},{"id":308336898,"identity":"edbbe613-e472-4a85-845b-9b07cfed3f7d","order_by":4,"name":"Qiuju Chen","email":"","orcid":"","institution":"University of Science and Technology of China","correspondingAuthor":false,"prefix":"","firstName":"Qiuju","middleName":"","lastName":"Chen","suffix":""}],"badges":[],"createdAt":"2024-05-19 11:08:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4444035/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4444035/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s12293-025-00436-8","type":"published","date":"2025-01-31T15:58:16+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":75351376,"identity":"51e538e8-7cd8-4cfd-ae4a-ff84e653aa3f","added_by":"auto","created_at":"2025-02-03 16:10:23","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":489006,"visible":true,"origin":"","legend":"","description":"","filename":"EventExtractionBasedonSelfDataAugmentationwithLargeLanguageModelsCopy.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4444035/v1_covered_b2caecf6-fcd5-410d-9b44-325e415acc3e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Event Extraction Based on Self-Data Augmentation with Large Language Models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"memetic-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meme","sideBox":"Learn more about [Memetic Computing](http://link.springer.com/journal/12289)","snPcode":"12293","submissionUrl":"https://submission.nature.com/new-submission/12293/3","title":"Memetic Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Event extraction, Data augmentation, Large language models, Logical Thoughts for Self-Data Augmentation (LoTSA)","lastPublishedDoi":"10.21203/rs.3.rs-4444035/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4444035/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\n\nEvent extraction plays a crucial role in natural language processing (NLP), facilitating the transformation of unstructured text into structured representations. This conversion significantly enhances the performance of various applications, such as automated question answering and information retrieval systems. However, traditional event extraction methodologies often encounter challenges stemming from limited datasets, imbalanced sample distributions, the necessity for extra resources to annotate large datasets, and the potential for data quality degradation during the augmentation process. To surmount these obstacles, this study introduces an innovative self-data augmentation strategy that leverages a single large language model (LLM) to concurrently perform data augmentation and event extraction. By dynamically assessing and refining the quality of generated samples, this approach mitigates the inclusion of noisy data, ultimately bolstering the model's performance. Demonstrable enhancements in precision, recall, and F1 scores across various model configurations underscore the efficacy of this strategy in managing small and imbalanced datasets. Furthermore, the incorporation of Logical Thoughts for Self-Data Augmentation (LoTSA) ensures the superior quality of augmented data, culminating in more accurate and reliable extraction outcomes. \n","manuscriptTitle":"Event Extraction Based on Self-Data Augmentation with Large Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-03 06:35:23","doi":"10.21203/rs.3.rs-4444035/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-06-17T15:46:21+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-06T04:31:44+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-03T16:58:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"83241495549519088279402733976928195874","date":"2024-06-03T14:06:04+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"253453366846159136490537790594600068682","date":"2024-06-03T10:40:22+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-06-03T10:11:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-21T17:33:37+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-20T18:18:30+00:00","index":"","fulltext":""},{"type":"submitted","content":"Memetic Computing","date":"2024-05-19T10:57:22+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"memetic-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meme","sideBox":"Learn more about [Memetic Computing](http://link.springer.com/journal/12289)","snPcode":"12293","submissionUrl":"https://submission.nature.com/new-submission/12293/3","title":"Memetic Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"30cb0ac6-8656-40f7-9c13-ef733e027b73","owner":[],"postedDate":"June 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-02-03T16:04:56+00:00","versionOfRecord":{"articleIdentity":"rs-4444035","link":"https://doi.org/10.1007/s12293-025-00436-8","journal":{"identity":"memetic-computing","isVorOnly":false,"title":"Memetic Computing"},"publishedOn":"2025-01-31 15:58:16","publishedOnDateReadable":"January 31st, 2025"},"versionCreatedAt":"2024-06-03 06:35:23","video":"","vorDoi":"10.1007/s12293-025-00436-8","vorDoiUrl":"https://doi.org/10.1007/s12293-025-00436-8","workflowStages":[]},"version":"v1","identity":"rs-4444035","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4444035","identity":"rs-4444035","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}