Research on a Denoising Model for Entity-Relation Extraction Using Hierarchical Contrastive Learning with Distant Supervision

Research on a Denoising Model for Entity-Relation Extraction Using Hierarchical Contrastive Learning with Distant Supervision | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Research on a Denoising Model for Entity-Relation Extraction Using Hierarchical Contrastive Learning with Distant Supervision Ayiguli Halike*, Aishan Wumaier, kahaerjiang abiderexiti, Tuergen Yibulayin This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6429029/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Jul, 2025 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract Distant supervision is a technique that utilizes knowledge base information to automatically generate labels for text samples, enabling the large-scale creation of labeled data. However, this approach often encounters the issue of noisy labels in practice, which arises from inaccuracies in the alignment between the text and the knowledge base, leading to erroneous generated labels that adversely affect the model's performance. In the task of relation extraction, such noise not only diminishes extraction accuracy but may also cause the model to favor the recognition of common relations while neglecting long-tail relations. To address these issues, this paper proposes an innovative hierarchical contrastive learning framework, specifically applied to the Uyghur language using pre-trained models for XML and CINO minority language modeling. This framework effectively integrates both global structural information and local fine-grained interactions to reduce noise within sentences. Specifically, a three-layer learning architecture is designed, which incorporates interactions at different levels and employs a multi-head self-attention mechanism to generate denoised context-aware representations, referred to as multi-granular re-contextualization. Additionally, a dynamic gradient adversarial perturbation data augmentation strategy is introduced to provide pseudo-positive samples for contrastive learning, further enhancing the model's capabilities in recognizing rare relations. Experimental results demonstrate that the proposed framework significantly improves accuracy and robustness in the task of Uyghur relation extraction, validating its effectiveness and innovativeness. This research offers new perspectives and methodologies for the field of distant supervision in relation extraction, advancing further development in this area. Biological sciences/Computational biology and bioinformatics Physical sciences/Engineering Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 01 Jul, 2025 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 06 May, 2025 Reviews received at journal 01 May, 2025 Reviews received at journal 28 Apr, 2025 Reviewers agreed at journal 28 Apr, 2025 Reviewers agreed at journal 28 Apr, 2025 Reviewers invited by journal 28 Apr, 2025 Editor assigned by journal 28 Apr, 2025 Editor invited by journal 22 Apr, 2025 Submission checks completed at journal 21 Apr, 2025 First submitted to journal 11 Apr, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6429029","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":449279054,"identity":"c86cdee2-15ed-47c1-8f8b-4467b5bbe8fa","order_by":0,"name":"Ayiguli Halike*","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABB0lEQVRIie3RMUvDQBTA8RcOrsvDOF6o6Fd4QRDFkH6VyEGyHEUQSsdAVz+AYj9MJOAtznLikuDQJUMmJ0GvloIOl+ImeP8h5I734wUC4PP91RpKMByVX+9s/RA7SXaZH0TXVVD+gvR1QiYLyu3FIAlviieREUN4Xq1eEeoRVez+BSGduogwamYJx+BOxQuEnFHF5TmCvHKuMSq3BJGNVWBJYgmejBGqi9IhjjZEII90a4mwJHwbJGSKB0vsGgHxdgsfJPFjx04zylCgim+XlLOo5sdnS5JOcqiL1vTvH5OJ1k3fzWu5pxet6eapk8C+ou/fCXLzM8kxvi7UzY9zOjDr8/l8/7RPjZxQBa8ZeVYAAAAASUVORK5CYII=","orcid":"","institution":"Institute of Medical Engineering Interdisciplinary Research, Xinjiang Medical University, Urumqi 830017, China","correspondingAuthor":true,"prefix":"","firstName":"Ayiguli","middleName":"","lastName":"Halike*","suffix":""},{"id":449279055,"identity":"c2c8131f-9f9a-4902-a949-fa5be9512463","order_by":1,"name":"Aishan Wumaier","email":"","orcid":"","institution":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China;","correspondingAuthor":false,"prefix":"","firstName":"Aishan","middleName":"","lastName":"Wumaier","suffix":""},{"id":449279056,"identity":"bfad5329-82ab-4ad1-9de7-8ba0ed09ae0d","order_by":2,"name":"kahaerjiang abiderexiti","email":"","orcid":"","institution":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China;","correspondingAuthor":false,"prefix":"","firstName":"kahaerjiang","middleName":"","lastName":"abiderexiti","suffix":""},{"id":449279057,"identity":"18ca6fc8-a3d0-4393-bd50-99ca2a6f6455","order_by":3,"name":"Tuergen Yibulayin","email":"","orcid":"","institution":"Xinjiang Multilingual Information Technology Laboratory Research Center, Urumqi 830046, China;","correspondingAuthor":false,"prefix":"","firstName":"Tuergen","middleName":"","lastName":"Yibulayin","suffix":""}],"badges":[],"createdAt":"2025-04-11 14:08:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6429029/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6429029/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-04474-7","type":"published","date":"2025-07-01T15:57:26+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":86178973,"identity":"3c6da916-93c8-45c3-b519-14297f011356","added_by":"auto","created_at":"2025-07-07 16:13:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":376966,"visible":true,"origin":"","legend":"","description":"","filename":"202504211539SRResearchonaDenoisingModelforEntityRelationExtractionUsingHierarchicalContrastiveLearningwithDistantSupervision.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6429029/v1_covered_34fe1daf-82ae-45af-b8c0-dac95fda140d.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Research on a Denoising Model for Entity-Relation Extraction Using Hierarchical Contrastive Learning with Distant Supervision","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6429029/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6429029/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDistant supervision is a technique that utilizes knowledge base information to automatically generate labels for text samples, enabling the large-scale creation of labeled data. However, this approach often encounters the issue of noisy labels in practice, which arises from inaccuracies in the alignment between the text and the knowledge base, leading to erroneous generated labels that adversely affect the model's performance. In the task of relation extraction, such noise not only diminishes extraction accuracy but may also cause the model to favor the recognition of common relations while neglecting long-tail relations. To address these issues, this paper proposes an innovative hierarchical contrastive learning framework, specifically applied to the Uyghur language using pre-trained models for XML and CINO minority language modeling. This framework effectively integrates both global structural information and local fine-grained interactions to reduce noise within sentences. Specifically, a three-layer learning architecture is designed, which incorporates interactions at different levels and employs a multi-head self-attention mechanism to generate denoised context-aware representations, referred to as multi-granular re-contextualization. Additionally, a dynamic gradient adversarial perturbation data augmentation strategy is introduced to provide pseudo-positive samples for contrastive learning, further enhancing the model's capabilities in recognizing rare relations. Experimental results demonstrate that the proposed framework significantly improves accuracy and robustness in the task of Uyghur relation extraction, validating its effectiveness and innovativeness. This research offers new perspectives and methodologies for the field of distant supervision in relation extraction, advancing further development in this area.\u003c/p\u003e","manuscriptTitle":"Research on a Denoising Model for Entity-Relation Extraction Using Hierarchical Contrastive Learning with Distant Supervision","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-30 06:30:01","doi":"10.21203/rs.3.rs-6429029/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-05-06T08:35:14+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-01T07:09:25+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-28T18:12:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"204947333712304912277199715175312755261","date":"2025-04-28T06:12:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"311317841760218914160974460491673072805","date":"2025-04-28T04:38:48+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-04-28T04:30:48+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-04-28T04:27:55+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-04-22T12:10:23+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-04-21T08:43:26+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-04-11T13:53:41+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"79f5e60c-ce6f-4fb8-bfac-39a744494555","owner":[],"postedDate":"April 30th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":47810483,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":47810484,"name":"Physical sciences/Engineering"}],"tags":[],"updatedAt":"2025-07-07T16:01:41+00:00","versionOfRecord":{"articleIdentity":"rs-6429029","link":"https://doi.org/10.1038/s41598-025-04474-7","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2025-07-01 15:57:26","publishedOnDateReadable":"July 1st, 2025"},"versionCreatedAt":"2025-04-30 06:30:01","video":"","vorDoi":"10.1038/s41598-025-04474-7","vorDoiUrl":"https://doi.org/10.1038/s41598-025-04474-7","workflowStages":[]},"version":"v1","identity":"rs-6429029","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6429029","identity":"rs-6429029","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}