An Incremental Entity Resolution Approach based on Deep Metric Learning

An Incremental Entity Resolution Approach based on Deep Metric Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An Incremental Entity Resolution Approach based on Deep Metric Learning Yaoli Xu, Chenglin Li, Pu Li, Xiaoyu Duan, Yanyan Wang, Wanhua Qi, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4982747/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Entity resolution (ER), a crucial component of data fusion, aims to identify and aggregate disparate entity descriptions describing the same real-world object. While existing approaches have achieved good performance in ER tasks, they fall short in scenarios with dynamic data sources. The inherent dynamism of these data sources requires ER models to progressively resolve entities with incoming data while maintaining high accuracy. To address such issue, we propose a novel dynamic data processing framework that leverages the matching model from the previous time step to parse incoming incremental data, and implement an Incremental Entity Resolution Approach based on Deep Metric Learning (IER-DML), developed to handle dynamic data and enhance matching capability. an uncertainty metric-based data augmentation mechanism is proposed to mitigate calibration issues caused by suboptimal training data, and to generate high-confidence labeled entity pairs. These labeled data are then used for the iterative optimization of IER-DML, enhancing ER capabilities of IER-DML. For modeling dynamic changes in the distributions of clustering structure among entity pairs, we propose a deep metric learning-based multi-center matcher. This matcher learns metric relationships between the deep semantic features of entity pairs, and models intrinsic structures using "soft centers". These centers capture multiple information clusters among entities, and flexibly represent clustering structures. By integrating deep metric learning with a multi-center strategy, it can dynamically adjust classification boundaries and metric standards in complex scenarios. Comprehensive experiments demonstrate that our approach outperforms state-of-the-art methods across all four dynamic benchmark datasets. Incremental Entity Resolution Deep Metric Learning Dynamic data scenarios Data Augmentation Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4982747","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":356354302,"identity":"e3e946b4-4b30-4ecc-ad09-e13ce4fda2bc","order_by":0,"name":"Yaoli Xu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1klEQVRIiWNgGAWjYHACxgMJDAxyMAZjAzF6QCqNYQwitQBxYgOUQViLwfHDBw48bDucPr//jMGBBww2shsOMD97gFfLmbSEAwlnDuc2NgC1JDCkGW84wGZugFfLgRygyorDuc2MPSAthxM3HOBhk8Cr5fwboEqDw+lszDwgLf+J0HIDYksCDxtYywHCWiRvPAP5Jd1wBg9bAdC6ZOOZh9nM8GrhO5988OHPNmt5+f7DGx/+qLCT7Tve/AyvFoUDqO4EYmZ86oFAvoGAglEwCkbBKBgFDABrB1K67hMhGAAAAABJRU5ErkJggg==","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":true,"prefix":"","firstName":"Yaoli","middleName":"","lastName":"Xu","suffix":""},{"id":356354303,"identity":"d664d493-eae8-4b9c-af67-ca9087a626db","order_by":1,"name":"Chenglin Li","email":"","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":false,"prefix":"","firstName":"Chenglin","middleName":"","lastName":"Li","suffix":""},{"id":356354304,"identity":"64c2d15f-1813-44dc-8add-8e42148b8c6c","order_by":2,"name":"Pu Li","email":"","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":false,"prefix":"","firstName":"Pu","middleName":"","lastName":"Li","suffix":""},{"id":356354305,"identity":"575c0e36-170a-4f24-95a1-1e45466db648","order_by":3,"name":"Xiaoyu Duan","email":"","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":false,"prefix":"","firstName":"Xiaoyu","middleName":"","lastName":"Duan","suffix":""},{"id":356354306,"identity":"8e05ad33-78eb-4943-9bdc-8de3d2057562","order_by":4,"name":"Yanyan Wang","email":"","orcid":"","institution":"Shan xi University","correspondingAuthor":false,"prefix":"","firstName":"Yanyan","middleName":"","lastName":"Wang","suffix":""},{"id":356354307,"identity":"2071730c-8bb4-4fa0-a0f6-68426009e2f8","order_by":5,"name":"Wanhua Qi","email":"","orcid":"","institution":"Anyang Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Wanhua","middleName":"","lastName":"Qi","suffix":""},{"id":356354308,"identity":"d28478c5-00fc-41b3-917e-6ac18197cac9","order_by":6,"name":"Yanbu Guo","email":"","orcid":"","institution":"Anyang Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Yanbu","middleName":"","lastName":"Guo","suffix":""},{"id":356354312,"identity":"cfc50679-bd43-41c0-9c60-7c2e8cf5e4e4","order_by":7,"name":"Yuhua Li","email":"","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":false,"prefix":"","firstName":"Yuhua","middleName":"","lastName":"Li","suffix":""},{"id":356354314,"identity":"b368f620-ae2e-4d22-9fd3-48e88dc7a3f5","order_by":8,"name":"xiayang Shi","email":"","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":false,"prefix":"","firstName":"xiayang","middleName":"","lastName":"Shi","suffix":""},{"id":356354315,"identity":"92411d63-b45e-4df1-89be-f50a8bf8a1fb","order_by":9,"name":"Suzhi Zhang","email":"","orcid":"","institution":"Zhengzhou University of Light Industry","correspondingAuthor":false,"prefix":"","firstName":"Suzhi","middleName":"","lastName":"Zhang","suffix":""}],"badges":[],"createdAt":"2024-08-27 08:25:00","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4982747/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4982747/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":66313407,"identity":"555582d8-26bb-43fd-94f2-4308dca5316a","added_by":"auto","created_at":"2024-10-10 09:09:25","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":658361,"visible":true,"origin":"","legend":"","description":"","filename":"AnIncrementalEntityResolutionApproachbasedonDeepMetricLearning.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4982747/v1_covered_6bb3c7b5-201b-4d9b-a1f0-f1ec5bc58a62.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"An Incremental Entity Resolution Approach based on Deep Metric Learning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Incremental Entity Resolution, Deep Metric Learning, Dynamic data scenarios, Data Augmentation","lastPublishedDoi":"10.21203/rs.3.rs-4982747/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4982747/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Entity resolution (ER), a crucial component of data fusion, aims to identify and aggregate disparate entity descriptions describing the same real-world object. While existing approaches have achieved good performance in ER tasks, they fall short in scenarios with dynamic data sources. The inherent dynamism of these data sources requires ER models to progressively resolve entities with incoming data while maintaining high accuracy. To address such issue, we propose a novel dynamic data processing framework that leverages the matching model from the previous time step to parse incoming incremental data, and implement an Incremental Entity Resolution Approach based on Deep Metric Learning (IER-DML), developed to handle dynamic data and enhance matching capability. an uncertainty metric-based data augmentation mechanism is proposed to mitigate calibration issues caused by suboptimal training data, and to generate high-confidence labeled entity pairs. These labeled data are then used for the iterative optimization of IER-DML, enhancing ER capabilities of IER-DML. For modeling dynamic changes in the distributions of clustering structure among entity pairs, we propose a deep metric learning-based multi-center matcher. This matcher learns metric relationships between the deep semantic features of entity pairs, and models intrinsic structures using \"soft centers\". These centers capture multiple information clusters among entities, and flexibly represent clustering structures. By integrating deep metric learning with a multi-center strategy, it can dynamically adjust classification boundaries and metric standards in complex scenarios. Comprehensive experiments demonstrate that our approach outperforms state-of-the-art methods across all four dynamic benchmark datasets. ","manuscriptTitle":"An Incremental Entity Resolution Approach based on Deep Metric Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-09-27 12:05:34","doi":"10.21203/rs.3.rs-4982747/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"cf4423c5-3535-466d-a71f-96f3b9563382","owner":[],"postedDate":"September 27th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-11-29T10:23:23+00:00","versionOfRecord":[],"versionCreatedAt":"2024-09-27 12:05:34","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4982747","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4982747","identity":"rs-4982747","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}