Truth Inference for Crowdsourcing via Bias-Variance Fusion Graph Embedding

Truth Inference for Crowdsourcing via Bias-Variance Fusion Graph Embedding | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Truth Inference for Crowdsourcing via Bias-Variance Fusion Graph Embedding Wei Zong, Huize Feng, Zongyao Nie This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7637663/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Crowdsourcing has served as an effective method for data collection and labeling. However, the varying abilities of individual workers lead to inconsistent quality in the collected data. The existing truth inference methods always assign initial reliability values to workers and refine them iteratively. This methodology for estimating worker reliability fails to accurately assess worker reliability and consequently reduces the accuracy of truth inference. In this article, we explore a novel strategy to measure worker reliability by estimating the label quality of each worker for individual instances. Specifically, we introduce two metrics—bias and variance—to quantify, respectively, the discrepancy between a worker’s labels and those of other workers on similar instances (bias), and the inconsistency in a worker’s own labels across different instances (variance). These components are integrated to compute the label quality of each worker for inference instances, enabling accurate assessment of their actual labeling capability and determining their reliability. Building upon this, we propose Graph Embedding for Truth Inference with Worker Reliability (GETI-WR), a bias-variance fusion graph embedding method for crowdsourced truth inference. GETI-WR leverages the obtained worker reliability to map the "task-worker-label" graph into a low-dimensional continuous vector space. Then we use a graph neural network to transform the crowdsourcing problem into a graph node prediction task to enhance truth discovery efficiency.Experimental results on two real-world datasets demonstrate that compared to six state-of-the-art baseline methods, our method achieves superior performance in both accuracy and F1-score. Crowdsourcing Truth Inference Worker reliability Graph Embedding Convolutional Neural Network Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 08 Mar, 2026 Reviews received at journal 30 Oct, 2025 Reviewers agreed at journal 04 Oct, 2025 Reviewers invited by journal 03 Oct, 2025 Editor assigned by journal 25 Sep, 2025 Submission checks completed at journal 17 Sep, 2025 First submitted to journal 17 Sep, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7637663","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":525615314,"identity":"71dae738-4a1d-44ff-aafa-6c33f4f468e0","order_by":0,"name":"Wei Zong","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6UlEQVRIiWNgGAWjYFACxgYQKcfAzGBwACxwgEgtxqRogYBEoEYDBqK0GBxvbv7wcUdt+objzBsP/NzBIMd3I4HxcwE+LWcONhjOPHM8d2YzW8HB3jMMxpI3EpilZ+DRYnYjsSGZt+1Ybj8zj8EB3jaGxA03EtiYefBpuf+w4TBQSzpQmcHBv20M9YS13GBsbOZtq0ngB2oB6mVIMCCkxf5MYjPjzLYDhiC/HJZtkwB67GGzND4tku3HH3/42FYnb3D+8OaPb9ts5PmOJx/8jE8LFByGMSQYYJFLCNQRo2gUjIJRMApGKgAAYwlRc/5rHkMAAAAASUVORK5CYII=","orcid":"","institution":"Xidian University","correspondingAuthor":true,"prefix":"","firstName":"Wei","middleName":"","lastName":"Zong","suffix":""},{"id":525615315,"identity":"999b57d4-147a-435c-aa48-d33e9b2fe142","order_by":1,"name":"Huize Feng","email":"","orcid":"","institution":"Xidian University","correspondingAuthor":false,"prefix":"","firstName":"Huize","middleName":"","lastName":"Feng","suffix":""},{"id":525615316,"identity":"ed164522-51bf-4bb9-a625-4148edd0e6b9","order_by":2,"name":"Zongyao Nie","email":"","orcid":"","institution":"Xidian University","correspondingAuthor":false,"prefix":"","firstName":"Zongyao","middleName":"","lastName":"Nie","suffix":""}],"badges":[],"createdAt":"2025-09-17 08:23:48","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7637663/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7637663/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93651235,"identity":"22f73456-8013-4a22-a181-53ce19400171","added_by":"auto","created_at":"2025-10-16 05:52:56","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":345347,"visible":true,"origin":"","legend":"","description":"","filename":"KISTruthInferenceforCrowdsourcingviaBiasVarianceFusionGraphEmbedding3.docx","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/3e9f08e790530aca25dfe5fa.docx"},{"id":93650809,"identity":"5f8b258f-afc2-4923-9f9d-626c1a2b8237","added_by":"auto","created_at":"2025-10-16 05:44:56","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5164,"visible":true,"origin":"","legend":"","description":"","filename":"66ec7305f43841d990ad4649048ffd4d.json","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/9e41f9f3eb0528960b4e73aa.json"},{"id":93650813,"identity":"02c01cd5-6495-4a36-9cda-d08b8899f11c","added_by":"auto","created_at":"2025-10-16 05:44:56","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":131393,"visible":true,"origin":"","legend":"","description":"","filename":"66ec7305f43841d990ad4649048ffd4d1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/33d7f6a3d691918f3eced7df.xml"},{"id":93650094,"identity":"394c47f6-bcae-46df-8423-0d34294a339e","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"eps","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":164748,"visible":true,"origin":"","legend":"","description":"","filename":"drawingimage2.eps","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/abd4e621bd66598da3e8a32b.eps"},{"id":93650087,"identity":"71b62351-ccb8-4144-9d45-72d2ba3e5d3e","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"eps","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64516,"visible":true,"origin":"","legend":"","description":"","filename":"drawingimage3.eps","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/6a42b5e170de4aad6dbf976d.eps"},{"id":93650084,"identity":"3a989c45-c18c-4ea2-94fa-38843fe58fef","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"jpeg","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":3369,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/bde7c27f2fb0b2712fbaf734.jpeg"},{"id":93650089,"identity":"05ba49ad-1e3a-4b36-b532-2a49a77060c5","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":76175,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/1028d707e12d9bcba8f69219.png"},{"id":93650088,"identity":"0eac846c-da7f-401a-961e-49bb8b976666","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":42382,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/2429d2ed5d4f02bd531a5464.png"},{"id":93650085,"identity":"61995284-6d1c-446d-ba65-dadad7cf59f0","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":50759,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/24a928ad6b9a9b5f94c89289.png"},{"id":93650816,"identity":"b1cd71c3-c041-4b93-87f3-60901630b55a","added_by":"auto","created_at":"2025-10-16 05:44:56","extension":"jpeg","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":901966,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/26d198b81e4432974d6455c0.jpeg"},{"id":93650100,"identity":"fe4f8d98-bf9f-47bd-a55e-610e87956b54","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"jpeg","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1092143,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage6.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/e2422eeb05bd73cda96e6537.jpeg"},{"id":93651233,"identity":"c838036e-926a-4639-96b9-1f4f9e128e1e","added_by":"auto","created_at":"2025-10-16 05:52:56","extension":"jpeg","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1047582,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage7.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/64f1f4d8e5ade4bd664e1269.jpeg"},{"id":93650096,"identity":"1719b9d1-c511-4555-9471-bebe47fc6739","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":998,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/7264b0219032815fc2b95b7d.png"},{"id":93651863,"identity":"0f53a962-4f05-4064-912e-35091d9cd6ff","added_by":"auto","created_at":"2025-10-16 06:00:56","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":18637,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/1d0f1e32a56bc45989054e98.png"},{"id":93650815,"identity":"86d6221e-a543-4778-a9f5-d40af23b1a6d","added_by":"auto","created_at":"2025-10-16 05:44:56","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":16579,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/4f0172f8ab444f608f88ae79.png"},{"id":93650098,"identity":"ead7d817-58c8-4baf-b6c4-cd0ddb7e1b2c","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":17897,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/589d46723b8c2afeeefc03ac.png"},{"id":93650810,"identity":"6bd69efa-7039-484b-8816-4db29ac22565","added_by":"auto","created_at":"2025-10-16 05:44:56","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":65894,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/697adc0d8903fde2d5dd624f.png"},{"id":93650103,"identity":"2e43eb53-986f-4685-ad36-66003d380546","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":82484,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/90dcbb8a4aa2a5294d9a61a1.png"},{"id":93650099,"identity":"634c4ce6-71a7-4c9c-91db-21a35e9790aa","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"png","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":79692,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/03e9b15f02d5472de48cc99b.png"},{"id":93650104,"identity":"bc0baefb-854a-4aee-9126-6fd1b7319a72","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"xml","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":129821,"visible":true,"origin":"","legend":"","description":"","filename":"66ec7305f43841d990ad4649048ffd4d1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/3c8d8b5edfbebaa96b154811.xml"},{"id":93650102,"identity":"ba7e510e-1a4b-46df-9d8b-3557353ff619","added_by":"auto","created_at":"2025-10-16 05:36:56","extension":"html","order_by":21,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":157474,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1/f6045c0e8759c407abee67b4.html"},{"id":93652417,"identity":"b7ee7d99-8247-4bf1-b895-42ac7cd5a909","added_by":"auto","created_at":"2025-10-16 06:08:58","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":734659,"visible":true,"origin":"","legend":"","description":"","filename":"KISTruthInferenceforCrowdsourcingviaBiasVarianceFusionGraphEmbedding3.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7637663/v1_covered_12ce406b-6b56-447e-ba54-c69031659ff0.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Truth Inference for Crowdsourcing via Bias-Variance Fusion Graph Embedding","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"knowledge-and-information-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"kais","sideBox":"Learn more about [Knowledge and Information Systems](http://link.springer.com/journal/10115)","snPcode":"10115","submissionUrl":"https://submission.nature.com/new-submission/10115/3","title":"Knowledge and Information Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Crowdsourcing, Truth Inference, Worker reliability, Graph Embedding, Convolutional Neural Network","lastPublishedDoi":"10.21203/rs.3.rs-7637663/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7637663/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCrowdsourcing has served as an effective method for data collection and labeling. However, the varying abilities of individual workers lead to inconsistent quality in the collected data. The existing truth inference methods always assign initial reliability values to workers and refine them iteratively. This methodology for estimating worker reliability fails to accurately assess worker reliability and consequently reduces the accuracy of truth inference. In this article, we explore a novel strategy to measure worker reliability by estimating the label quality of each worker for individual instances. Specifically, we introduce two metrics\u0026mdash;bias and variance\u0026mdash;to quantify, respectively, the discrepancy between a worker\u0026rsquo;s labels and those of other workers on similar instances (bias), and the inconsistency in a worker\u0026rsquo;s own labels across different instances (variance). These components are integrated to compute the label quality of each worker for inference instances, enabling accurate assessment of their actual labeling capability and determining their reliability. Building upon this, we propose Graph Embedding for Truth Inference with Worker Reliability (GETI-WR), a bias-variance fusion graph embedding method for crowdsourced truth inference. GETI-WR leverages the obtained worker reliability to map the \"task-worker-label\" graph into a low-dimensional continuous vector space. Then we use a graph neural network to transform the crowdsourcing problem into a graph node prediction task to enhance truth discovery efficiency.Experimental results on two real-world datasets demonstrate that compared to six state-of-the-art baseline methods, our method achieves superior performance in both accuracy and F1-score.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e","manuscriptTitle":"Truth Inference for Crowdsourcing via Bias-Variance Fusion Graph Embedding","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-16 05:36:51","doi":"10.21203/rs.3.rs-7637663/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-08T16:54:25+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-31T01:24:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"5855272588788880706190324134181960597","date":"2025-10-04T22:35:32+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-10-03T13:54:10+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-25T13:38:36+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-17T10:18:19+00:00","index":"","fulltext":""},{"type":"submitted","content":"Knowledge and Information Systems","date":"2025-09-17T08:11:52+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"knowledge-and-information-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"kais","sideBox":"Learn more about [Knowledge and Information Systems](http://link.springer.com/journal/10115)","snPcode":"10115","submissionUrl":"https://submission.nature.com/new-submission/10115/3","title":"Knowledge and Information Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"b883c793-eefb-4f43-8930-0d2443e31f42","owner":[],"postedDate":"October 16th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-19T16:53:20+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-16 05:36:51","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7637663","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7637663","identity":"rs-7637663","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}