LLM-KGMQA: Large Language Model-Augmented Multi-Hop Question-Answering System based on Knowledge Graph in Medical Field

LLM-KGMQA: Large Language Model-Augmented Multi-Hop Question-Answering System based on Knowledge Graph in Medical Field | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article LLM-KGMQA: Large Language Model-Augmented Multi-Hop Question-Answering System based on Knowledge Graph in Medical Field FeiLong Wang, Donghui Shi, Jose Aguilar, Xinyi Cui, Jinsong Jiang, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4721418/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Apr, 2025 Read the published version in Knowledge and Information Systems → Version 1 posted 11 You are reading this latest preprint version Abstract In response to the problems of poor performance of large language models in specific domains, limited research on knowledge graphs and question-answering systems incorporating large language models, this paper proposed a multi-hop question-answering system framework based on a knowledge graph in the medical field, which was fully augmented by large language models (LLM-KGMQA). The method primarily addressed the problems of entity linking and multi-hop knowledge path reasoning. To address the entity linking problem, an entity fast-linking algorithm was proposed, which categorized entities based on multiple attributes. Then, it used user mentions to obtain the target attribute set of attributes and further narrowed the entity search scope through attribute intersection operations. Finally, for entities that remained too numerous after the intersection, the method suggested using a pre-trained model for similarity calculation and ranking, and to determine the final entity through construction instructions. Regarding multi-hop knowledge path reasoning, the paper proposed a three-step reasoning framework that included an n-hop subgraph construction algorithm, a knowledge fusion algorithm, and a semantics-based knowledge pruning algorithm. In the entity fast-linking experiments, the maximum computational complexity was reduced by 99.9% through intersection operations. Additionally, a new evaluation metric, top@n, was introduced. When using the Roberta model for similarity calculations, the top@n score reached a maximum of 96.4, and the entity fast-linking accuracy was 96.6%. In multi-hop knowledge path reasoning, the paper first validated the need for knowledge fusion by constructing three different forms of instructions. Subsequently, experiments were conducted with several large language models, concluded that the GLM4 model showed the best performance in Chinese semantic reasoning. The accuracy rates for GLM4 after pruning were 99.9%, 83.3%, and 86.6% for 1-hop, 2-hop, and 3-hop, respectively, compared to 95.0%, 6.6%, and 5.0% before pruning. The average response time was reduced by 1.36s, 6.21s and 27.07s after pruning compared to before pruning. Knowledge Graph Large Language Model Entity Linking Multi-Hop Knowledge Reasoning question-answering System Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 21 Apr, 2025 Read the published version in Knowledge and Information Systems → Version 1 posted Editorial decision: Revision requested 30 Nov, 2024 Reviews received at journal 23 Nov, 2024 Reviewers agreed at journal 27 Oct, 2024 Reviews received at journal 08 Sep, 2024 Reviewers agreed at journal 31 Aug, 2024 Reviewers agreed at journal 14 Aug, 2024 Reviewers agreed at journal 14 Aug, 2024 Reviewers invited by journal 12 Aug, 2024 Editor assigned by journal 19 Jul, 2024 Submission checks completed at journal 11 Jul, 2024 First submitted to journal 10 Jul, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4721418","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":332405807,"identity":"cbb61b89-e5bc-4d72-84f3-669d5452841f","order_by":0,"name":"FeiLong Wang","email":"","orcid":"","institution":"Anhui Jianzhu University","correspondingAuthor":false,"prefix":"","firstName":"FeiLong","middleName":"","lastName":"Wang","suffix":""},{"id":332405808,"identity":"085b27f6-32e7-4078-b1b1-fd770947af36","order_by":1,"name":"Donghui Shi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/UlEQVRIiWNgGAWjYDACCcYGCSCVwMDMfICZgQcsZkCsFrYEYrWAEVALA48BM1QMvxb52c2NNz7uqM0zOM7z8XOBTF1iA3vzNgmGmjs4tRjcOdhsOfPM8WKDw7ybpWfwsCU28Bwrk2A49gy3FonENmnetmOJGw7zbpDm4eFJbJDIMQN68DBuh82Aa+F5/JuHRyKxQf4Nfi0MN8BaakBa2IC2GABt4cGvxeBGItAvbQcSZx5mM7Pm4UkwbuNJK7ZIOIbPYekPb3xsq0vsO3/48W3enjrZfvbDG298qMHjMAiAKmDsYWBgAzESCGlgYKiD0j8IKx0Fo2AUjIKRBwBqmFSkB3nKOQAAAABJRU5ErkJggg==","orcid":"","institution":"Anhui Jianzhu University","correspondingAuthor":true,"prefix":"","firstName":"Donghui","middleName":"","lastName":"Shi","suffix":""},{"id":332405809,"identity":"17ccb84d-c64a-4532-8117-7e863757a5ca","order_by":2,"name":"Jose Aguilar","email":"","orcid":"","institution":"IMDEA Networks Institute","correspondingAuthor":false,"prefix":"","firstName":"Jose","middleName":"","lastName":"Aguilar","suffix":""},{"id":332405810,"identity":"f92840d5-6a94-4073-8d84-b7ebec1d1c04","order_by":3,"name":"Xinyi Cui","email":"","orcid":"","institution":"Anhui Jianzhu University","correspondingAuthor":false,"prefix":"","firstName":"Xinyi","middleName":"","lastName":"Cui","suffix":""},{"id":332405811,"identity":"53cb0118-7313-4c33-928f-9f61b790dfd4","order_by":4,"name":"Jinsong Jiang","email":"","orcid":"","institution":"Anhui Jianzhu University","correspondingAuthor":false,"prefix":"","firstName":"Jinsong","middleName":"","lastName":"Jiang","suffix":""},{"id":332405812,"identity":"b76e07a4-6945-4617-9f64-b272bdea21e9","order_by":5,"name":"Longjian Shen","email":"","orcid":"","institution":"Anhui Jianzhu University","correspondingAuthor":false,"prefix":"","firstName":"Longjian","middleName":"","lastName":"Shen","suffix":""},{"id":332405813,"identity":"27db8e40-bc9b-469a-b1b8-a6c0bf912773","order_by":6,"name":"Mengya Li","email":"","orcid":"","institution":"Anhui Jianzhu University","correspondingAuthor":false,"prefix":"","firstName":"Mengya","middleName":"","lastName":"Li","suffix":""}],"badges":[],"createdAt":"2024-07-11 03:15:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4721418/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4721418/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10115-025-02399-1","type":"published","date":"2025-04-21T15:58:23+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":81569773,"identity":"b4aadbc4-0275-4bfb-b159-601dc36528b0","added_by":"auto","created_at":"2025-04-28 16:11:03","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":731158,"visible":true,"origin":"","legend":"","description":"","filename":"LLMKGMQA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4721418/v1_covered_c51bab81-657b-4db1-aad1-e66a0e2e8de1.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"LLM-KGMQA: Large Language Model-Augmented Multi-Hop Question-Answering System based on Knowledge Graph in Medical Field","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"knowledge-and-information-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"kais","sideBox":"Learn more about [Knowledge and Information Systems](http://link.springer.com/journal/10115)","snPcode":"10115","submissionUrl":"https://submission.nature.com/new-submission/10115/3","title":"Knowledge and Information Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Knowledge Graph, Large Language Model, Entity Linking, Multi-Hop Knowledge Reasoning, question-answering System","lastPublishedDoi":"10.21203/rs.3.rs-4721418/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4721418/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn response to the problems of poor performance of large language models in specific domains, limited research on knowledge graphs and question-answering systems incorporating large language models, this paper proposed a multi-hop question-answering system framework based on a knowledge graph in the medical field, which was fully augmented by large language models (LLM-KGMQA). The method primarily addressed the problems of entity linking and multi-hop knowledge path reasoning. To address the entity linking problem, an entity fast-linking algorithm was proposed, which categorized entities based on multiple attributes. Then, it used user mentions to obtain the target attribute set of attributes and further narrowed the entity search scope through attribute intersection operations. Finally, for entities that remained too numerous after the intersection, the method suggested using a pre-trained model for similarity calculation and ranking, and to determine the final entity through construction instructions. Regarding multi-hop knowledge path reasoning, the paper proposed a three-step reasoning framework that included an n-hop subgraph construction algorithm, a knowledge fusion algorithm, and a semantics-based knowledge pruning algorithm. In the entity fast-linking experiments, the maximum computational complexity was reduced by 99.9% through intersection operations. Additionally, a new evaluation metric, top@n, was introduced. When using the Roberta model for similarity calculations, the top@n score reached a maximum of 96.4, and the entity fast-linking accuracy was 96.6%. In multi-hop knowledge path reasoning, the paper first validated the need for knowledge fusion by constructing three different forms of instructions. Subsequently, experiments were conducted with several large language models, concluded that the GLM4 model showed the best performance in Chinese semantic reasoning. The accuracy rates for GLM4 after pruning were 99.9%, 83.3%, and 86.6% for 1-hop, 2-hop, and 3-hop, respectively, compared to 95.0%, 6.6%, and 5.0% before pruning. The average response time was reduced by 1.36s, 6.21s and 27.07s after pruning compared to before pruning.\u003c/p\u003e","manuscriptTitle":"LLM-KGMQA: Large Language Model-Augmented Multi-Hop Question-Answering System based on Knowledge Graph in Medical Field","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-05 03:25:29","doi":"10.21203/rs.3.rs-4721418/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-12-01T04:33:25+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-23T17:05:51+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"266735158248953704482214255377580889658","date":"2024-10-27T04:15:29+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-09-08T04:53:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"201578170873732435872215375794583601172","date":"2024-08-31T15:52:19+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"7455551844356231168751613842296923552","date":"2024-08-15T00:50:38+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"91941519987882675025010373109799200242","date":"2024-08-14T15:05:26+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-08-12T12:06:58+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-07-19T04:13:09+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-07-11T08:38:07+00:00","index":"","fulltext":""},{"type":"submitted","content":"Knowledge and Information Systems","date":"2024-07-11T03:13:52+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"knowledge-and-information-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"kais","sideBox":"Learn more about [Knowledge and Information Systems](http://link.springer.com/journal/10115)","snPcode":"10115","submissionUrl":"https://submission.nature.com/new-submission/10115/3","title":"Knowledge and Information Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"c20ce331-2e53-4634-8d81-1cc0a0e2593c","owner":[],"postedDate":"August 5th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-04-28T16:04:06+00:00","versionOfRecord":{"articleIdentity":"rs-4721418","link":"https://doi.org/10.1007/s10115-025-02399-1","journal":{"identity":"knowledge-and-information-systems","isVorOnly":false,"title":"Knowledge and Information Systems"},"publishedOn":"2025-04-21 15:58:23","publishedOnDateReadable":"April 21st, 2025"},"versionCreatedAt":"2024-08-05 03:25:29","video":"","vorDoi":"10.1007/s10115-025-02399-1","vorDoiUrl":"https://doi.org/10.1007/s10115-025-02399-1","workflowStages":[]},"version":"v1","identity":"rs-4721418","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4721418","identity":"rs-4721418","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}