Combined GNN specialized in inductive prediction and PLM for natural language inductive reasoning

Combined GNN specialized in inductive prediction and PLM for natural language inductive reasoning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Combined GNN specialized in inductive prediction and PLM for natural language inductive reasoning Koki Tomei, Masafumi Hagiwara This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6440243/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 15 Sep, 2025 Read the published version in Journal of Intelligent Information Systems → Version 1 posted 11 You are reading this latest preprint version Abstract Inductive reasoning involves abstracting general principles from specific instances. It primarily relies on rules derived from relationships and is minimally dependent on specific data about entities, such as people, places, or organizations.Pre-trained language models (PLM) tend to focus on learning statistical features within a corpus. Thus, when inductive reasoning is expressed in a natural language, PLMs face challenges in learning the logical relations behind the text. Recently, researchers have explored graph neural network (GNN) architectures that excel in inductive inference on knowledge graphs (KGs) for inductive link prediction tasks; however, their application to natural language remains limited. To address the natural language inductive reasoning tasks, we propose a framework that utilizes a specific GNN module specialized in inductive link prediction as a reasoning mechanism. To construct inputs inferable to GNN from natural language, we first apply insights from a study of relation extraction tasks and use PLM to obtain embeddings for edge-related inferences. Subsequently, the newly designed module performs edge scoring and initializes the relation embeddings. The scores are used to prune the edges and are learned through edge weighting within the decoder. Experimental results on text datasets requiring logical inductive reasoning demonstrate that the proposed method notably improves PLM performance, outperforming the baselines. Furthermore, robustness evaluation on subsets provided by CLUTRR shows that our model surpasses other relational reasoning-based models in its ability to learn from and generalize noisy data. Deep learning Graph neural network Inductive reasoning Natural language processing Inductive relation prediction Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 15 Sep, 2025 Read the published version in Journal of Intelligent Information Systems → Version 1 posted Editorial decision: Revision requested 23 May, 2025 Reviews received at journal 23 May, 2025 Reviews received at journal 11 May, 2025 Reviews received at journal 28 Apr, 2025 Reviewers agreed at journal 26 Apr, 2025 Reviewers agreed at journal 26 Apr, 2025 Reviewers agreed at journal 25 Apr, 2025 Reviewers invited by journal 25 Apr, 2025 Editor assigned by journal 22 Apr, 2025 Submission checks completed at journal 22 Apr, 2025 First submitted to journal 13 Apr, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6440243","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":449280220,"identity":"4b725cf5-5014-44bc-bfb5-d1612b28d44c","order_by":0,"name":"Koki Tomei","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0ElEQVRIie3QMQqDMBSA4ScBu0S6RgR7hYhrbtJFEdJJ6BGEgl0EV0t7iIIHaOCBPYgXELo4ONTapV1qupWSf3rLx3sJgMn0gzGwQU0TIQjAAaxMi0QPYif6BCYCNNQ7zN3nsbJ6sV4u6I13WwHkOLPGo40aD5PpYefUccUlWCf1mfhsk40E0zM6NVKOYFWRJrkgbXHQIR6T6rmFUJKADnGLJlKRlGmFdhgUXNLZt7BrHnadEGlZYsv6QfjB3I9NvV5Cg0pDvLdiXxOTyWT68+7XHUMAhZLd8QAAAABJRU5ErkJggg==","orcid":"","institution":"Keio University","correspondingAuthor":true,"prefix":"","firstName":"Koki","middleName":"","lastName":"Tomei","suffix":""},{"id":449280221,"identity":"5d983c23-c5c2-451a-a561-661ca0767e83","order_by":1,"name":"Masafumi Hagiwara","email":"","orcid":"","institution":"Keio University","correspondingAuthor":false,"prefix":"","firstName":"Masafumi","middleName":"","lastName":"Hagiwara","suffix":""}],"badges":[],"createdAt":"2025-04-13 16:08:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6440243/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6440243/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10844-025-00979-9","type":"published","date":"2025-09-15T15:57:21+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":91890001,"identity":"0c747767-d88e-4069-b82b-6aacf644af14","added_by":"auto","created_at":"2025-09-22 16:03:43","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2872190,"visible":true,"origin":"","legend":"","description":"","filename":"ktomeiJIIStechnicalChecked.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6440243/v1_covered_64ac8c89-d322-46d5-bd03-c7077bc710b2.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Combined GNN specialized in inductive prediction and PLM for natural language inductive reasoning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"journal-of-intelligent-information-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jiis","sideBox":"Learn more about [Journal of Intelligent Information Systems](http://link.springer.com/journal/10844)","snPcode":"10844","submissionUrl":"https://submission.nature.com/new-submission/10844/3","title":"Journal of Intelligent Information Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Deep learning, Graph neural network, Inductive reasoning, Natural language processing, Inductive relation prediction","lastPublishedDoi":"10.21203/rs.3.rs-6440243/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6440243/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Inductive reasoning involves abstracting general principles from specific instances. It primarily relies on rules derived from relationships and is minimally dependent on specific data about entities, such as people, places, or organizations.Pre-trained language models (PLM) tend to focus on learning statistical features within a corpus. Thus, when inductive reasoning is expressed in a natural language, PLMs face challenges in learning the logical relations behind the text. Recently, researchers have explored graph neural network (GNN) architectures that excel in inductive inference on knowledge graphs (KGs) for inductive link prediction tasks; however, their application to natural language remains limited. To address the natural language inductive reasoning tasks, we propose a framework that utilizes a specific GNN module specialized in inductive link prediction as a reasoning mechanism. To construct inputs inferable to GNN from natural language, we first apply insights from a study of relation extraction tasks and use PLM to obtain embeddings for edge-related inferences. Subsequently, the newly designed module performs edge scoring and initializes the relation embeddings. The scores are used to prune the edges and are learned through edge weighting within the decoder. Experimental results on text datasets requiring logical inductive reasoning demonstrate that the proposed method notably improves PLM performance, outperforming the baselines. Furthermore, robustness evaluation on subsets provided by CLUTRR shows that our model surpasses other relational reasoning-based models in its ability to learn from and generalize noisy data. ","manuscriptTitle":"Combined GNN specialized in inductive prediction and PLM for natural language inductive reasoning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-29 09:58:37","doi":"10.21203/rs.3.rs-6440243/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-05-23T05:37:06+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-23T04:12:15+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-11T09:22:06+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-28T18:21:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"28361319553961326406891075047656843501","date":"2025-04-27T01:16:15+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"266493712987371394719865092466458613113","date":"2025-04-26T06:32:55+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"311317841760218914160974460491673072805","date":"2025-04-25T17:21:05+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-04-25T17:06:20+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-04-22T23:49:16+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-04-22T17:11:56+00:00","index":"","fulltext":""},{"type":"submitted","content":"Journal of Intelligent Information Systems","date":"2025-04-13T16:01:47+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-intelligent-information-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jiis","sideBox":"Learn more about [Journal of Intelligent Information Systems](http://link.springer.com/journal/10844)","snPcode":"10844","submissionUrl":"https://submission.nature.com/new-submission/10844/3","title":"Journal of Intelligent Information Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"88f87713-c668-4817-a316-aab2d8330483","owner":[],"postedDate":"April 29th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-09-22T16:02:46+00:00","versionOfRecord":{"articleIdentity":"rs-6440243","link":"https://doi.org/10.1007/s10844-025-00979-9","journal":{"identity":"journal-of-intelligent-information-systems","isVorOnly":false,"title":"Journal of Intelligent Information Systems"},"publishedOn":"2025-09-15 15:57:21","publishedOnDateReadable":"September 15th, 2025"},"versionCreatedAt":"2025-04-29 09:58:37","video":"","vorDoi":"10.1007/s10844-025-00979-9","vorDoiUrl":"https://doi.org/10.1007/s10844-025-00979-9","workflowStages":[]},"version":"v1","identity":"rs-6440243","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6440243","identity":"rs-6440243","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}