Counterfactual Divergence Singularity: A Theoretical Model of High-Similarity Instability and Micro-Counterfactual Drift in Large Language Models

Counterfactual Divergence Singularity: A Theoretical Model of High-Similarity Instability and Micro-Counterfactual Drift in Large Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Counterfactual Divergence Singularity: A Theoretical Model of High-Similarity Instability and Micro-Counterfactual Drift in Large Language Models Som Subhro Nath This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8368276/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Large language models (LLMs) demonstrate remarkable fluency and contextual sensitivity, yet their behaviour under minimal semantic perturbations remains poorly understood. In particular, near-identical paraphrastic inputs—expected to yield stable and equivalent responses—often produce disproportionately divergent generative behaviour. This paper introduces the Counterfactual Divergence Singularity (CDS) , a theoretical and empirical framework that characterizes a previously underexplored instability regime in transformer-based language models. This work formalizes micro-counterfactual perturbations as infinitesimal semantic variations in embedding space and show that, as semantic similarity approaches unity, divergence metrics exhibit a hyperbolic blow-up. Using a controlled experimental pipeline based on FLAN-T5-generated paraphrases and Sentence-BERT semantic embeddings, this paper empirically demonstrate that divergence, curvature, and embedding drift collectively reveal a singular boundary where semantic proximity no longer guarantees behavioural stability. Despite minimal embedding displacement and near-maximal cosine similarity, reciprocal divergence measures increase sharply, exposing a structural nonlinearity in the input–output mapping of LLMs. The results suggest that LLM semantic manifolds are locally non-smooth and that standard embedding-based similarity metrics fail to capture instability near the identity boundary. This phenomenon provides a geometric and mathematical explanation for prompt sensitivity, counterfactual failure, and certain forms of hallucination observed in generative systems. By reframing these behaviours as consequences of latent-space singularities rather than isolated decoding artifacts, this work contributes a novel theoretical lens for evaluating robustness, interpretability, and reliability in large language models. Artificial Intelligence and Machine Learning Counterfactual Divergence Singularity Micro-counterfactual perturbations Large Language Models Semantic instability Embedding geometry Paraphrase sensitivity Hallucination analysis Semantic curvature Representation drift Generative robustness Full Text Additional Declarations The authors declare no competing interests. Supplementary Files code.py Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8368276","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":560697795,"identity":"2f87fff9-0449-4aa6-ae03-9743824287d3","order_by":0,"name":"Som Subhro Nath","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+UlEQVRIiWNgGAWjYHACNgglwcPAwPjPBshibDyAXwczkhYGtjSQlgaStBwGs/FqMWc/f+zhj4pt9ga3e499+MBz3m5t+2GgLTU20bi0WPYksxvznLmduOHOueSZMyRuJ287kwjUciwttwGHFoMDyWzSjG23Ewxu5Bgz8xjcTjY7ANTC2HAYt5bzj9kkf7bdtodoSTiXbHb+IQEtN5LZJHjbbjNuAGs5cMDO7AYhW248NpMG+WXmnTPGjDMbkhPMbgBtScDnl/OJzyR/VNy257vdY8zwscHO3ux8+sMHH2pscGrBAIlglQnEKgcBe1IUj4JRMApGwcgAAETJZRj/OVUPAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0009-0007-6842-0019","institution":"","correspondingAuthor":true,"prefix":"","firstName":"Som","middleName":"Subhro","lastName":"Nath","suffix":""}],"badges":[],"createdAt":"2025-12-15 16:14:11","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8368276/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8368276/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":98398080,"identity":"474c7a6d-566d-445a-b38a-52fad6d289bf","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":244117,"visible":true,"origin":"","legend":"","description":"","filename":"CounterfactualDivergenceSingularity.docx","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/34047e87a57b29ed3f5a4c73.docx"},{"id":98440622,"identity":"5e5db6ac-cf96-4383-82bf-587d91583e47","added_by":"auto","created_at":"2025-12-17 17:04:06","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":342,"visible":true,"origin":"","legend":"","description":"","filename":"rs8368276.json","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/6672162a6c07e91b7fe1300c.json"},{"id":98441648,"identity":"d6ca31af-137c-4158-8cdd-8a67676372fe","added_by":"auto","created_at":"2025-12-17 17:05:40","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":85927,"visible":true,"origin":"","legend":"","description":"","filename":"rs83682760enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/581aac857f702264ae3ef41c.xml"},{"id":98398075,"identity":"b52312e0-8ca7-45a1-b89a-82761794659e","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":36510,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/9e8c72303ffa8c2fa8d0f1b3.png"},{"id":98440788,"identity":"10605de5-46f1-427d-81d3-80aabd0a423c","added_by":"auto","created_at":"2025-12-17 17:04:21","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":57062,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/982d8369f87ac291e8df488e.png"},{"id":98439653,"identity":"7b7e54fc-1720-4b9b-8d62-89c111ca7e6d","added_by":"auto","created_at":"2025-12-17 17:02:27","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":37386,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/687fbface8b30e0a45b87c50.png"},{"id":98398085,"identity":"f1da85e6-b5db-4c21-82b8-f04ab87b3ee1","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":53546,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/7c208425c86dfd715e9de9c4.png"},{"id":98398078,"identity":"9aa0f7c2-11a4-4c5b-bac3-0cd8f1bbb427","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9840,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/d40d4f68ae20cfbe8a1cbb9d.png"},{"id":98440985,"identity":"7286824b-078f-4248-b81c-4166da54af36","added_by":"auto","created_at":"2025-12-17 17:04:43","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14126,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/056226a8dfd2061ecb305aaf.png"},{"id":98398087,"identity":"fafa46f7-f9ed-49a5-be37-c56fdc47d828","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8967,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/53d290251e6b14489e28cbea.png"},{"id":98398084,"identity":"9ce0c647-a9d2-4e8f-95e5-0b7a55a378bc","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":12555,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/02e7148e88c35db8b2e54dab.png"},{"id":98441560,"identity":"fad41e27-9b24-4564-8c83-99be49f83f2b","added_by":"auto","created_at":"2025-12-17 17:05:35","extension":"xml","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":85343,"visible":true,"origin":"","legend":"","description":"","filename":"rs83682760structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/81a4d692bcf2623fe37f62f4.xml"},{"id":98441019,"identity":"a2e330d5-ecb7-437c-9a20-7757d67c7c5c","added_by":"auto","created_at":"2025-12-17 17:04:46","extension":"html","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":104166,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/ef5a6de4766837bc42441891.html"},{"id":98445851,"identity":"4469f374-4e79-45ef-bc12-fdab8e8f57ad","added_by":"auto","created_at":"2025-12-17 17:22:01","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":561337,"visible":true,"origin":"","legend":"","description":"","filename":"CounterfactualDivergenceSingularity.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1_covered_95f16b20-4eda-4cc0-9584-2bceb3d9b255.pdf"},{"id":98398074,"identity":"72ad2e79-6faa-4bef-aa90-78db74a6cce9","added_by":"auto","created_at":"2025-12-17 10:58:02","extension":"py","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":10778,"visible":true,"origin":"","legend":"","description":"","filename":"code.py","url":"https://assets-eu.researchsquare.com/files/rs-8368276/v1/601b356bb6e00863795ce1ca.py"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eCounterfactual Divergence Singularity: A Theoretical Model of High-Similarity Instability and Micro-Counterfactual Drift in Large Language Models\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Independent Researcher","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Counterfactual Divergence Singularity, Micro-counterfactual perturbations, Large Language Models, Semantic instability, Embedding geometry, Paraphrase sensitivity, Hallucination analysis, Semantic curvature, Representation drift, Generative robustness","lastPublishedDoi":"10.21203/rs.3.rs-8368276/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8368276/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eLarge language models (LLMs) demonstrate remarkable fluency and contextual sensitivity, yet their behaviour under minimal semantic perturbations remains poorly understood. In particular, near-identical paraphrastic inputs\u0026mdash;expected to yield stable and equivalent responses\u0026mdash;often produce disproportionately divergent generative behaviour. This paper introduces the \u003cb\u003eCounterfactual Divergence Singularity (CDS)\u003c/b\u003e, a theoretical and empirical framework that characterizes a previously underexplored instability regime in transformer-based language models.\u003c/p\u003e \u003cp\u003eThis work formalizes micro-counterfactual perturbations as infinitesimal semantic variations in embedding space and show that, as semantic similarity approaches unity, divergence metrics exhibit a hyperbolic blow-up. Using a controlled experimental pipeline based on FLAN-T5-generated paraphrases and Sentence-BERT semantic embeddings, this paper empirically demonstrate that divergence, curvature, and embedding drift collectively reveal a singular boundary where semantic proximity no longer guarantees behavioural stability. Despite minimal embedding displacement and near-maximal cosine similarity, reciprocal divergence measures increase sharply, exposing a structural nonlinearity in the input\u0026ndash;output mapping of LLMs.\u003c/p\u003e \u003cp\u003eThe results suggest that LLM semantic manifolds are locally non-smooth and that standard embedding-based similarity metrics fail to capture instability near the identity boundary. This phenomenon provides a geometric and mathematical explanation for prompt sensitivity, counterfactual failure, and certain forms of hallucination observed in generative systems. By reframing these behaviours as consequences of latent-space singularities rather than isolated decoding artifacts, this work contributes a novel theoretical lens for evaluating robustness, interpretability, and reliability in large language models.\u003c/p\u003e","manuscriptTitle":"Counterfactual Divergence Singularity: A Theoretical Model of High-Similarity Instability and Micro-Counterfactual Drift in Large Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-17 10:57:57","doi":"10.21203/rs.3.rs-8368276/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d1a4fc50-ef56-4c34-a001-ce5bece9a2e7","owner":[],"postedDate":"December 17th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":59693188,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-12-17T10:57:57+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-17 10:57:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8368276","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8368276","identity":"rs-8368276","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}