Estimation of a Countably Infinite-Dimensional Transition Probability Matrix Using a Stick-Breaking Prior

Estimation of a Countably Infinite-Dimensional Transition Probability Matrix Using a Stick-Breaking Prior | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Estimation of a Countably Infinite-Dimensional Transition Probability Matrix Using a Stick-Breaking Prior Souvik Roy, Agamani Saha This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8603162/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 12 You are reading this latest preprint version Abstract We consider the problem of estimation of the transition probability matrix (TPM) of a Markov chain where the state space is countably infinite. At present, there is no methodology in the literature for direct estimation of infinite-dimensional TPMs. Standard approaches, including maximum likelihood estimation and Bayesian methods based on Dirichlet priors, are inherently finite-state and, when applied in this setting, assign zero probability to all unobserved transitions. To derive a method for estimation of infinite-dimensional TPMs, we propose a Bayesian nonparametric approach that employs a random measure prior. Specifically, we utilize hierarchical and generalized hierarchical stick-breaking processes as priors on the rows of the TPM. The generalized hierarchical stick-breaking prior, in particular, allow for both positive and negative correlations among any pair of transition probabilities, thereby enabling more realistic modeling of the dependence across the state transitions. We develop a blocked Gibbs sampling algorithm for posterior computation under a generalized hierarchical stick-breaking prior that is fast, highly efficient, and well suited to large-scale problems. The proposed method is evaluated through extensive simulations as well as applications to real-world datasets on historical trading volume for the United States Oil Fund (USO), and daily precipitation records from Heathrow, London. The empirical results demonstrate strong and stable predictive performance in all scenarios considered. Bayesian Nonparametric Transition Probability Matrix Dirichlet Process Generalized Hierarchical Stick-Breaking Process Full Text Additional Declarations No competing interests reported. Supplementary Files SupplementaryStatisticsandComputing.pdf Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 31 Mar, 2026 Reviews received at journal 13 Mar, 2026 Reviews received at journal 05 Feb, 2026 Reviewers agreed at journal 02 Feb, 2026 Reviewers agreed at journal 30 Jan, 2026 Reviewers agreed at journal 22 Jan, 2026 Reviews received at journal 22 Jan, 2026 Reviewers agreed at journal 21 Jan, 2026 Reviewers invited by journal 20 Jan, 2026 Editor assigned by journal 17 Jan, 2026 Submission checks completed at journal 16 Jan, 2026 First submitted to journal 14 Jan, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8603162","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":578352592,"identity":"a02ebeda-cdd2-4364-add0-c9291c0a33fc","order_by":0,"name":"Souvik Roy","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABIklEQVRIiWNgGAWjYLACxgYEm0eCHcQzsMCv5SCKFp4DIC0SxGthkJBIAFM4Vcu7n058/HGHjZz5jATmDx/bDstIznx+dcOPAgkG/vbuBGxaDM/kbjY4eCbNWOZGApvkzLbDPNLSOWU3e4AOkzhzdgNWLQ252yQOth1OnCGRwMbM25bGIyedk3aDB6jFQCIXu5b+t9t/HGz7D9LC/BmsRfJM2s0/eLTIS+RuYzjYdgCkhUGat82GR1qC/dhtfLYYSLzdLHG2LdlYgudhm+SMczY8kj05bLdlDCR4cPlFvj9344fKNjs5Cfbkwx8+lEnYSxw//uzmmz82cvztvdhtOQBjCSQ2QFk8BmASm3KwLTB1DPxwzewPcKkeBaNgFIyCkQkA08tjZSDg9UEAAAAASUVORK5CYII=","orcid":"","institution":"Indian Statistical Institute","correspondingAuthor":true,"prefix":"","firstName":"Souvik","middleName":"","lastName":"Roy","suffix":""},{"id":578352593,"identity":"3b4b4c1c-e7ab-4788-b9de-626ef64fcaf1","order_by":1,"name":"Agamani Saha","email":"","orcid":"","institution":"Indian Statistical Institute","correspondingAuthor":false,"prefix":"","firstName":"Agamani","middleName":"","lastName":"Saha","suffix":""}],"badges":[],"createdAt":"2026-01-14 15:08:44","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8603162/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8603162/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":100951120,"identity":"72b253ed-76a8-46b2-87ae-34aa7f40245e","added_by":"auto","created_at":"2026-01-23 07:10:00","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4858,"visible":true,"origin":"","legend":"","description":"","filename":"1adba1b4d7954121a5ccc951d4626a5a.json","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/b640fd250b635232b16863ba.json"},{"id":100896690,"identity":"cad81fed-1dff-45a2-9e45-386d36fe2552","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"xml","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":123135,"visible":true,"origin":"","legend":"","description":"","filename":"1adba1b4d7954121a5ccc951d4626a5a1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/0f4efd141706fac88a5dd335.xml"},{"id":100950985,"identity":"b2cdc4ce-9e19-4f8a-b8d5-d9b47b4f2d84","added_by":"auto","created_at":"2026-01-23 07:09:45","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":482640,"visible":true,"origin":"","legend":"","description":"","filename":"EstimationofTPMStatisticsandComputing.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/6803de3b04af95e8271033fe.pdf"},{"id":100950363,"identity":"918c021e-77f7-46c6-8db2-509c757be5e4","added_by":"auto","created_at":"2026-01-23 07:07:49","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5191,"visible":true,"origin":"","legend":"","description":"","filename":"PoissonConc.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/d300ebef374500b3bb60bf42.pdf"},{"id":100896694,"identity":"07260bbf-95e5-4195-961b-8e83c9b19d50","added_by":"auto","created_at":"2026-01-22 14:17:07","extension":"pdf","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1374851,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryStatisticsandComputing.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/28b8db203679fc89184add09.pdf"},{"id":100950422,"identity":"c8c616d1-959c-41a7-8fee-b6d41c743882","added_by":"auto","created_at":"2026-01-23 07:08:03","extension":"eps","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":2890,"visible":true,"origin":"","legend":"","description":"","filename":"empty.eps","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/05476595be336e3e8772bc68.eps"},{"id":100896689,"identity":"6f3ee512-f02e-4e52-b982-b78d313a79e1","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"eps","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":91593,"visible":true,"origin":"","legend":"","description":"","filename":"fig.eps","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/1b74362100a44e776f7020fa.eps"},{"id":100950310,"identity":"dd2a1479-f26a-41a5-9f03-d605837cd9d3","added_by":"auto","created_at":"2026-01-23 07:07:36","extension":"bst","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":146013,"visible":true,"origin":"","legend":"","description":"","filename":"snapacite.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/160ed477bf47da7a58e5f9dd.bst"},{"id":100896678,"identity":"7e7f24a3-fc81-4268-beeb-fc957a87855f","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"bst","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":29828,"visible":true,"origin":"","legend":"","description":"","filename":"snaps.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/2a677aca448d3c656ccd9d83.bst"},{"id":100896683,"identity":"6f30f001-8cce-4c9d-aea8-b69e76b4a48d","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"pdf","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":421391,"visible":true,"origin":"","legend":"","description":"","filename":"snarticle.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/50a12cbb3ebc1e190f7a45c7.pdf"},{"id":100950433,"identity":"66f65946-390f-4197-83e9-f2e95c77c110","added_by":"auto","created_at":"2026-01-23 07:08:09","extension":"bst","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":35515,"visible":true,"origin":"","legend":"","description":"","filename":"snbasic.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/7adbb95f6ea837466d9d2144.bst"},{"id":101202496,"identity":"8aae66e5-850e-4914-920d-627b835b6e6c","added_by":"auto","created_at":"2026-01-27 09:35:15","extension":"bst","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":33968,"visible":true,"origin":"","legend":"","description":"","filename":"snchicago.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/796efe1a5d1d3e74680d640b.bst"},{"id":100896680,"identity":"e134c9b0-c730-40dc-97a8-7d359a6cb94c","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"cls","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":55857,"visible":true,"origin":"","legend":"","description":"","filename":"snjnl.cls","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/f8c871cfc62c28775ad2395c.cls"},{"id":100896684,"identity":"5912a5ec-d4d7-4ab5-b3b9-b22b6a900c5b","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"bst","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64023,"visible":true,"origin":"","legend":"","description":"","filename":"snmathphysay.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/1b3fceafb7c3066f079c46bd.bst"},{"id":100896682,"identity":"4227e66c-2e2d-4669-8ee6-2ec4276ca387","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"bst","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64166,"visible":true,"origin":"","legend":"","description":"","filename":"snmathphysnum.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/91c05a26b8568da22cb7375f.bst"},{"id":100896687,"identity":"24cc4c8e-c3b6-4afa-a8ea-3be91bf434e3","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"bst","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":37333,"visible":true,"origin":"","legend":"","description":"","filename":"snnature.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/acf6f78d42e08cdceba8ae45.bst"},{"id":100896693,"identity":"c83be282-2218-4434-b947-3c7ed2070009","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"bst","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":39951,"visible":true,"origin":"","legend":"","description":"","filename":"snvancouveray.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/5c20982734bca11b0debc568.bst"},{"id":100950088,"identity":"576fb3e4-4953-4532-95cd-d62c32138e31","added_by":"auto","created_at":"2026-01-23 07:06:51","extension":"bst","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":40758,"visible":true,"origin":"","legend":"","description":"","filename":"snvancouvernum.bst","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/cffa6e76b988cc6935c5c356.bst"},{"id":100950790,"identity":"52c85fa5-9c6a-45eb-b341-0a3a00d00d89","added_by":"auto","created_at":"2026-01-23 07:09:13","extension":"pdf","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":418495,"visible":true,"origin":"","legend":"","description":"","filename":"usermanual.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/27d15c101aa4676ce519a75d.pdf"},{"id":100896685,"identity":"9eece5a7-a891-49e3-8c34-aeb5ed2b5fa7","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"xml","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":132034,"visible":true,"origin":"","legend":"","description":"","filename":"1adba1b4d7954121a5ccc951d4626a5a1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/576d63c867bbac3d93d0aac0.xml"},{"id":100896688,"identity":"c135028f-51ff-4b93-9c34-419b1820fba4","added_by":"auto","created_at":"2026-01-22 14:17:06","extension":"html","order_by":21,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":153550,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/15c0d0142f6a6a16f0b4f487.html"},{"id":101297090,"identity":"0980597f-9ec7-47b1-b4a0-0ffa2a81d5a3","added_by":"auto","created_at":"2026-01-28 09:25:06","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":379003,"visible":true,"origin":"","legend":"","description":"","filename":"EstimationofTPMStatisticsandComputing.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1_covered_4484e9af-9d89-49a5-baed-36e6527bcaef.pdf"},{"id":100896695,"identity":"0775448f-6792-4f7b-97bd-30621a4a5643","added_by":"auto","created_at":"2026-01-22 14:17:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":1374851,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryStatisticsandComputing.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603162/v1/c378b3ef0fff00794c0c6b6e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Estimation of a Countably Infinite-Dimensional Transition Probability Matrix Using a Stick-Breaking Prior","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Bayesian Nonparametric, Transition Probability Matrix, Dirichlet Process, Generalized Hierarchical Stick-Breaking Process","lastPublishedDoi":"10.21203/rs.3.rs-8603162/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8603162/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWe consider the problem of estimation of the transition probability matrix (TPM) of a Markov chain where the state space is countably infinite. At present, there is no methodology in the literature for direct estimation of infinite-dimensional TPMs. Standard approaches, including maximum likelihood estimation and Bayesian methods based on Dirichlet priors, are inherently finite-state and, when applied in this setting, assign zero probability to all unobserved transitions. To derive a method for estimation of infinite-dimensional TPMs, we propose a Bayesian nonparametric approach that employs a random measure prior. Specifically, we utilize hierarchical and generalized hierarchical stick-breaking processes as priors on the rows of the TPM. The generalized hierarchical stick-breaking prior, in particular, allow for both positive and negative correlations among any pair of transition probabilities, thereby enabling more realistic modeling of the dependence across the state transitions. We develop a blocked Gibbs sampling algorithm for posterior computation under a generalized hierarchical stick-breaking prior that is fast, highly efficient, and well suited to large-scale problems. The proposed method is evaluated through extensive simulations as well as applications to real-world datasets on historical trading volume for the United States Oil Fund (USO), and daily precipitation records from Heathrow, London. The empirical results demonstrate strong and stable predictive performance in all scenarios considered.\u003c/p\u003e","manuscriptTitle":"Estimation of a Countably Infinite-Dimensional Transition Probability Matrix Using a Stick-Breaking Prior","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-22 14:16:54","doi":"10.21203/rs.3.rs-8603162/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-31T10:19:10+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-13T12:14:16+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-05T19:37:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"60917506946935292424838170975006929306","date":"2026-02-03T02:23:49+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"252315023892380277415015826301455456662","date":"2026-01-31T02:39:08+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"3100774508731468173450650711522844330","date":"2026-01-23T03:43:58+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-01-22T21:58:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"32195341144433990788568495653997508503","date":"2026-01-21T20:35:50+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-01-21T01:17:20+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-17T10:33:04+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-01-17T01:30:08+00:00","index":"","fulltext":""},{"type":"submitted","content":"Statistics and Computing","date":"2026-01-14T14:59:42+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"c07097c6-c28b-4b44-9f56-9f12f411b270","owner":[],"postedDate":"January 22nd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-03-31T10:26:27+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-22 14:16:54","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8603162","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8603162","identity":"rs-8603162","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}