Maximum Penalized Likelihood Estimation of Shannon and Past Entropy under Diverse Sampling Schemes for the Geometric Distribution

Maximum Penalized Likelihood Estimation of Shannon and Past Entropy under Diverse Sampling Schemes for the Geometric Distribution | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Maximum Penalized Likelihood Estimation of Shannon and Past Entropy under Diverse Sampling Schemes for the Geometric Distribution V. Sudha, Jeevanand E. S. This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8757406/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Entropy measures are crucial in measuring uncertainty and unpredictability in complex systems and have found applications in various fields, including reliability theory, economic statistics, cybersecurity, pattern recognition, and artificial intelligence. This paper explores the estimation of Shannon and Past entropy of the Geometric distribution with Maximum Penalised Likelihood Estimation (MPLE), a method that aims to reduce overfitting and increase robustness in small or sparse samples. In contrast to traditional Maximum Likelihood Estimation (MLE), MPLE employs penalty functions to balance the accuracy and complexity of the model. There are two penalty structures considered: a Quadratic (ridge-type) penalty and a Lasso-type penalty. The results of the proposed estimators are compared under a complete sampling scheme, a right-censored sampling scheme, and a Type I censored sampling scheme, using extensive Monte Carlo simulations of 10,000 iterations with different sample sizes (n = 25, 50, 75) and parameter values (0.15, 0.5, 0.9). The criteria applied in evaluation are bias, mean square error (MSE) and relative efficiency (RE). The findings repeatedly indicate that the Quadratic Penalty 1 (QP1) estimator is the best with the lowest bias, least MSE and maximum RE in all cases. It is essential to note that QP1 can sustain a RE of above 80% with moderate sample sizes (n = 50) and mid-range parameter values. The results are helpful in entropy applications, especially in the design of cryptographic systems, the analysis of random number generators, and reliability analysis. The suggested MPLE framework enhances the accuracy of estimates and is robust, thereby contributing to improved predictive performance and the development of safe systems across a wide range of disciplines. Applied Statistics Geometric distribution Shannon entropy Past entropy Maximum penalized likelihood. Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8757406","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":583822186,"identity":"5326e736-364f-4f9b-9cfe-fff5e29b9220","order_by":0,"name":"V. Sudha","email":"","orcid":"https://orcid.org/0009-0007-4640-2215","institution":"Department of Statistics, Mar Athanasius College (Autonomous), Kothamangalam, India","correspondingAuthor":false,"prefix":"","firstName":"V.","middleName":"","lastName":"Sudha","suffix":""},{"id":583822187,"identity":"8a6825c6-7e26-4b2a-afa0-82fa624c0f9d","order_by":1,"name":"Jeevanand E. S.","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAElEQVRIiWNgGAWjYHACMwYGHhibzQZIMDYeIEVLGkhLAxFa4IDtMJjCq8W8/fC2Bz9kbOQMjp89+Lmg7Lzd2vbDQFtqbKJxaZE5k1Zu2MOTZmxwJi9Zesa528nbziQCtRxLy23AoUWCIcdMgofncOLMhhwDad6228lmB4BaGBsO49bC/8ZM8g/P//qZ/W+Mf/O2nUs2O/+QgBaJHDNpHp4DCfwgBm/bATuzG4RskXhWJi3Dk2zYL/HGzJrnXHKC2Q2gLQn4/MKfvE3ybY+dPBt/jvFtnjI7e7Pz6Q8ffKixwakFDBh7EOxEsMoEfMrB4AeCaU9Q8SgYBaNgFIw4AACV0l0ESxoQAAAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-7102-4179","institution":"Department of Statistics and Data Science, Christ University, Karnataka, India","correspondingAuthor":true,"prefix":"","firstName":"Jeevanand","middleName":"E.","lastName":"S.","suffix":""}],"badges":[],"createdAt":"2026-02-01 15:41:51","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8757406/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8757406/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101731980,"identity":"2324691d-6d27-4b65-b847-f00c2ffaafa2","added_by":"auto","created_at":"2026-02-03 06:19:44","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":246527,"visible":true,"origin":"","legend":"","description":"","filename":"sudhamacfinal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8757406/v1_covered_85f53879-ee35-465a-af4a-1c85ab56f83b.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eMaximum Penalized Likelihood Estimation of Shannon and Past Entropy under Diverse Sampling Schemes for the Geometric Distribution\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Geometric distribution, Shannon entropy, Past entropy, Maximum penalized likelihood.","lastPublishedDoi":"10.21203/rs.3.rs-8757406/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8757406/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEntropy measures are crucial in measuring uncertainty and unpredictability in complex systems and have found applications in various fields, including reliability theory, economic statistics, cybersecurity, pattern recognition, and artificial intelligence. This paper explores the estimation of Shannon and Past entropy of the Geometric distribution with Maximum Penalised Likelihood Estimation (MPLE), a method that aims to reduce overfitting and increase robustness in small or sparse samples. In contrast to traditional Maximum Likelihood Estimation (MLE), MPLE employs penalty functions to balance the accuracy and complexity of the model.\u003c/p\u003e\n\u003cp\u003eThere are two penalty structures considered: a Quadratic (ridge-type) penalty and a Lasso-type penalty. The results of the proposed estimators are compared under a complete sampling scheme, a right-censored sampling scheme, and a Type I censored sampling scheme, using extensive Monte Carlo simulations of 10,000 iterations with different sample sizes (n = 25, 50, 75) and parameter values (0.15, 0.5, 0.9). The criteria applied in evaluation are bias, mean square error (MSE) and relative efficiency (RE).\u003c/p\u003e\n\u003cp\u003eThe findings repeatedly indicate that the Quadratic Penalty 1 (QP1) estimator is the best with the lowest bias, least MSE and maximum RE in all cases. It is essential to note that QP1 can sustain a RE of above 80% with moderate sample sizes (n = 50) and mid-range parameter values.\u003c/p\u003e\n\u003cp\u003eThe results are helpful in entropy applications, especially in the design of cryptographic systems, the analysis of random number generators, and reliability analysis. The suggested MPLE framework enhances the accuracy of estimates and is robust, thereby contributing to improved predictive performance and the development of safe systems across a wide range of disciplines.\u003c/p\u003e","manuscriptTitle":"Maximum Penalized Likelihood Estimation of Shannon and Past Entropy under Diverse Sampling Schemes for the Geometric Distribution","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-03 06:19:32","doi":"10.21203/rs.3.rs-8757406/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"b3724ffd-d4fb-45d7-826d-61f786c6c2c0","owner":[],"postedDate":"February 3rd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":62110534,"name":"Applied Statistics"}],"tags":[],"updatedAt":"2026-02-03T06:19:32+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-03 06:19:32","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8757406","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8757406","identity":"rs-8757406","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}