Enhancing Logistic Regression Performance Through Hyperparameter Tuning: A Comparative Evaluation Across Datasets

Enhancing Logistic Regression Performance Through Hyperparameter Tuning: A Comparative Evaluation Across Datasets | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Enhancing Logistic Regression Performance Through Hyperparameter Tuning: A Comparative Evaluation Across Datasets Mueed Ahmad, Noman Javed, Awais Muzafar, Mateen Muzafar, Hadia Naseer, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8304042/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background: Logistic regression (LR) is widely used in binary and multi-class classification tasks, yet its predictive performance is highly sensitive to hyperparameter configuration. Suboptimal choices can lead to overfitting, underfitting, reduced generalization, and inconsistent model behavior across datasets. This study aims to systematically enhance LR performance by applying a comprehensive hyperparameter optimization framework and evaluating its impact across four diverse datasets: breast cancer, heart disease, liver disorders, and handwritten digits. Methods: A Python-based experimental framework was developed using Scikit-learn, NumPy, and Pandas to examine how hyperparameters influence LR performance. A combinatorial optimization strategy was applied to tune regularization strength (C), penalty type (L1), solver choice ( liblinear , saga ), class-weight settings, and maximum iterations. Model evaluation was conducted using both train--test splits (20%, 30%, 40%) and k-fold cross-validation ( \((k = 3, 5, 10)\) ). Performance was assessed using accuracy, F1-score, AUC, and cross-validation accuracy. Tableau-based visual analytics were used to compare model behaviors under different configurations. Results: Optimized hyperparameters consistently improved model performance across all datasets. The breast cancer and digits datasets achieved the most substantial gains, with maximum test accuracies of 97% and 98%, respectively, and AUC values up to 0.99. Cross-validation scores indicated strong generalization, with the best-performing models showing CV accuracies above 0.90. In contrast, performance improvements on heart disease and liver disorder datasets were present but more modest due to noisier features and class imbalance. Hyperparameter combinations involving L1 penalty, balanced class weights, and the liblinear solver produced the highest accuracy and F1-scores across several datasets. Conclusions: Systematic hyperparameter tuning significantly enhances logistic regression performance, generalization, and discrimination ability. The results demonstrate that even simple models can achieve high accuracy when appropriately optimized. This framework provides practical guidance for improving LR across heterogeneous datasets and highlights the importance of penalty choice, regularization strength, and solver selection. Future work should explore advanced optimization techniques such as Bayesian optimization and evolutionary algorithms to further improve efficiency and performance. Machine Learning Logistic Regression Hyperparameter Tuning Generalization Model Performance Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 07 Jan, 2026 Reviewers agreed at journal 07 Jan, 2026 Reviewers invited by journal 07 Jan, 2026 Editor assigned by journal 05 Jan, 2026 Editor invited by journal 12 Dec, 2025 Submission checks completed at journal 12 Dec, 2025 First submitted to journal 12 Dec, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8304042","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":571162118,"identity":"26ce5e8a-b503-4cc8-bdfd-45b075987f22","order_by":0,"name":"Mueed Ahmad","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"prefix":"","firstName":"Mueed","middleName":"","lastName":"Ahmad","suffix":""},{"id":571162119,"identity":"5746d396-d9fc-455a-a6d8-5df2b0bf139b","order_by":1,"name":"Noman Javed","email":"","orcid":"","institution":"University of the West of Scotland","correspondingAuthor":false,"prefix":"","firstName":"Noman","middleName":"","lastName":"Javed","suffix":""},{"id":571162122,"identity":"e07f4a84-ff55-49e3-a876-ec243ca63669","order_by":2,"name":"Awais Muzafar","email":"","orcid":"","institution":"Beijing Institute of Graphic Communication","correspondingAuthor":false,"prefix":"","firstName":"Awais","middleName":"","lastName":"Muzafar","suffix":""},{"id":571162123,"identity":"0f671639-d95b-4f37-8f93-dad7edac5e72","order_by":3,"name":"Mateen Muzafar","email":"","orcid":"","institution":"University of Évry Val d'Essonne","correspondingAuthor":false,"prefix":"","firstName":"Mateen","middleName":"","lastName":"Muzafar","suffix":""},{"id":571162125,"identity":"84efb3ca-f5f2-4714-86ae-9b2d36ff5eb1","order_by":4,"name":"Hadia Naseer","email":"","orcid":"","institution":"University of Agriculture Faisalabad","correspondingAuthor":false,"prefix":"","firstName":"Hadia","middleName":"","lastName":"Naseer","suffix":""},{"id":571162128,"identity":"3326f790-087e-4b56-b36d-65913ba1d1c4","order_by":5,"name":"Guantian Huang","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"prefix":"","firstName":"Guantian","middleName":"","lastName":"Huang","suffix":""},{"id":571162130,"identity":"7a559eb9-b478-4422-81fd-cf64732774fb","order_by":6,"name":"Dianning He","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAApElEQVRIiWNgGAWjYBACAxD6AGcTq4VxBslamHlI0mLO3rztsU3NtsQGIEOCoeYOYS2WPcfKjXOO3U5s4DlWJsFw7BkRDruRYyad2wDUIpFjJsHYcJgILfffmElbgrTIvyFWyw0eM2lGsC08xGo5k1Ym2XPstnEbT1qxRcIxYrQcP7xN4kfNbdl+9sMbb3yoIUILHLCBiAQSNIyCUTAKRsEowAMAo444U5XATlsAAAAASUVORK5CYII=","orcid":"","institution":"China Medical University","correspondingAuthor":true,"prefix":"","firstName":"Dianning","middleName":"","lastName":"He","suffix":""}],"badges":[],"createdAt":"2025-12-08 06:08:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8304042/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8304042/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99911560,"identity":"923300bf-9ebc-4aa5-a5eb-1c0d1ff40c73","added_by":"auto","created_at":"2026-01-09 18:19:40","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10080,"visible":true,"origin":"","legend":"","description":"","filename":"a66829aedc3c4819bb045f1c8736dea2.json","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/c8d43a40cf1db0b265e1e9cb.json"},{"id":99911561,"identity":"bb33f2c8-340b-410a-93eb-f1dacd44510b","added_by":"auto","created_at":"2026-01-09 18:19:40","extension":"xml","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":115736,"visible":true,"origin":"","legend":"","description":"","filename":"a66829aedc3c4819bb045f1c8736dea21enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/daa806cb35ae4b363bf14ac1.xml"},{"id":100359513,"identity":"458520fc-88ec-47b8-890a-d337a8080a44","added_by":"auto","created_at":"2026-01-16 07:22:41","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":119530,"visible":true,"origin":"","legend":"","description":"","filename":"Coverletter.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/c73cafd965023e3d05265a18.pdf"},{"id":99911568,"identity":"0c4b0487-0946-4269-9817-19cfc0358624","added_by":"auto","created_at":"2026-01-09 18:19:41","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":3723387,"visible":true,"origin":"","legend":"","description":"","filename":"EnhancingLogisticRegressionPerformanceThroughHyperparameterTuningAComparativeEvaluationAcrossDatasets.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/618ecf75dcb6c2399fb20c1b.pdf"},{"id":100359288,"identity":"d2dff0c3-fd53-4bad-a8cd-f79dadbbd961","added_by":"auto","created_at":"2026-01-16 07:21:57","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":97077,"visible":true,"origin":"","legend":"","description":"","filename":"ResponsetotheEditor.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/905c5f382d5e2097ac9c51a4.pdf"},{"id":100359184,"identity":"bbbbdf14-fa9a-40cf-960e-205eae8a0bec","added_by":"auto","created_at":"2026-01-16 07:21:48","extension":"cls","order_by":22,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":55857,"visible":true,"origin":"","legend":"","description":"","filename":"snjnl.cls","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/1e58f7ff77d4e82e07f40ad2.cls"},{"id":99911564,"identity":"63de6159-cf9b-4872-9d3c-35a8cd99f718","added_by":"auto","created_at":"2026-01-09 18:19:40","extension":"bst","order_by":23,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64166,"visible":true,"origin":"","legend":"","description":"","filename":"snmathphysnum.bst","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/1a6ec81da923ce204e92e0c1.bst"},{"id":100359301,"identity":"085bf9b4-2ec9-462e-b00f-b55e71dc8b90","added_by":"auto","created_at":"2026-01-16 07:21:59","extension":"xml","order_by":24,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":121696,"visible":true,"origin":"","legend":"","description":"","filename":"a66829aedc3c4819bb045f1c8736dea21structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/e41021804096d9edbde738a5.xml"},{"id":99911566,"identity":"3e673323-82ec-427c-8fc7-f61b519a006c","added_by":"auto","created_at":"2026-01-09 18:19:40","extension":"html","order_by":25,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":133182,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1/ebda557249c4cc8cba40d6a4.html"},{"id":100377349,"identity":"2d6465ea-b590-48ed-96c5-85799dc3fcf4","added_by":"auto","created_at":"2026-01-16 08:47:43","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":983777,"visible":true,"origin":"","legend":"","description":"","filename":"EnhancingLogisticRegressionPerformanceThroughHyperparameterTuningAComparativeEvaluationAcrossDatasets.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8304042/v1_covered_2ba6db95-4d3d-4d27-b20c-9958e31df23a.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Enhancing Logistic Regression Performance Through Hyperparameter Tuning: A Comparative Evaluation Across Datasets","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Machine Learning, Logistic Regression, Hyperparameter Tuning, Generalization, Model Performance","lastPublishedDoi":"10.21203/rs.3.rs-8304042/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8304042/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eBackground:\u003c/b\u003e Logistic regression (LR) is widely used in binary and multi-class classification tasks, yet its predictive performance is highly sensitive to hyperparameter configuration. Suboptimal choices can lead to overfitting, underfitting, reduced generalization, and inconsistent model behavior across datasets. This study aims to systematically enhance LR performance by applying a comprehensive hyperparameter optimization framework and evaluating its impact across four diverse datasets: breast cancer, heart disease, liver disorders, and handwritten digits.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods:\u003c/b\u003e A Python-based experimental framework was developed using Scikit-learn, NumPy, and Pandas to examine how hyperparameters influence LR performance. A combinatorial optimization strategy was applied to tune regularization strength (C), penalty type (L1), solver choice (\u003cem\u003eliblinear\u003c/em\u003e, \u003cem\u003esaga\u003c/em\u003e), class-weight settings, and maximum iterations. Model evaluation was conducted using both train--test splits (20%, 30%, 40%) and k-fold cross-validation (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((k = 3, 5, 10)\\)\u003c/span\u003e\u003c/span\u003e). Performance was assessed using accuracy, F1-score, AUC, and cross-validation accuracy. Tableau-based visual analytics were used to compare model behaviors under different configurations.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults:\u003c/b\u003e Optimized hyperparameters consistently improved model performance across all datasets. The breast cancer and digits datasets achieved the most substantial gains, with maximum test accuracies of 97% and 98%, respectively, and AUC values up to 0.99. Cross-validation scores indicated strong generalization, with the best-performing models showing CV accuracies above 0.90. In contrast, performance improvements on heart disease and liver disorder datasets were present but more modest due to noisier features and class imbalance. Hyperparameter combinations involving L1 penalty, balanced class weights, and the \u003cem\u003eliblinear\u003c/em\u003e solver produced the highest accuracy and F1-scores across several datasets.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusions:\u003c/b\u003e Systematic hyperparameter tuning significantly enhances logistic regression performance, generalization, and discrimination ability. The results demonstrate that even simple models can achieve high accuracy when appropriately optimized. This framework provides practical guidance for improving LR across heterogeneous datasets and highlights the importance of penalty choice, regularization strength, and solver selection. Future work should explore advanced optimization techniques such as Bayesian optimization and evolutionary algorithms to further improve efficiency and performance.\u003c/p\u003e","manuscriptTitle":"Enhancing Logistic Regression Performance Through Hyperparameter Tuning: A Comparative Evaluation Across Datasets","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-09 18:19:36","doi":"10.21203/rs.3.rs-8304042/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2026-01-08T02:29:51+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"70652590320409985522323122439856723544","date":"2026-01-08T01:37:53+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-01-07T18:26:25+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-05T09:52:19+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-12-12T12:18:47+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-12-12T11:44:06+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Research Methodology","date":"2025-12-12T11:41:31+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"52034ca5-7132-45bc-a8bb-d5d53cba058e","owner":[],"postedDate":"January 9th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-01-09T18:19:36+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-09 18:19:36","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8304042","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8304042","identity":"rs-8304042","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}