Loss Function Matters More Than Framework: A Comparative Study of Gradient Boosting Robustness to Outliers

Loss Function Matters More Than Framework: A Comparative Study of Gradient Boosting Robustness to Outliers | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Loss Function Matters More Than Framework: A Comparative Study of Gradient Boosting Robustness to Outliers Mikhail Ulyanin This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9158378/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract We present a systematic empirical study comparing the robustness of four major tree-based ensemble algorithms — XGBoost, LightGBM, CatBoost, and Random Forest — to controlled training data contamination. Unlike prior work that compares frameworks as monolithic units, we test multiple loss functions (MSE, Huber, MAE) within each boosting framework, yielding 13 regression and 5 classification configurations. Experiments on California Housing, Kaggle House Prices, and Adult Census Income datasets at contamination levels 0-40% reveal that loss function choice affects robustness radically more than framework choice. Within-framework retention index spread averages 0.63, roughly three times the between-framework spread of 0.51. LightGBM with MAE loss retains 96.6% of R2 at 40% label noise, while the same framework with MSE loss retains only 26.6%. Random Forest ranks only 8th out of 12 configurations. We provide theoretical justification through influence function analysis, report anomalous collapse of Huber loss when miscalibrated, and propose a retention index for standardized robustness comparison. For classification under symmetric label noise, CatBoost achieves the highest MCC retention (71.1%), significantly outperforming Random Forest (60.7%). Gradient boosting XGBoost LightGBM CatBoost Random forest Robustness Loss function Label noise Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9158378","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":610508348,"identity":"d2c275e7-a0ef-4e85-9357-90d1154a2358","order_by":0,"name":"Mikhail Ulyanin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA00lEQVRIiWNgGAWjYDACZh4QaQHEjA0MHxgYePiI1CIB1sI4A6iFjbA1cC1Q7QS16LbzHvzA2CaR2D+7ue2xbY6dDBsD7zMJfFrMDvMlS4C0zLhzsN04d1sy0GHsZgS08BiAtTTcSGyTzt3GDNTCxkZIi/EPkJb5IC2W2+qJ0mIGtmUDSAvjtsPEabFgOCdhvBGoRbJ323EeNmY2Zgu8Ws6fMb7BUGYjO+9G+jOJn9uq7fnZ2xhv4NMCAsx/GBgcGxBcQuqhwJ5IdaNgFIyCUTASAQD8djsRUrqWeQAAAABJRU5ErkJggg==","orcid":"","institution":"","correspondingAuthor":true,"prefix":"","firstName":"Mikhail","middleName":"","lastName":"Ulyanin","suffix":""}],"badges":[],"createdAt":"2026-03-18 10:53:57","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9158378/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9158378/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105564707,"identity":"b47225d6-708a-4e13-88f6-8558ca13f7ec","added_by":"auto","created_at":"2026-03-27 12:50:34","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4231830,"visible":true,"origin":"","legend":"","description":"","filename":"submission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9158378/v1_covered_66cb6969-ed43-410c-8f43-1ddce648db18.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Loss Function Matters More Than Framework: A Comparative Study of Gradient Boosting Robustness to Outliers","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Gradient boosting, XGBoost, LightGBM, CatBoost, Random forest, Robustness, Loss function, Label noise","lastPublishedDoi":"10.21203/rs.3.rs-9158378/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9158378/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"We present a systematic empirical study comparing the robustness of four major tree-based ensemble algorithms — XGBoost, LightGBM, CatBoost, and Random Forest — to controlled training data contamination. Unlike prior work that compares frameworks as monolithic units, we test multiple loss functions (MSE, Huber, MAE) within each boosting framework, yielding 13 regression and 5 classification configurations. Experiments on California Housing, Kaggle House Prices, and Adult Census Income datasets at contamination levels 0-40% reveal that loss function choice affects robustness radically more than framework choice. Within-framework retention index spread averages 0.63, roughly three times the between-framework spread of 0.51. LightGBM with MAE loss retains 96.6% of R2 at 40% label noise, while the same framework with MSE loss retains only 26.6%. Random Forest ranks only 8th out of 12 configurations. We provide theoretical justification through influence function analysis, report anomalous collapse of Huber loss when miscalibrated, and propose a retention index for standardized robustness comparison. For classification under symmetric label noise, CatBoost achieves the highest MCC retention (71.1%), significantly outperforming Random Forest (60.7%).","manuscriptTitle":"Loss Function Matters More Than Framework: A Comparative Study of Gradient Boosting Robustness to Outliers","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-25 06:19:49","doi":"10.21203/rs.3.rs-9158378/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"fa4d3378-482a-4e45-a4bc-c52e558893f6","owner":[],"postedDate":"March 25th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-03-25T06:19:49+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-25 06:19:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9158378","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9158378","identity":"rs-9158378","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}