MEGB: An R package for Mixed Effect GradientBoosting for High-dimensional Longitudinal Data

MEGB: An R package for Mixed Effect GradientBoosting for High-dimensional Longitudinal Data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article MEGB: An R package for Mixed Effect GradientBoosting for High-dimensional Longitudinal Data Oyebayo Ridwan Olaniran, Saidat Fehintola Olaniran, Jeza Allohibi, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5947814/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 22 Aug, 2025 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract High-dimensional longitudinal data present significant analytical challenges due to intricate within-subject correlations and an overwhelming ratio of predictors to observations. To address these challenges, we introduce Mixed-Effect Gradient Boosting (MEGB), a novel R package that synergizes gradient boosting with mixed-effects modeling to simultaneously account for population-level fixed effects and subject-specific random variability. MEGB provides a unified framework for analyzing repeated measures data that accommodate complex covariance structures while harnessing gradient boosting’s inherent regularization for robust feature selection and prediction. In comprehensive simulations spanning linear and nonlinear data-generating processes, MEGB achieved 35–76% lower mean squared error (MSE) compared to state-of-the-art alternatives like Mixed- Effect Random Forests (MERF) and REEMForest, while maintaining 55–70% true positive rates for variable selection in ultra-high-dimensional regimes (p = 2000). Demonstrating practical utility, we applied MEGB to maternal cell-free plasma RNA data (n = 12 subjects, p = 33,297 transcripts), where it identified 9 key placental transcripts driving fetal RNA dynamics across pregnancy trimesters. Physical sciences/Mathematics and computing/Software Physical sciences/Mathematics and computing/Statistics Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 22 Aug, 2025 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 23 Apr, 2025 Reviews received at journal 22 Apr, 2025 Reviews received at journal 30 Mar, 2025 Reviewers agreed at journal 20 Mar, 2025 Reviewers agreed at journal 20 Mar, 2025 Reviewers invited by journal 20 Mar, 2025 Editor assigned by journal 20 Mar, 2025 Editor invited by journal 05 Feb, 2025 Submission checks completed at journal 04 Feb, 2025 First submitted to journal 02 Feb, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5947814","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":411367825,"identity":"7048fbf2-fede-4ac5-9dae-c4a58a4211a9","order_by":0,"name":"Oyebayo Ridwan Olaniran","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEElEQVRIiWNgGAWjYDACdiBOgDAZHzAwSMgA6QYghxm3FmaEFmYDoBYe4rRAAZsEkODBFEcD/MzMzx483GOXx8De+6yad4cFD7/04bYHDBXWiQ3S7RewaZFsZjM3SHiWXMzAc9zsNu8ZCR7JvsR2A4Yz6YkNMmcKsGkxOMxgJpFwgDmxQSKN7TZvmwSPwRnGNgnGtsNAkZwE7FrYvwG11IO1FCO0/MOnhQdky2GwFmaElgaQSPoB7H7hKQNqOZ7YxnOMWXIuUItkD1BLwrF04zaJHOwhxt6+TfLHgerEfvY2xg9v2+rk+HnYn0l8qLGW7ZdIf4BVDwywofASwCI8Bni1YAPs+G0ZBaNgFIyCkQIAwjNTirFLMi4AAAAASUVORK5CYII=","orcid":"","institution":"University of Ilorin","correspondingAuthor":true,"prefix":"","firstName":"Oyebayo","middleName":"Ridwan","lastName":"Olaniran","suffix":""},{"id":411367826,"identity":"3016f1a4-f2bb-4bf9-adee-22805f7d517c","order_by":1,"name":"Saidat Fehintola Olaniran","email":"","orcid":"","institution":"Kwara State University","correspondingAuthor":false,"prefix":"","firstName":"Saidat","middleName":"Fehintola","lastName":"Olaniran","suffix":""},{"id":411367827,"identity":"dd51a6f4-8151-49a7-a741-9060ba903725","order_by":2,"name":"Jeza Allohibi","email":"","orcid":"","institution":"Taibah University","correspondingAuthor":false,"prefix":"","firstName":"Jeza","middleName":"","lastName":"Allohibi","suffix":""},{"id":411367829,"identity":"70585fd5-02c7-49d3-a7b1-7fadd749598c","order_by":3,"name":"Abdulmajeed Alharbi","email":"","orcid":"","institution":"Taibah University","correspondingAuthor":false,"prefix":"","firstName":"Abdulmajeed","middleName":"","lastName":"Alharbi","suffix":""},{"id":411367830,"identity":"cbde7b7a-6804-48d3-b747-15d56b62d5df","order_by":4,"name":"Nada MohammedSaeed Alharbi","email":"","orcid":"","institution":"Taibah University","correspondingAuthor":false,"prefix":"","firstName":"Nada","middleName":"MohammedSaeed","lastName":"Alharbi","suffix":""}],"badges":[],"createdAt":"2025-02-03 01:53:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5947814/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5947814/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-16526-z","type":"published","date":"2025-08-22T16:28:56+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":89847151,"identity":"01b8bddc-4f7d-4ffa-b8ff-84a08b8afe95","added_by":"auto","created_at":"2025-08-25 16:41:21","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":389781,"visible":true,"origin":"","legend":"","description":"","filename":"TemplateforsubmissionstoScientificReportsMEEM.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5947814/v1_covered_0a2dc4f1-cb55-4b73-90d4-99e6484955ec.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"MEGB: An R package for Mixed Effect GradientBoosting for High-dimensional Longitudinal Data","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5947814/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5947814/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eHigh-dimensional longitudinal data present significant analytical challenges due to intricate within-subject correlations and an overwhelming ratio of predictors to observations. To address these challenges, we introduce Mixed-Effect Gradient Boosting (MEGB), a novel R package that synergizes gradient boosting with mixed-effects modeling to simultaneously account for population-level fixed effects and subject-specific random variability. MEGB provides a unified framework for analyzing repeated measures data that accommodate complex covariance structures while harnessing gradient boosting’s inherent regularization for robust feature selection and prediction. In comprehensive simulations spanning linear and nonlinear data-generating processes, MEGB achieved 35–76% lower mean squared error (MSE) compared to state-of-the-art alternatives like Mixed- Effect Random Forests (MERF) and REEMForest, while maintaining 55–70% true positive rates for variable selection in ultra-high-dimensional regimes (p = 2000). Demonstrating practical utility, we applied MEGB to maternal cell-free plasma RNA data (n = 12 subjects, p = 33,297 transcripts), where it identified 9 key placental transcripts driving fetal RNA dynamics across pregnancy trimesters.\u003c/p\u003e","manuscriptTitle":"MEGB: An R package for Mixed Effect GradientBoosting for High-dimensional Longitudinal Data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-02-06 04:55:16","doi":"10.21203/rs.3.rs-5947814/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-04-23T09:19:08+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-22T18:32:30+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-03-30T05:58:49+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"68963565514948436253229201069042364271","date":"2025-03-20T18:40:04+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"238925419294146998273673387228934601362","date":"2025-03-20T13:45:26+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-20T12:49:16+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-03-20T12:47:44+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-02-05T06:07:21+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-02-04T08:48:50+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-02-03T01:51:11+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2f7dd794-7d90-4706-80ff-82635f38aa8e","owner":[],"postedDate":"February 6th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":43861449,"name":"Physical sciences/Mathematics and computing/Software"},{"id":43861450,"name":"Physical sciences/Mathematics and computing/Statistics"}],"tags":[],"updatedAt":"2025-08-25T16:31:36+00:00","versionOfRecord":{"articleIdentity":"rs-5947814","link":"https://doi.org/10.1038/s41598-025-16526-z","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2025-08-22 16:28:56","publishedOnDateReadable":"August 22nd, 2025"},"versionCreatedAt":"2025-02-06 04:55:16","video":"","vorDoi":"10.1038/s41598-025-16526-z","vorDoiUrl":"https://doi.org/10.1038/s41598-025-16526-z","workflowStages":[]},"version":"v1","identity":"rs-5947814","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5947814","identity":"rs-5947814","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}