Understanding Merge Request Deviations Across MR Types and Their Impact on Code Review Effort Analysis

Understanding Merge Request Deviations Across MR Types and Their Impact on Code Review Effort Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Understanding Merge Request Deviations Across MR Types and Their Impact on Code Review Effort Analysis Samah Kansab, Francis Bordeleau, Ali Tizghadam This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9593471/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In DevOps environments, code review plays a central role in maintaining code quality, supporting collaboration, and controlling the integration of software changes. While prior research has examined many aspects of this process, most studies implicitly assume that all code reviews follow a standard evaluation workflow. However, our industrial partner, which relies on the Merge Request (MR) mechanism for code review, reports that this assumption does not always hold in practice. A substantial portion of MRs are opened for purposes other than rigorous code evaluation. These MRs often bypass the intended review process and require limited or no review oversight. We refer to such cases as deviations, as they depart from the expected review workflow. Examples include work-in-progress MRs used as draft implementations, largechange MRs created for rebasing or restructuring, and library updates that involve dependency version changes with little review effort. Ignoring such cases can bias MR analytics and reduce the reliability of machine learning (ML) models used to explain code review effort. This study investigates MR deviations in a large industrial setting. First, we identify and characterize MR deviations, showing that they account for up to 37.02% of MRs across seven categories. Second, we develop an automated detection approach based on few-shot learning, which achieves up to 91% accuracy in identifying deviations. Third, we examine the impact of excluding deviations on ML models used to analyze code review effort. For MR completion time, removing deviations significantly improves model performance in 53.33% of cases, with gains of up to 2.25×, while substantially altering feature importance rankings. We then extend this analysis to a second effort dimension, namely the amount of changes required to complete an MR, and show that excluding deviations significantly improves performance in 46.67% of cases, with improvements of up to 3.15× in MSE and 1.91× in MAE. In addition, we show that deviations are not uniformly distributed across MR types: configuration MRs are mainly associated with library updates and build or configuration adjustments, whereas development MRs exhibit a more heterogeneous profile including code cleaning, experimental work, and revert-related cases. Finally, we show that the analytical impact of deviations differs across configuration and development MRs, and depends on whether effort is measured through completion time or through the amount of changes required to complete an MR. Our contributions are fourfold: (1) a clear definition and categorization of MR deviations, (2) an AI-based detection approach leveraging few-shot learning, (3) a multi-dimensional empirical analysis of how deviations impact code review effort modeling, and (4) evidence that deviations vary across MR types and should be analyzed in a type-aware manner. These findings help practitioners better prioritize review effort and support researchers in building more reliable MR analytics. Code Review Merge Requests Deviations Few-shot Learning Machine Learning MR types Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9593471","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":633324860,"identity":"706c82b3-e229-4a16-a1a1-4027a5cdaa46","order_by":0,"name":"Samah Kansab","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABB0lEQVRIie2Qu2rDMABFbyg0i4pXQyD+BZlAMvThX5ExuIsF2eot2boIsvozFAyGbgYNHdI0qyAdPGULGApd2qFyPBbR0KmDziIkONyDAIfjn3IBsO68BOhN/9Scr8xTEGCw7O7kPKVVvyvB4yZ+nyMNZp46NKC7ceRt7huGt3FkUegLL0cFsvCpSGcUdD8hPl+blcPEtkLB5YggH0iNqW+UWPhXnaJiYQtbHctPo0Ry9/xhlNeF8LYnZWFToHllVrJY1lm3UjOCU5hi1jB9rK4JTROpswfKaBIKzdcFoyoUtS2Ml3uSJ7cmrGrar7tguNrKts1VMFzayvpP6GE/XhwOh8PxJ74ByTxbMqteknoAAAAASUVORK5CYII=","orcid":"","institution":"École de technologie supérieure (ÉTS)","correspondingAuthor":true,"prefix":"","firstName":"Samah","middleName":"","lastName":"Kansab","suffix":""},{"id":633324861,"identity":"2f86ef4f-8469-45fd-89c4-de9ae9360db3","order_by":1,"name":"Francis Bordeleau","email":"","orcid":"","institution":"École de technologie supérieure (ÉTS)","correspondingAuthor":false,"prefix":"","firstName":"Francis","middleName":"","lastName":"Bordeleau","suffix":""},{"id":633324862,"identity":"dafbbd63-9348-4b67-b370-1814a51c1e77","order_by":2,"name":"Ali Tizghadam","email":"","orcid":"","institution":"TELUS","correspondingAuthor":false,"prefix":"","firstName":"Ali","middleName":"","lastName":"Tizghadam","suffix":""}],"badges":[],"createdAt":"2026-05-02 11:42:14","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-9593471/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9593471/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108493821,"identity":"6ebc8a61-f370-4588-a22e-737c55c5eba4","added_by":"auto","created_at":"2026-05-05 10:01:52","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":552669,"visible":true,"origin":"","legend":"","description":"","filename":"preprintrevisedEMSEdeviationsextension.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9593471/v1_covered_22f8e690-3ffe-41e9-b130-d19ec0db95a9.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eUnderstanding Merge Request Deviations Across MR Types and Their Impact on Code Review Effort Analysis\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"École de Technologie Supérieure","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Code Review, Merge Requests, Deviations, Few-shot Learning, Machine Learning, MR types","lastPublishedDoi":"10.21203/rs.3.rs-9593471/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9593471/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn DevOps environments, code review plays a central role in maintaining code quality, supporting collaboration, and controlling the integration of software changes. While prior research has examined many aspects of this process, most studies implicitly assume that all code reviews follow a standard evaluation workflow. However, our industrial partner, which relies on the Merge Request (MR) mechanism for code review, reports that this assumption does not always hold in practice. A substantial portion of MRs are opened for purposes other than rigorous code evaluation. These MRs often bypass the intended review process and require limited or no review oversight. We refer to such cases as deviations, as they depart from the expected review workflow. Examples include work-in-progress MRs used as draft implementations, largechange MRs created for rebasing or restructuring, and library updates that involve dependency version changes with little review effort. Ignoring such cases can bias MR analytics and reduce the reliability of machine learning (ML) models used to explain code review effort. This study investigates MR deviations in a large industrial setting. First, we identify and characterize MR deviations, showing that they account for up to 37.02% of MRs across seven categories. Second, we develop an automated detection approach based on few-shot learning, which achieves up to 91% accuracy in identifying deviations. Third, we examine the impact of excluding deviations on ML models used to analyze code review effort. For MR completion time, removing deviations significantly improves model performance in 53.33% of cases, with gains of up to 2.25×, while substantially altering feature importance rankings. We then extend this analysis to a second effort dimension, namely the amount of changes required to complete an MR, and show that excluding deviations significantly improves performance in 46.67% of cases, with improvements of up to 3.15× in MSE and 1.91× in MAE. In addition, we show that deviations are not uniformly distributed across MR types: configuration MRs are mainly associated with library updates and build or configuration adjustments, whereas development MRs exhibit a more heterogeneous profile including code cleaning, experimental work, and revert-related cases. Finally, we show that the analytical impact of deviations differs across configuration and development MRs, and depends on whether effort is measured through completion time or through the amount of changes required to complete an MR. Our contributions are fourfold: (1) a clear definition and categorization of MR deviations, (2) an AI-based detection approach leveraging few-shot learning, (3) a multi-dimensional empirical analysis of how deviations impact code review effort modeling, and (4) evidence that deviations vary across MR types and should be analyzed in a type-aware manner. These findings help practitioners better prioritize review effort and support researchers in building more reliable MR analytics.\u003c/p\u003e","manuscriptTitle":"Understanding Merge Request Deviations Across MR Types and Their Impact on Code Review Effort Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-05 03:51:54","doi":"10.21203/rs.3.rs-9593471/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"77d1e436-6a2e-4122-9999-aab2ad4cdb8e","owner":[],"postedDate":"May 5th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-05T03:51:54+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-05 03:51:54","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9593471","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9593471","identity":"rs-9593471","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}