Comparative Analysis of Test Coverage Metrics in Agile vs. Traditional Software Development

Comparative Analysis of Test Coverage Metrics in Agile vs. Traditional Software Development | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Comparative Analysis of Test Coverage Metrics in Agile vs. Traditional Software Development Wumi Ajayi, Akeem Olamide Arikeuyo, Ayodeji Olamilekan Suleiman, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7470780/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Test coverage metrics are broadly used in software engineering to assess the thoroughness of automated testing, up till now little empirical work has directly compared their behavior across Agile and Traditional development methodologies. This research conducts a mixed-method analysis combining industrial case studies, mining of public repositories, controlled experiments, and machine learning modeling to study statement, branch, and path coverage patterns in 30 projects spanning multiple domains and languages. Statistical analysis has x-rayed that Agile projects achieved significantly higher and more stable statement and branch coverage than Traditional projects, with smaller but consistent differences in path coverage. Continuous Integration (CI) occurrence was robustly connected with coverage stability in Agile settings but yields mixed benefits in Traditional contexts. Coverage volatility appeared as a robust predictor of methodology, enabling a Random Forest classifier to differentiate Agile from Traditional projects with 87% accuracy. Contextual features such as project size, domain, and language also moderate coverage results, with embedded systems exhibiting systematically lower path coverage due to hardware and regulatory constraints. These investigation revealed actionable guidance for test managers, including the use of coverage volatility and CI frequency as process health indicators, and highlight the trade-offs between earlier, incremental coverage growth in Agile and concentrated, late-phase coverage in Traditional methods. Artificial Intelligence and Machine Learning Software testing Agile software development Waterfall model Test coverage metrics Continuous integration Software quality assurance Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 I. INTRODUCTION IN contemporary software engineering, testing and quality assurance (QA) are fundamental components that ensure software systems align with user expectations, comply with regulatory standards, and achieve long-term maintainability objectives. As software systems become more intricate and the exigency to convey products increases, organizations are driven to execute development methodologies that can provide both high functionality and superior quality. In this situation, test coverage metrics quantitative indicators that divulged the extent to which a system’s code was evaluated by automated tests have become vital measures of software reliability and development diligence. Coverage metrics, including statement coverage, branch coverage, and path coverage, offer valuable approach into the comprehensiveness of the evaluation procedure. They function not only as technical standards but also as decision-making aids for project managers, test leads, and quality engineers. However, the evolution, interpretation, and impact of coverage metrics can vary significantly between Agile and Traditional (plan-driven) software development methodologies. Agile focuses on iterative development, continuous integration, and regular refactoring, which may lead to more fluid and incremental coverage patterns. Conversely, Traditional models like the Waterfall or V-Model typically emphasize extensive test planning and execution in later phases, which may result in different coverage trends. Despite the acknowledged significance of coverage metrics, there is a notable lack of empirical research that directly compares their performance across Agile and Traditional environments using industry-standard data. The existing literature frequently examines either methodology in isolation or discusses testing practices in general terms without detailed metric-based comparisons. This absence of direct comparative evidence restricts practitioners’ capacity to make well-informed decisions regarding test strategies when shifting between methodologies or adopting hybrid approaches. A. Problem Statement and Research Questions Problem Statement Test coverage metrics such as statement, branch, and path coverage are widely used to assess the thoroughness of software testing efforts. While these metrics are well understood in isolation, their behavior and practical implications can vary significantly depending on the development methodology employed. Agile methods emphasize iterative delivery, continuous integration, and incremental test evolution, whereas Traditional (plan-driven) approaches concentrate testing activities in later phases with extensive upfront planning. Despite the prevalence of both methodologies in industry, there is limited empirical evidence that directly compares coverage metric trends, stability, and influencing factors across these contexts. This lack of comparative, metric-level insights hampers practitioners’ ability to select, interpret, and act on coverage data when transitioning between methodologies or adopting hybrid strategies. This study seeks to address this gap by conducting a structured, longitudinal comparison of coverage metric patterns in Agile and Traditional projects, integrating industrial datasets, public repository mining, and controlled experiments. The goal is to provide actionable, evidence-based guidance for software managers, testers, and engineers. B. Research Questions To guide the investigation, the following research questions (RQs) are posed: · RQ1: How do statement, branch, and path coverage metrics evolve over time in Agile versus Traditional software development projects? · RQ2: How does the adoption and use of Continuous Integration (CI) influence the stability and growth patterns of coverage metrics in each methodology? · RQ3: What contextual factors such as project size, domain, programming language, and toolchain moderate observed differences in coverage metrics between Agile and Traditional projects? II. RELATED WORK 1) Agile vs. Traditional Testing Approaches Agile and Traditional (plan-driven) development differ fundamentally in how testing is organised and executed. Traditional models such as Waterfall and the V-Model treat testing as a distinct, often late, project phase with extensive upfront test planning and large test suites executed towards the end of the lifecycle. By contrast, Agile promotes continuous testing across short iterations: tests are created and run alongside development, integrated with continuous integration (CI) pipelines, and are subject to frequent refactoring and incremental updates. These differences influence not only the timing of test execution but also how teams maintain and prioritize tests, how quickly test suites diverge from code, and how teams interpret test signals for release decisions. Empirical studies of testing in Agile settings highlight testers’ evolving roles and the human factors that shape testing practice in iterative teams. 2) Coverage Metrics: Definitions and Uses Test coverage metrics are structural measures that quantify the extent to which tests exercise program elements. Common metrics include: · Statement (line) coverage: fraction of executable statements executed by the test suite. · Branch (decision) coverage: fraction of control-flow branches (true/false outcomes of decisions) executed. · Path coverage: measures distinct execution paths through a method or routine (often infeasible to exhaustively achieve for nontrivial programs). · Condition/decision coverage: finer-grained criteria that consider individual boolean sub-expressions. These metrics serve multiple practical roles: they guide test development (what to add), provide quality dashboards for managers, and act as acceptance gates in CI pipelines. Tooling such as JaCoCo, Cobertura, and Istanbul implement these counters and remain the de-facto way practitioners collect coverage data in modern builds. 3) Recent Empirical and Methodological Work on Coverage Recent empirical work has explored coverage from different angles: studies have investigated the achievable coverage levels in real-world test suites, the relationship between coverage and fault detection, and coverage as an input to automated test generation and program repair. For example, an ACM empirical analysis of performance test suites examined the magnitude and variability of coverage across 28 open-source systems, illustrating that coverage numbers vary widely and are sensitive to test purpose and tooling. Other studies compare coverage criteria (branch vs. mutation coverage) and show that simple structural metrics may miss test weaknesses revealed by stronger criteria. Advances in coverage-directed test generation and fuzzing further demonstrate that branch/line coverage remain central optimization targets for automated tools 4) Existing Comparisons Between Agile and Traditional Testing Although there is a rich literature on Agile testing practices and separate bodies of work on coverage metrics and coverage-directed testing, relatively few empirical studies directly compare coverage metric patterns between Agile and Traditional projects using industry-grade data. Systematic mappings of Agile testing research show strong interest in regression testing, CI integration, and test automation within Agile contexts, but they also highlight gaps in metric-level comparisons and longitudinal coverage analyses across methodologies. Where comparisons exist, they often focus on process outcomes (e.g., defect rates, delivery speed) rather than on how coverage metrics evolve over time, respond to refactoring, or correlate with fault detection under different development cadences. This study addresses the empirical gap noted above by providing a structured, metric-level comparison of coverage behaviour in Agile versus Traditional projects using industrial and open-source datasets. Concretely, our contribution is threefold: 1. Longitudinal metric comparison: we quantify how statement, branch, and (where available) path/condition coverage evolve through a project’s lifecycle under Agile and Traditional cadences. 2. Contextual analysis: we identify contextual moderators (project size, domain, language/toolchain, CI usage) that explain between-project differences in coverage patterns. 3. Practitioner guidance: we translate empirical observations into actionable recommendations for managers and test leads, what coverage metrics to prioritize and how to interpret them under each methodology. By combining coverage traces from CI systems with qualitative context from project artefacts, this work aims to bridge the methodological divide: delivering rigorously analysed metrics while keeping results immediately useful for software practitioners. Numerous empirical studies have examined the correlation between test coverage metrics and software development approaches. Alégroth et al. [1], in an empirical case study of Agile teams, observed variable coverage trends resulting from regular refactoring, but mentioned that incremental branch coverage remained reliably high. Likewise, Panichella et al. [2] utilized a mixed-method strategy—integrating surveys and repository analysis—to find that Agile testers mainly depend on automated unit tests, continuously tracking coverage, while acceptance tests are given less attention. Ampatzoglou et al. [3] performed a quantitative study on open-source software projects, uncovering a moderate relationship between increased branch and statement coverage and decreased post-release defects. In a similar industrial survey, Rahman and Zulkernine [4] discovered that Agile projects had higher average statement coverage than Traditional methods, although with increased variance. Recent extensive and industrial research has enhanced these insights even more. Herzig and Nagappan [5] found that analyzing CI/CD data shows that running smaller tests more often stabilizes coverage trends better than running larger tests later in the process. Yoo and Harman [6], in their examination of regression testing, emphasized the ongoing importance of coverage metrics for selecting regression tests, regardless of the development method used. Supporting these findings, Zhu et al. [7] showed that continuous integration within Agile settings promotes consistent incremental growth in coverage, while Traditional projects display substantial but sporadic surges in coverage towards the end of the development process. Together, these studies suggest that although Agile practices generally produce greater and more consistent coverage, variations in methodology and testing strategies significantly impact the development of coverage III. METHODOLOGY This study employs a mixed-method empirical method, incorporating commercial case studies, mining of public repositories, controlled setup, and machine learning assisted investigation. The aim is to provide a all-inclusive and reproducible comparison of test coverage metrics between Agile and Traditional software development approaches. A. Data Sources Data was gathered from three complementary sources to ensure coverage of both industrial and open-source contexts: 1. Industrial Case Studies o Selection criteria: Partner companies with well documented methodologies (Agile Scrum, Kanban, or Traditional Waterfall/V-Model) and recognized as automated testing pipelines. o Projects: 6 Agile and 4 Traditional projects across finance, e-commerce, and embedded systems domains. o Data collected: Historical CI build logs, coverage reports, and release notes spanning a minimum of 6 months per project. 2. Public Repositories with CI/CD Reports o Source: GitHub and GitLab projects with publicly available CI pipelines (GitHub Actions, Travis CI, Jenkins). o Projects: 12 Agile-oriented repositories and 8 repositories adhering to staged release cycles. o Data collected: Coverage reports generated at each commit or tagged release, extracted via APIs and repository-mining scripts. 3. Controlled Experiments o Participants: 4 student teams (2 Agile, 2 Traditional) and 2 small industry teams developing the same medium-scale Java web application. o Setup: All teams implemented identical functional requirements under different lifecycle models. Coverage metrics were measured at predefined milestones. B. Sampling Rationale The selected projects and teams were chosen to balance: · Methodological diversity: Ensuring representation from both iterative (Agile) and sequential (Traditional) lifecycles. · Domain variety: Including consumer-facing, enterprise, and embedded software to reflect different testing pressures. · Tooling diversity: Covering Java, JavaScript/TypeScript, Python, and C/C++ to generalize across ecosystems. This combination mirrors common industry practice and helps ensure the findings are transferable to a wide range of software contexts. C. Data Size Across all sources, the dataset comprised: · Commits analyzed: 48,732 total commits · Coverage datapoints: 19,426 unique coverage reports · Test executions: 3,187 CI pipeline runs · Time span: 6–24 months per project, depending on source availability D. Coverage Tools and Measurement To maintain comparability, coverage was measured using industry-standard tools: · Java: JaCoCo (statement, branch coverage) · JavaScript/TypeScript: Istanbul/nyc (statement, branch, function coverage) · Python: Coverage.py (statement, branch coverage) · C/C++: gcov/lcov (line and branch coverage) All raw outputs were normalized to percentages and aligned to the three primary metrics: statement, branch, and path coverage. E. Analytical Methods Two complementary analytical approaches were used: 1. Statistical Analysis o Mann–Whitney U tests to compare median coverage levels between Agile and Traditional groups. o Spearman’s rank correlation to assess the relationship between coverage levels and defect density. o Time-series trend analysis to study metric evolution over project lifecycles. 2. Machine Learning Model o Algorithm: Random Forest Classifier o Purpose: Predict development methodology (Agile or Traditional) from coverage patterns, CI frequency, and project metadata. o Features: Average statement coverage, average branch coverage, coverage volatility, CI build frequency, codebase size, and refactoring frequency. o Evaluation: 10-fold cross-validation, reporting accuracy, precision, recall, and feature importance scores to identify the most predictive factors. F. Threats to Validity · Construct validity: The study assumes that statement, branch, and path coverage are valid proxies for testing thoroughness. Other dimensions (e.g., mutation coverage, fault detection rate) are not directly analyzed. · Internal validity: Industrial case studies may contain undocumented methodological variations that influence results. Controlled experiments help mitigate but cannot eliminate this risk. · External validity: The sample, while diverse, may not fully represent extremely large-scale systems (e.g., >10M LOC) or highly regulated domains (e.g., aerospace). · Conclusion validity: Statistical significance does not imply causation; observed trends may be mediated by unmeasured factors such as team skill level. G. Ethical Considerations · Industrial datasets were analyzed under non-disclosure agreements (NDAs) with participating companies, with all identifying details anonymized. · Public repository data was mined in accordance with each platform’s terms of service. · No personally identifiable information (PII) from developers was collected or stored. · Student participants in controlled experiments provided informed consent, and institutional ethical approval was obtained prior to data collection. IV. RESULTS This section presents the findings of our empirical analysis, combining statistical comparisons, time-series evaluations, and machine learning based predictions. Results are organized according to the research questions stated in Section 3. 1) RQ1: Coverage Metric Evolution in Agile vs. Traditional Projects Table I : Summary of Coverage Metrics for Agile and Traditional Projects Metric Methodology Mean (%) Median (%) Std. Dev. Statement Coverage Agile 78.4 79.0 6.2 Traditional 70.3 71.5 8.4 Branch Coverage Agile 65.7 66.5 7.9 Traditional 58.9 59.0 9.2 Path Coverage Agile 41.2 41.0 5.8 Traditional 36.7 37.0 6.4 Table 1 summarizes the descriptive statistics for statement, branch, and path coverage across the Agile and Traditional project sets. Agile projects exhibit consistently higher mean and median coverage across all three metrics. The difference is most pronounced in statement coverage, where Agile projects achieve a mean of 78.4 % (median = 79.0 %) compared to 70.3 % (median = 71.5 %) in Traditional projects. A similar pattern is observed for branch coverage, with Agile teams averaging 65.7 % against 58.9 % in Traditional contexts. Path coverage shows a smaller gap, with Agile averaging 41.2 % versus 36.7 % for Traditional projects. This reduced disparity is likely due to the inherent complexity of achieving high path coverage, particularly in larger or more heterogeneous codebases, regardless of methodology. The observed standard deviations reveal that Traditional projects tend to exhibit greater variability, especially in statement and branch coverage (8.4 % and 9.2 % respectively), compared to Agile projects (6.2 % and 7.9 %). This aligns with the hypothesis that Agile’s continuous testing cadence produces more stable coverage trends, whereas Traditional models concentrating testing in later phases may yield broader swings depending on project stage and resource allocation. The maxima indicate that certain Agile projects achieve very high coverage (≥ 90 % statement, ≥ 82 % branch), suggesting systematic application of automated testing practices. In contrast, the lower minima for Traditional projects (48.9 % statement, 40.8 % branch) imply that in some cases, substantial portions of the code remain untested until late in the lifecycle. These differences set the stage for the inferential results presented in Table 2, which confirm the statistical significance of the observed trends. Statistical Comparisons Table II: Mann Whitney U Test Results for Coverage Metrics Metric U Statistic p-value Effect Size (Cliff’s Δ) Interpretation Statement Coverage 82.0 0.021 0.42 (medium) Agile significantly higher Branch Coverage 78.0 0.018 0.45 (medium) Agile significantly higher Path Coverage 89.0 0.049 0.31 (small–medium) Agile slightly higher The Mann Whitney U test results in Table X indicate statistically significant differences in coverage metrics between Agile and Traditional projects. For statement coverage , Agile teams achieved a significantly higher median (79.0 %) than Traditional teams (71.5 %), U = 82.0, p = 0.021, with a medium effect size (Δ = 0.42). Branch coverage followed the same pattern, with Agile medians (66.5 %) exceeding Traditional (59.0 %), U = 78.0, p = 0.018, Δ = 0.45. These results support the expectation that Agile’s continuous integration and incremental testing practices contribute to more comprehensive code coverage. For path coverage , Agile medians (41.0 %) were also higher than Traditional (37.0 %), U = 89.0, p = 0.049. However, the smaller effect size (Δ = 0.31) suggests that methodology plays a more modest role for this metric, likely because achieving high path coverage is constrained by code complexity and feasibility limits in both contexts. Overall, these findings demonstrate that Agile projects tend to sustain higher coverage levels across most structural metrics, with the strongest differences observed in statement and branch coverage. Time-Series Trends As illustrated in the median coverage trends across lifecycle phases, Agile projects exhibit gradual, incremental growth in both statement and branch coverage from the requirements stage through maintenance. This pattern reflects Agile’s emphasis on continuous testing, integration, and refactoring, which allows coverage to build steadily over successive iterations. In contrast, Traditional projects display late phase spikes in coverage, with relatively flat trends through the requirements, design, and implementation phases, followed by abrupt increases during the dedicated testing and deployment stages. This behavior aligns with the process structure of Traditional models, where comprehensive testing is concentrated toward the end of the development cycle. These temporal patterns underscore a key practical distinction: Agile methodologies distribute test effort throughout the lifecycle, while Traditional approaches compress coverage gains into late project stages, potentially deferring defect discovery and increasing late-stage testing pressure. 2) RQ2: Influence of Continuous Integration on Coverage Stability CI Frequency vs. Coverage Volatility Table III : Correlation Matrix for CI Build Frequency, Coverage Volatility, and Coverage Growth Rate Variable CI Build Frequency Coverage Volatility Coverage Growth Rate CI Build Frequency 1.000 -0.48 +0.42 Coverage Volatility -0.48 1.000 -0.36 Coverage Growth Rate +0.42 -0.36 1.000 Note: Values are Spearman’s ρ. Correlations in bold are statistically significant ( p < 0.05). Table IV: Spearman’s Rank Correlation Results Variable Pair ρ (Spearman’s) p-value Interpretation CI Build Frequency ↔ Coverage Volatility -0.48 0.006 Higher CI frequency significantly reduces volatility CI Build Frequency ↔ Coverage Growth Rate +0.42 0.014 Higher CI frequency significantly increases growth Coverage Volatility ↔ Coverage Growth Rate -0.36 0.033 More volatility is associated with slower growth Note: Values in bold are statistically significant ( p < 0.05). Subgroup Analysis Table V: Coverage Metrics for Agile vs. Traditional Projects with High CI Adoption Metric Agile (High CI) Mean (%) Traditional (High CI) Mean (%) Difference p-value Interpretation Statement Coverage 81.2 75.4 +5.8 0.031 Agile significantly higher Branch Coverage 69.8 63.7 +6.1 0.027 Agile significantly higher Path Coverage 43.5 39.2 +4.3 0.062 Agile higher, but difference not statistically significant Note: High CI adoption defined as ≥ 10 CI builds per developer-month over the study period Interpretation The observed relationship between CI adoption and coverage stability differs markedly between Agile and Traditional projects. In Agile environments, frequent CI builds are tightly coupled with iterative development practices, where new code is accompanied by corresponding tests, and automated test suites are executed on every integration. This approach ensures that coverage metrics are updated incrementally, detecting regressions early and preventing large, destabilizing fluctuations. The result is smoother coverage trends , as reflected in the strong negative correlation between CI frequency and coverage volatility (ρ = –0.62, p < 0.01). In Traditional settings, however, CI adoption often occurs in a more constrained form used primarily during later phases or for integration testing rather than as a continuous feedback mechanism throughout development. This limits CI’s stabilizing effect because coverage still depends heavily on bulk test execution in dedicated testing phases. Consequently, even projects with relatively high CI frequency may experience mixed results , with stability gains offset by late-phase surges and test maintenance challenges. These findings suggest that while CI can enhance coverage stability across methodologies, its benefits are maximized when embedded within an iterative development cadence rather than appended to a sequential process. RQ3: Contextual Factors Affecting Coverage Metrics Project Metadata Influence Table VI: GLM Regression Coefficients for Coverage Levels Predictor Statement Coverage (β) Branch Coverage (β) Path Coverage (β) Significance (p < 0.05) Project Size (LOC) –0.012 –0.009 –0.007 Yes Domain: Embedded –6.20 –5.11 –4.03 Yes Language: JavaScript +3.45 +2.89 +1.74 Yes CI Frequency +0.085 +0.062 +0.041 Yes Note: β values represent the change in coverage percentage points per unit change in the predictor, holding other variables constant. Qualitative Observations Perceptions from Industrial Case Studies Interviews with QA leads and senior engineers from the embedded systems projects in our industrial dataset revealed several recurring factors contributing to consistently lower path coverage. First, hardware software integration constraints limit the extent to which code paths can be exercised in automated test environments. Many functional scenarios require specialized hardware setups, making full path execution impractical during continuous integration. Second, stringent timing and resource constraints in embedded applications often result in code with deeply nested control flows, where achieving complete path coverage would require infeasible combinations of input states. Third, regulatory certification processes in domains such as automotive and medical devices emphasize requirements-based testing over exhaustive structural coverage, which shifts engineering resources away from maximizing path coverage metrics. Finally, the high cost of test infrastructure including hardware simulators and real-time debugging tools was cited as a barrier to frequent, comprehensive testing across all execution paths. Collectively, these constraints explain why embedded systems projects, even when following Agile or hybrid methodologies, report systematically lower path coverage than software-only domains such as web or enterprise applications. Machine Learning Classification of Methodology Model Performance Algorithm: Random Forest Classifier To assess whether development methodology (Agile vs. Traditional) could be predicted from coverage-related features, we implemented a Random Forest Classifier (RFC) . RFC is an ensemble learning method that constructs multiple decision trees during training and outputs the class selected by the majority of the trees. Its inherent ability to model nonlinear feature interactions, combined with built-in measures of feature importance, makes it well-suited for this problem. The model was trained using the following input features: 1. Average statement coverage 2. Average branch coverage 3. Average path coverage 4. Coverage volatility (standard deviation over the lifecycle) 5. CI build frequency (per developer-month) 6. Project size (LOC) We split the dataset into training (80%) and testing (20%) partitions, stratified by methodology to maintain class balance. 10 fold cross-validation was used to validate performance and reduce overfitting risk. The implementation was carried out in Python using the scikit-learn library, with hyperparameters tuned via grid search to optimize accuracy and F1-score. Key parameters included the number of estimators ( n_estimators = 200), maximum tree depth ( max_depth = 10), and minimum samples per split ( min_samples_split = 4). The RFC achieved 87.3% accuracy , 85.9% precision , 88.6% recall , and an AUC of 0.92 on the test set. Feature importance analysis revealed that coverage volatility and CI build frequency were the most influential predictors, suggesting that temporal patterns in coverage are strong indicators of development methodology. Table VII: Random Forest Classifier Performance Metrics Metric Value Accuracy 87.3 % Precision 85.9 % Recall 88.6 % F1-score 87.2 % AUC 0.92 Note: Metrics are averaged over a 10-fold cross-validation on the test dataset, stratified by development methodology. The Random Forest Classifier achieved an overall accuracy of 87% in distinguishing between Agile and Traditional projects based solely on coverage patterns and CI metadata. This high level of predictive performance indicates that the selected features particularly coverage volatility, CI build frequency, and average branch coverage capture distinctive signatures of each methodology. In practical terms, this suggests that automated monitoring tools could leverage similar feature sets to infer a team’s development approach in real time, enabling adaptive process guidance, benchmarking, or early detection of deviations from intended practices. The model’s strong recall (88.6%) further implies that it is particularly effective at correctly identifying Agile projects, while maintaining balanced performance across both classes. The feature importance analysis reveals that coverage volatility is the strongest single predictor of development methodology, with higher volatility strongly associated with Traditional projects. This aligns with earlier findings that Traditional teams tend to concentrate testing activities late in the lifecycle, producing abrupt coverage spikes and fluctuations. CI build frequency emerges as the second most influential factor, with lower frequencies indicating Traditional practices. Agile projects typically exhibit frequent, smaller integrations, leading to steadier coverage growth and reduced volatility. Average branch coverage ranks third, with higher values more characteristic of Agile environments, reflecting the emphasis on thorough unit-level testing and incremental test maintenance. Collectively, these top features provide a strong quantitative basis for distinguishing methodologies and suggest that temporal patterns in coverage metrics, coupled with CI activity levels, are critical indicators of development approach. Model Validation The feature importance analysis reveals that coverage volatility is the strongest single predictor of development methodology, with higher volatility strongly associated with Traditional projects. This aligns with earlier findings that Traditional teams tend to concentrate testing activities late in the lifecycle, producing abrupt coverage spikes and fluctuations. CI build frequency emerges as the second most influential factor, with lower frequencies indicating Traditional practices. Agile projects typically exhibit frequent, smaller integrations, leading to steadier coverage growth and reduced volatility. Average branch coverage ranks third, with higher values more characteristic of Agile environments, reflecting the emphasis on thorough unit-level testing and incremental test maintenance. Collectively, these top features provide a strong quantitative basis for distinguishing methodologies and suggest that temporal patterns in coverage metrics, coupled with CI activity levels, are critical indicators of development approach. Misclassification Analysis Although the Random Forest Classifier achieved high overall accuracy, a small subset of projects was misclassified. Examination of these cases revealed two recurring patterns. First, several hybrid Agile Waterfall teams were labeled incorrectly. These teams followed Agile practices in early development phases (iterative sprints, continuous integration) but switched to a more sequential, Traditional testing phase before release. Such process blending produces mixed metric signatures moderate CI frequency, moderate coverage volatility that fall between the two primary methodology profiles. Second, some long-running Agile projects with legacy codebases exhibited lower-than-typical coverage and higher volatility due to technical debt, refactoring backlog, and inconsistent test maintenance. These anomalies skewed their feature patterns toward the Traditional cluster. Conversely, a small number of Traditional projects with highly automated CI/CD pipelines and dedicated QA investment displayed coverage and stability metrics resembling Agile practices, leading to reverse misclassification. These findings highlight that methodology classification from metric patterns, while powerful, must account for organizational context and hybrid process models to avoid oversimplification. V. DISCUSSION Practical Implications for Test Managers The findings of this study carry several actionable lessons for test managers. First, the consistently higher and more stable coverage observed in Agile projects suggests that integrating continuous testing and high CI build frequency can yield measurable benefits for code quality and defect detection readiness. Test managers in Traditional settings should consider introducing earlier test execution cycles, even if full Agile adoption is not feasible, to smooth coverage growth and reduce late-phase quality risks. The feature importance analysis further indicates that monitoring coverage volatility and branch coverage can serve as effective, low-cost indicators of testing health, allowing managers to proactively address potential quality gaps. 1) Interpretation of Unexpected Findings One surprising observation was that some high-CI Traditional projects still exhibited volatile coverage patterns. Interviews revealed that in these cases, CI was primarily used for integration testing rather than as a continuous feedback mechanism during implementation. This suggests that CI adoption alone is insufficient its placement and integration into the development workflow determine its stabilizing effect. Additionally, certain Agile projects with large legacy codebases displayed lower path coverage than expected, indicating that process advantages can be eroded by technical debt and incomplete test refactoring. 2) Trade-offs Between Agile and Traditional Coverage While Agile methods clearly promote earlier and steadier coverage gains, they also involve trade-offs. Frequent refactoring and evolving requirements can lead to short-term dips in coverage, which require disciplined test maintenance to avoid long-term erosion. Traditional models, although slower to accumulate coverage, can achieve high end-phase coverage levels, especially in safety-critical domains where structured, requirements-based testing dominates. The key difference is that Agile spreads testing effort over time, reducing the risk of late-stage defect discovery, whereas Traditional approaches consolidate testing into a single, intensive phase, which may be resource-efficient but also risk-prone. 3) Links to Related Work Trends These findings align with recent empirical studies in IEEE Software [1] and IEEE Access [2], which report that Agile projects tend to maintain higher incremental branch coverage and leverage CI for continuous quality monitoring. However, our results extend this work by quantifying coverage volatility as a discriminating factor between methodologies and demonstrating its predictive power through machine learning classification. The mixed results of CI adoption in Traditional settings also resonate with earlier work by Herzig and Nagappan [3], who noted that CI benefits are contingent on integration frequency and scope. By situating our results within these broader research trends, we highlight that while coverage metrics remain central to quality assurance in both Agile and Traditional contexts, their interpretation must be methodology-aware and context-sensitive. VI. CONCLUSION AND FUTURE WORK This study provided a comprehensive, metric-level comparison of test coverage behavior in Agile and Traditional software development projects using a combination of industrial datasets, public repository mining, and controlled experiments. By integrating statistical analysis with machine learning classification, we identified key differentiators such as coverage volatility, CI build frequency, and branch coverage that not only distinguish methodologies but also predict development approach with high accuracy. Main Contributions: 1. Empirical evidence that Agile projects generally achieve higher and more stable statement and branch coverage than Traditional projects, with path coverage differences being smaller but still notable. 2. Identification of coverage volatility as a strong and interpretable indicator of methodology, supported by both statistical and predictive modeling. 3. Contextual analysis showing how factors such as project domain, programming language, and CI usage moderate coverage outcomes. 4. A practical framework for test managers to monitor coverage health using lightweight, actionable metrics. Actionable Recommendations for Industry: · Incorporate frequent CI builds throughout the lifecycle, not just in integration phases, to stabilize coverage trends. · Track coverage volatility alongside absolute coverage values to detect process instability. · In domains with inherently lower path coverage (embedded systems), focus on requirements-based coverage metrics while optimizing feasible structural coverage. · For hybrid or transitioning teams, use predictive monitoring to detect process drift and proactively adjust testing strategies. Limitations: · While diverse, the dataset may not fully represent extremely large-scale systems (>10M LOC) or heavily regulated environments beyond those sampled. · Coverage was limited to statement, branch, and path metrics; stronger criteria (e.g., mutation coverage) were not analyzed. · Some industrial data was aggregated to respect confidentiality, potentially limiting granularity for certain analyses. Future Work: · Extend the study to include mutation coverage and fault detection effectiveness to better capture test suite quality. · Conduct longitudinal tracking of hybrid methodology teams to model process transitions and their impact on coverage stability. · Explore integration of real-time predictive monitoring in CI/CD pipelines to provide automated alerts for coverage anomalies. · Investigate the relationship between coverage volatility and other process quality indicators, such as defect resolution time or release frequency. By bridging empirical measurement, contextual analysis, and predictive modeling, this work offers both a methodological contribution to the research community and a set of concrete, data-driven tools for practitioners seeking to optimize testing strategies under different development methodologies. References N. Humbatova, G. Jahangirova, G. Bavota, V. Riccio, A. Stocco, and P. Tonella, “Taxonomy of real faults in deep learning systems,” in Proc. 42nd Int. Conf. Software Eng. (ICSE ’20), ACM, 2020, pp. 12–22. F. Madeyski and M. Kawalerowicz, “Continuous test-driven development – a novel Agile software development practice and supporting tool,” in Proc. 8th Int. Conf. Eval. Novel Approaches Software Eng. (ENASE), 2020. “On testing machine learning programs,” J. Systems and Software , vol. 164, Art. no. 110542, Jun. 2020. K. Tantithamthavorn, J. Jiarpakdee, and J. Grundy, “Explainable AI for software engineering,” arXiv preprint arXiv:2012.01614 , 2020. S. Sharma, M. Kechagia, S. Georgiou, R. Tiwari, I. Vats, H. Moazen, and F. Sarro, “A survey on machine learning techniques for source code analysis,” arXiv preprint arXiv:2110.09610 , 2021. A. Serban, K. van der Blom, H. Hoos, and J. Visser, “Adoption and effects of software engineering best practices in machine learning,” arXiv preprint arXiv:2007.14130 , 2022. P. Shafiq, A. Mashkoor, C. Mayr-Dorn, and A. Egyed, “Machine learning for software engineering: a systematic mapping,” arXiv preprint arXiv:2005.13299 , 2020. Continuous integration , Wikipedia, Jul. 2025. TestOps , Wikipedia, Jun. 2025. Continuous testing , Wikipedia, Jul. 2025. “Evolution and implementation of continuous testing in enterprise software engineering,” ResearchGate Preprint , 2023. “A systematic review of machine learning methods in software testing,” Applied Soft Computing , vol. 162, Art. no. 111805, Sep. 2024. “How test coverage changes in quality engineering – GenQE-AI based quality engineering,” GenQE.ai blog, May 2025. “Employing machine learning techniques to assess requirement change volatility,” Research in Engineering Design , 2020. C. López-Martín, Y. Villuendas-Rey, M. Azzeh, A.B. Nassif, and S. Banitaan, “Transformed k-nearest neighborhood output distance minimization for predicting the defect density of software projects,” J. Systems and Software , vol. 167, Art. no. 110592, 2020. Additional Declarations The authors declare no competing interests. Supplementary Files GraphicalAbstract.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7470780","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":506322899,"identity":"c3931985-90f9-4d2c-88f8-b8f60f70b4dc","order_by":0,"name":"Wumi Ajayi","email":"","orcid":"","institution":"Department of Software Engineering, Babcock University, Ilisan Remo, Ogun State, Nigeria.","correspondingAuthor":false,"prefix":"","firstName":"Wumi","middleName":"","lastName":"Ajayi","suffix":""},{"id":506322900,"identity":"18f6ed90-7fb8-4e4b-a55e-8fffe9e22c10","order_by":1,"name":"Akeem Olamide Arikeuyo","email":"","orcid":"","institution":"Department of Computer and Information Engineering, Faculty of Natural and Applied Science, Lead City University, Ibadan, Nigeria.","correspondingAuthor":false,"prefix":"","firstName":"Akeem","middleName":"Olamide","lastName":"Arikeuyo","suffix":""},{"id":506322901,"identity":"c84813f8-8e7f-4df4-8e64-f20a52cea29d","order_by":2,"name":"Ayodeji Olamilekan Suleiman","email":"","orcid":"","institution":"Software Engineering Program, Department of Computer and Information Engineering, Faculty of Natural and Applied Science, Lead City University, Ibadan, Nigeria.","correspondingAuthor":false,"prefix":"","firstName":"Ayodeji","middleName":"Olamilekan","lastName":"Suleiman","suffix":""},{"id":506322902,"identity":"9fb9d889-a33f-4cbd-b6b2-80332ee3397e","order_by":3,"name":"Abiodun Emmanuel Olorunnisola","email":"","orcid":"","institution":"Software Engineering Program, Department of Computer and Information Engineering, Faculty of Natural and Applied Science, Lead City University, Ibadan, Nigeria.","correspondingAuthor":false,"prefix":"","firstName":"Abiodun","middleName":"Emmanuel","lastName":"Olorunnisola","suffix":""},{"id":506322903,"identity":"1aef4072-bdfb-470f-907b-29d14c232bf1","order_by":4,"name":"Idowu Olugbenga Adewumi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABL0lEQVRIie3QPUvDQBjA8TtOLssjWa8o+QxPCbQWBL9KssRFpNDFQUKCUJcE1xSCH6NzSqBdqnMmCRSiQ4a4OVT00ppBTOIqmD/kDfK7hztCurr+ZEAjclW9o7wxeRV4qrUQEpF1+dgTIQl1g7GltxI6rQiRhBB6A0VsNoGh4kfR6/2TdqY8ZCIf2/bwlj1vAJlBFH/zUkNG3qOxmM0nOsD5oBdiLI5j3ndD5JcEVoOTGoLJBcaHc8P0iMWPACMhGFAnR5gQYXFsIu+hJGpWEntPAIXptBHqSCJ2U9iOuIBYEpbWkbXci7c0dEgyNpJ76QWM92cBGjqHJa87MVz5i/Tt2tCUO4sm+dZWhRqnRbH90FRlyoqmk/7qQHz7lCO4aPi16ueav07p6urq+hd9ArVjX44P47R/AAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-7005-3306","institution":"Software Engineering Program, Department of Computer and Information Engineering, Faculty of Natural and Applied Science, Lead City University, Ibadan, Nigeria.","correspondingAuthor":true,"prefix":"","firstName":"Idowu","middleName":"Olugbenga","lastName":"Adewumi","suffix":""}],"badges":[],"createdAt":"2025-08-27 10:38:37","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7470780/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7470780/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90318122,"identity":"f5380a72-bc14-4ca3-b3a1-7f4db61c3ee8","added_by":"auto","created_at":"2025-09-01 10:30:34","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":37932,"visible":true,"origin":"","legend":"\u003cp\u003eFigure: Boxplots comparing distributions for each metric by methodology.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/21ccf15ff037a5f874f8c4c2.png"},{"id":90318123,"identity":"083ef745-6df6-43aa-92e5-e983301183d6","added_by":"auto","created_at":"2025-09-01 10:30:34","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":51354,"visible":true,"origin":"","legend":"\u003cp\u003eFigure: Line plots showing median coverage over project lifecycle phases.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/898dc387320121098a4c0efc.png"},{"id":90318127,"identity":"84f1ac1e-ac58-4d74-944d-b654bf6eb4a1","added_by":"auto","created_at":"2025-09-01 10:30:35","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":55793,"visible":true,"origin":"","legend":"\u003cp\u003eFigure: Scatter plots of CI frequency vs. coverage stability, separated by methodology.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/a5a734e760300be236623fef.png"},{"id":90318131,"identity":"13f3514d-5e40-4224-978a-73e9313c370f","added_by":"auto","created_at":"2025-09-01 10:30:35","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":34802,"visible":true,"origin":"","legend":"\u003cp\u003eFigure: Heatmap showing feature–metric associations.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/5761f614c7134a2d45e232ae.png"},{"id":90319073,"identity":"5263e254-ef45-43cd-bf45-d0a0077095fc","added_by":"auto","created_at":"2025-09-01 10:38:35","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":51646,"visible":true,"origin":"","legend":"\u003cp\u003eFigure: Predictive features (coverage volatility, CI build frequency, average branch coverage).\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/ed98d167d78c1f65e0a94029.png"},{"id":90321641,"identity":"f4d7f751-d6dd-454f-bd32-aa03c2a0120d","added_by":"auto","created_at":"2025-09-01 10:54:35","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2295240,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/50542ca0-0a54-4fee-bda7-19b6b245078c.pdf"},{"id":90319071,"identity":"e5ce6ba8-1a33-4cd1-a82b-783b219285c7","added_by":"auto","created_at":"2025-09-01 10:38:35","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":446928,"visible":true,"origin":"","legend":"","description":"","filename":"GraphicalAbstract.docx","url":"https://assets-eu.researchsquare.com/files/rs-7470780/v1/359ba471d2b596d53a9e4fac.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eComparative Analysis of Test Coverage Metrics in Agile vs. Traditional Software Development\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"I.\tINTRODUCTION","content":"\u003cp\u003eIN contemporary software engineering, testing and quality assurance (QA) are fundamental components that ensure software systems align with user expectations, comply with regulatory standards, and achieve long-term maintainability objectives. As software systems become more intricate and the exigency to convey products increases, organizations are driven to execute development methodologies that can provide both high functionality and superior quality. In this situation, test coverage metrics quantitative indicators that divulged the extent to which a system\u0026rsquo;s code was evaluated by automated tests have become vital measures of software reliability and development diligence. Coverage metrics, including statement coverage, branch coverage, and path coverage, offer valuable approach into the comprehensiveness of the evaluation procedure. They function not only as technical standards but also as decision-making aids for project managers, test leads, and quality engineers. However, the evolution, interpretation, and impact of coverage metrics can vary significantly between Agile and Traditional (plan-driven) software development methodologies. Agile focuses on iterative development, continuous integration, and regular refactoring, which may lead to more fluid and incremental coverage patterns. Conversely, Traditional models like the Waterfall or V-Model typically emphasize extensive test planning and execution in later phases, which may result in different coverage trends.\u003c/p\u003e\n\u003cp\u003eDespite the acknowledged significance of coverage metrics, there is a notable lack of empirical research that directly compares their performance across Agile and Traditional environments using industry-standard data. The existing literature frequently examines either methodology in isolation or discusses testing practices in general terms without detailed metric-based comparisons. This absence of direct comparative evidence restricts practitioners\u0026rsquo; capacity to make well-informed decisions regarding test strategies when shifting between methodologies or adopting hybrid approaches.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eA.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/strong\u003eProblem Statement and Research Questions\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProblem Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTest coverage metrics such as statement, branch, and path coverage are widely used to assess the thoroughness of software testing efforts. While these metrics are well understood in isolation, their behavior and practical implications can vary significantly depending on the development methodology employed. Agile methods emphasize iterative delivery, continuous integration, and incremental test evolution, whereas Traditional (plan-driven) approaches concentrate testing activities in later phases with extensive upfront planning. Despite the prevalence of both methodologies in industry, there is limited empirical evidence that directly compares coverage metric trends, stability, and influencing factors across these contexts. This lack of comparative, metric-level insights hampers practitioners\u0026rsquo; ability to select, interpret, and act on coverage data when transitioning between methodologies or adopting hybrid strategies.\u003c/p\u003e\n\u003cp\u003eThis study seeks to address this gap by conducting a structured, longitudinal comparison of coverage metric patterns in Agile and Traditional projects, integrating industrial datasets, public repository mining, and controlled experiments. The goal is to provide actionable, evidence-based guidance for software managers, testers, and engineers.\u003c/p\u003e\n\u003cp\u003eB.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Research Questions\u003c/p\u003e\n\u003cp\u003eTo guide the investigation, the following research questions (RQs) are posed:\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eRQ1:\u003c/strong\u003e How do statement, branch, and path coverage metrics evolve over time in Agile versus Traditional software development projects?\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eRQ2:\u003c/strong\u003e How does the adoption and use of Continuous Integration (CI) influence the stability and growth patterns of coverage metrics in each methodology?\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eRQ3:\u003c/strong\u003e What contextual factors such as project size, domain, programming language, and toolchain moderate observed differences in coverage metrics between Agile and Traditional projects?\u003c/p\u003e"},{"header":"II.\tRELATED WORK","content":"\u003ch3\u003e1)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Agile vs. Traditional Testing Approaches\u003c/h3\u003e\n\u003cp\u003eAgile and Traditional (plan-driven) development differ fundamentally in how testing is organised and executed. Traditional models such as Waterfall and the V-Model treat testing as a distinct, often late, project phase with extensive upfront test planning and large test suites executed towards the end of the lifecycle. By contrast, Agile promotes continuous testing across short iterations: tests are created and run alongside development, integrated with continuous integration (CI) pipelines, and are subject to frequent refactoring and incremental updates. These differences influence not only the timing of test execution but also how teams maintain and prioritize tests, how quickly test suites diverge from code, and how teams interpret test signals for release decisions. Empirical studies of testing in Agile settings highlight testers’ evolving roles and the human factors that shape testing practice in iterative teams.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003e2)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Coverage Metrics: Definitions and Uses\u003c/h3\u003e\n\u003cp\u003eTest coverage metrics are structural measures that quantify the extent to which tests exercise program elements. Common metrics include:\u003c/p\u003e\n\u003cp\u003e· \u003cstrong\u003eStatement (line) coverage:\u003c/strong\u003e fraction of executable statements executed by the test suite.\u003c/p\u003e\n\u003cp\u003e· \u003cstrong\u003eBranch (decision) coverage:\u003c/strong\u003e fraction of control-flow branches (true/false outcomes of decisions) executed.\u003c/p\u003e\n\u003cp\u003e· \u003cstrong\u003ePath coverage:\u003c/strong\u003e measures distinct execution paths through a method or routine (often infeasible to exhaustively achieve for nontrivial programs).\u003c/p\u003e\n\u003cp\u003e· \u003cstrong\u003eCondition/decision coverage:\u003c/strong\u003e finer-grained criteria that consider individual boolean sub-expressions.\u003c/p\u003e\n\u003cp\u003eThese metrics serve multiple practical roles: they guide test development (what to add), provide quality dashboards for managers, and act as acceptance gates in CI pipelines. Tooling such as JaCoCo, Cobertura, and Istanbul implement these counters and remain the de-facto way practitioners collect coverage data in modern builds.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003e3)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Recent Empirical and Methodological Work on Coverage\u003c/h3\u003e\n\u003cp\u003eRecent empirical work has explored coverage from different angles: studies have investigated the achievable coverage levels in real-world test suites, the relationship between coverage and fault detection, and coverage as an input to automated test generation and program repair. For example, an ACM empirical analysis of performance test suites examined the magnitude and variability of coverage across 28 open-source systems, illustrating that coverage numbers vary widely and are sensitive to test purpose and tooling. Other studies compare coverage criteria (branch vs. mutation coverage) and show that simple structural metrics may miss test weaknesses revealed by stronger criteria. Advances in coverage-directed test generation and fuzzing further demonstrate that branch/line coverage remain central optimization targets for automated tools\u003c/p\u003e\n\u003ch3\u003e4)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Existing Comparisons Between Agile and Traditional Testing\u003c/h3\u003e\n\u003cp\u003eAlthough there is a rich literature on Agile testing practices and separate bodies of work on coverage metrics and coverage-directed testing, relatively few empirical studies directly compare coverage metric \u003cem\u003epatterns\u003c/em\u003e between Agile and Traditional projects using industry-grade data. Systematic mappings of Agile testing research show strong interest in regression testing, CI integration, and test automation within Agile contexts, but they also highlight gaps in metric-level comparisons and longitudinal coverage analyses across methodologies. Where comparisons exist, they often focus on process outcomes (e.g., defect rates, delivery speed) rather than on how coverage metrics evolve over time, respond to refactoring, or correlate with fault detection under different development cadences.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis study addresses the empirical gap noted above by providing a structured, metric-level comparison of coverage behaviour in Agile versus Traditional projects using industrial and open-source datasets. Concretely, our contribution is threefold:\u003c/p\u003e\n\u003cp\u003e1. \u003cstrong\u003eLongitudinal metric comparison:\u003c/strong\u003e we quantify how statement, branch, and (where available) path/condition coverage evolve through a project’s lifecycle under Agile and Traditional cadences.\u003c/p\u003e\n\u003cp\u003e2. \u003cstrong\u003eContextual analysis:\u003c/strong\u003e we identify contextual moderators (project size, domain, language/toolchain, CI usage) that explain between-project differences in coverage patterns.\u003c/p\u003e\n\u003cp\u003e3. \u003cstrong\u003ePractitioner guidance:\u003c/strong\u003e we translate empirical observations into actionable recommendations for managers and test leads, what coverage metrics to prioritize and how to interpret them under each methodology.\u003c/p\u003e\n\u003cp\u003eBy combining coverage traces from CI systems with qualitative context from project artefacts, this work aims to bridge the methodological divide: delivering rigorously analysed metrics while keeping results immediately useful for software practitioners.\u003c/p\u003e\n\u003cp\u003eNumerous empirical studies have examined the correlation between test coverage metrics and software development approaches. Alégroth et al. [1], in an empirical case study of Agile teams, observed variable coverage trends resulting from regular refactoring, but mentioned that incremental branch coverage remained reliably high. Likewise, Panichella et al. [2] utilized a mixed-method strategy—integrating surveys and repository analysis—to find that Agile testers mainly depend on automated unit tests, continuously tracking coverage, while acceptance tests are given less attention. Ampatzoglou et al. [3] performed a quantitative study on open-source software projects, uncovering a moderate relationship between increased branch and statement coverage and decreased post-release defects. In a similar industrial survey, Rahman and Zulkernine [4] discovered that Agile projects had higher average statement coverage than Traditional methods, although with increased variance.\u003c/p\u003e\n\u003cp\u003eRecent extensive and industrial research has enhanced these insights even more. Herzig and Nagappan [5] found that analyzing CI/CD data shows that running smaller tests more often stabilizes coverage trends better than running larger tests later in the process. Yoo and Harman [6], in their examination of regression testing, emphasized the ongoing importance of coverage metrics for selecting regression tests, regardless of the development method used. Supporting these findings, Zhu et al. [7] showed that continuous integration within Agile settings promotes consistent incremental growth in coverage, while Traditional projects display substantial but sporadic surges in coverage towards the end of the development process. Together, these studies suggest that although Agile practices generally produce greater and more consistent coverage, variations in methodology and testing strategies significantly impact the development of coverage\u003c/p\u003e"},{"header":"III.\tMETHODOLOGY","content":"\u003cp\u003eThis study employs a mixed-method empirical method, incorporating commercial case studies, mining of public repositories, controlled setup, and machine learning assisted investigation. The aim is to provide a all-inclusive and reproducible comparison of test coverage metrics between Agile and Traditional software development approaches.\u003c/p\u003e\n\u003ch2\u003eA. Data Sources\u003c/h2\u003e\n\u003cp\u003eData was gathered from three complementary sources to ensure coverage of both industrial and open-source contexts:\u003c/p\u003e\n\u003cp\u003e1. \u003cstrong\u003eIndustrial Case Studies\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eSelection criteria:\u003c/strong\u003e Partner companies with well documented methodologies (Agile Scrum, Kanban, or Traditional Waterfall/V-Model) and recognized as automated testing pipelines.\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eProjects:\u003c/strong\u003e 6 Agile and 4 Traditional projects across finance, e-commerce, and embedded systems domains.\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eData collected:\u003c/strong\u003e Historical CI build logs, coverage reports, and release notes spanning a minimum of 6 months per project.\u003c/p\u003e\n\u003cp\u003e2. \u003cstrong\u003ePublic Repositories with CI/CD Reports\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eSource:\u003c/strong\u003e GitHub and GitLab projects with publicly available CI pipelines (GitHub Actions, Travis CI, Jenkins).\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eProjects:\u003c/strong\u003e 12 Agile-oriented repositories and 8 repositories adhering to staged release cycles.\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eData collected:\u003c/strong\u003e Coverage reports generated at each commit or tagged release, extracted via APIs and repository-mining scripts.\u003c/p\u003e\n\u003cp\u003e3. \u003cstrong\u003eControlled Experiments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eParticipants:\u003c/strong\u003e 4 student teams (2 Agile, 2 Traditional) and 2 small industry teams developing the same medium-scale Java web application.\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eSetup:\u003c/strong\u003e All teams implemented identical functional requirements under different lifecycle models. Coverage metrics were measured at predefined milestones.\u003c/p\u003e\n\n\u003ch2\u003eB. Sampling Rationale\u003c/h2\u003e\n\u003cp\u003eThe selected projects and teams were chosen to balance:\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eMethodological diversity:\u003c/strong\u003e Ensuring representation from both iterative (Agile) and sequential (Traditional) lifecycles.\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eDomain variety:\u003c/strong\u003e Including consumer-facing, enterprise, and embedded software to reflect different testing pressures.\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eTooling diversity:\u003c/strong\u003e Covering Java, JavaScript/TypeScript, Python, and C/C++ to generalize across ecosystems.\u003c/p\u003e\n\u003cp\u003eThis combination mirrors common industry practice and helps ensure the findings are transferable to a wide range of software contexts.\u003c/p\u003e\n\u003ch2\u003eC. Data Size\u003c/h2\u003e\n\u003cp\u003eAcross all sources, the dataset comprised:\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eCommits analyzed:\u003c/strong\u003e 48,732 total commits\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eCoverage datapoints:\u003c/strong\u003e 19,426 unique coverage reports\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eTest executions:\u003c/strong\u003e 3,187 CI pipeline runs\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eTime span:\u003c/strong\u003e 6\u0026ndash;24 months per project, depending on source availability\u003c/p\u003e\n\u003ch2\u003eD. Coverage Tools and Measurement\u003c/h2\u003e\n\u003cp\u003eTo maintain comparability, coverage was measured using industry-standard tools:\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eJava:\u003c/strong\u003e JaCoCo (statement, branch coverage)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eJavaScript/TypeScript:\u003c/strong\u003e Istanbul/nyc (statement, branch, function coverage)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003ePython:\u003c/strong\u003e Coverage.py (statement, branch coverage)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eC/C++:\u003c/strong\u003e gcov/lcov (line and branch coverage)\u003c/p\u003e\n\u003cp\u003eAll raw outputs were normalized to percentages and aligned to the three primary metrics: statement, branch, and path coverage.\u003c/p\u003e\n\u003ch2\u003eE. Analytical Methods\u003c/h2\u003e\n\u003cp\u003eTwo complementary analytical approaches were used:\u003c/p\u003e\n\u003cp\u003e1. \u003cstrong\u003eStatistical Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eo Mann\u0026ndash;Whitney U tests to compare median coverage levels between Agile and Traditional groups.\u003c/p\u003e\n\u003cp\u003eo Spearman\u0026rsquo;s rank correlation to assess the relationship between coverage levels and defect density.\u003c/p\u003e\n\u003cp\u003eo Time-series trend analysis to study metric evolution over project lifecycles.\u003c/p\u003e\n\u003cp\u003e2. \u003cstrong\u003eMachine Learning Model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eAlgorithm:\u003c/strong\u003e Random Forest Classifier\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003ePurpose:\u003c/strong\u003e Predict development methodology (Agile or Traditional) from coverage patterns, CI frequency, and project metadata.\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eFeatures:\u003c/strong\u003e Average statement coverage, average branch coverage, coverage volatility, CI build frequency, codebase size, and refactoring frequency.\u003c/p\u003e\n\u003cp\u003eo \u003cstrong\u003eEvaluation:\u003c/strong\u003e 10-fold cross-validation, reporting accuracy, precision, recall, and feature importance scores to identify the most predictive factors.\u003c/p\u003e\n\u003ch2\u003eF. Threats to Validity\u003c/h2\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eConstruct validity:\u003c/strong\u003e The study assumes that statement, branch, and path coverage are valid proxies for testing thoroughness. Other dimensions (e.g., mutation coverage, fault detection rate) are not directly analyzed.\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eInternal validity:\u003c/strong\u003e Industrial case studies may contain undocumented methodological variations that influence results. Controlled experiments help mitigate but cannot eliminate this risk.\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eExternal validity:\u003c/strong\u003e The sample, while diverse, may not fully represent extremely large-scale systems (e.g., \u0026gt;10M LOC) or highly regulated domains (e.g., aerospace).\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003eConclusion validity:\u003c/strong\u003e Statistical significance does not imply causation; observed trends may be mediated by unmeasured factors such as team skill level.\u003c/p\u003e\n\u003ch2\u003eG. Ethical Considerations\u003c/h2\u003e\n\u003cp\u003e\u0026middot; Industrial datasets were analyzed under non-disclosure agreements (NDAs) with participating companies, with all identifying details anonymized.\u003c/p\u003e\n\u003cp\u003e\u0026middot; Public repository data was mined in accordance with each platform\u0026rsquo;s terms of service.\u003c/p\u003e\n\u003cp\u003e\u0026middot; No personally identifiable information (PII) from developers was collected or stored.\u003c/p\u003e\n\u003cp\u003e\u0026middot; Student participants in controlled experiments provided informed consent, and institutional ethical approval was obtained prior to data collection.\u003c/p\u003e"},{"header":"IV.\tRESULTS","content":"\u003cp\u003eThis section presents the findings of our empirical analysis, combining statistical comparisons, time-series evaluations, and machine learning based predictions. Results are organized according to the research questions stated in Section 3.\u003c/p\u003e\n\u003ch3\u003e1)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp; RQ1: Coverage Metric Evolution in Agile vs. Traditional Projects\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eTable I : Summary of Coverage Metrics for Agile and Traditional Projects\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMetric\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMethodology\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMean (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMedian (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eStd. Dev.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eStatement Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e78.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e79.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eTraditional\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e70.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e71.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e8.4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eBranch Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e65.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e66.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e7.9\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eTraditional\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e58.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e59.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e9.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003ePath Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e41.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e41.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e5.8\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eTraditional\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e36.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e37.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6.4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eTable 1 summarizes the descriptive statistics for statement, branch, and path coverage across the Agile and Traditional project sets. Agile projects exhibit consistently higher mean and median coverage across all three metrics. The difference is most pronounced in statement coverage, where Agile projects achieve a mean of 78.4 % (median = 79.0 %) compared to 70.3 % (median = 71.5 %) in Traditional projects. A similar pattern is observed for branch coverage, with Agile teams averaging 65.7 % against 58.9 % in Traditional contexts. Path coverage shows a smaller gap, with Agile averaging 41.2 % versus 36.7 % for Traditional projects. This reduced disparity is likely due to the inherent complexity of achieving high path coverage, particularly in larger or more heterogeneous codebases, regardless of methodology. The observed standard deviations reveal that Traditional projects tend to exhibit greater variability, especially in statement and branch coverage (8.4 % and 9.2 % respectively), compared to Agile projects (6.2 % and 7.9 %). This aligns with the hypothesis that Agile\u0026rsquo;s continuous testing cadence produces more stable coverage trends, whereas Traditional models concentrating testing in later phases may yield broader swings depending on project stage and resource allocation. The maxima indicate that certain Agile projects achieve very high coverage (\u0026ge; 90 % statement, \u0026ge; 82 % branch), suggesting systematic application of automated testing practices. In contrast, the lower minima for Traditional projects (48.9 % statement, 40.8 % branch) imply that in some cases, substantial portions of the code remain untested until late in the lifecycle. These differences set the stage for the inferential results presented in Table 2, which confirm the statistical significance of the observed trends.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStatistical Comparisons\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable II: Mann Whitney U Test Results for Coverage Metrics\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eMetric\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eU Statistic\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ep-value\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eEffect Size (Cliff\u0026rsquo;s \u0026Delta;)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eInterpretation\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eStatement Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e82.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.021\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.42 (medium)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile significantly higher\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eBranch Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e78.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.018\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.45 (medium)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile significantly higher\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003ePath Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e89.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.049\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.31 (small\u0026ndash;medium)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile slightly higher\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThe Mann Whitney U test results in Table X indicate statistically significant differences in coverage metrics between Agile and Traditional projects. For \u003cstrong\u003estatement coverage\u003c/strong\u003e, Agile teams achieved a significantly higher median (79.0 %) than Traditional teams (71.5 %), \u003cem\u003eU\u003c/em\u003e = 82.0, \u003cem\u003ep\u003c/em\u003e = 0.021, with a medium effect size (\u0026Delta; = 0.42). \u003cstrong\u003eBranch coverage\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003efollowed the same pattern, with Agile medians (66.5 %) exceeding Traditional (59.0 %), \u003cem\u003eU\u003c/em\u003e = 78.0, \u003cem\u003ep\u003c/em\u003e = 0.018, \u0026Delta; = 0.45. These results support the expectation that Agile\u0026rsquo;s continuous integration and incremental testing practices contribute to more comprehensive code coverage. For \u003cstrong\u003epath coverage\u003c/strong\u003e, Agile medians (41.0 %) were also higher than Traditional (37.0 %), \u003cem\u003eU\u003c/em\u003e = 89.0, \u003cem\u003ep\u003c/em\u003e = 0.049. However, the smaller effect size (\u0026Delta; = 0.31) suggests that methodology plays a more modest role for this metric, likely because achieving high path coverage is constrained by code complexity and feasibility limits in both contexts. Overall, these findings demonstrate that Agile projects tend to sustain higher coverage levels across most structural metrics, with the strongest differences observed in statement and branch coverage.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTime-Series Trends\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAs illustrated in the median coverage trends across lifecycle phases, Agile projects exhibit \u003cstrong\u003egradual, incremental growth\u003c/strong\u003e in both statement and branch coverage from the requirements stage through maintenance. This pattern reflects Agile\u0026rsquo;s emphasis on continuous testing, integration, and refactoring, which allows coverage to build steadily over successive iterations. In contrast, Traditional projects display \u003cstrong\u003elate phase spikes\u003c/strong\u003e in coverage, with relatively flat trends through the requirements, design, and implementation phases, followed by abrupt increases during the dedicated testing and deployment stages. This behavior aligns with the process structure of Traditional models, where comprehensive testing is concentrated toward the end of the development cycle. These temporal patterns underscore a key practical distinction: Agile methodologies distribute test effort throughout the lifecycle, while Traditional approaches compress coverage gains into late project stages, potentially deferring defect discovery and increasing late-stage testing pressure.\u003c/p\u003e\n\u003ch3\u003e2)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;RQ2: Influence of Continuous Integration on Coverage Stability\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;CI Frequency vs. Coverage Volatility\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable III : Correlation Matrix for CI Build Frequency, Coverage Volatility, and Coverage Growth Rate\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eVariable\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCI Build Frequency\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCoverage Volatility\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCoverage Growth Rate\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eCI Build Frequency\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1.000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e-0.48\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e+0.42\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eCoverage Volatility\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e-0.48\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1.000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e-0.36\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eCoverage Growth Rate\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e+0.42\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e-0.36\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1.000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eNote:\u003c/strong\u003e Values are Spearman\u0026rsquo;s \u0026rho;. Correlations in bold are statistically significant (\u003cem\u003ep\u003c/em\u003e \u0026lt; 0.05).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable IV: \u0026nbsp; Spearman\u0026rsquo;s Rank Correlation Results\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eVariable Pair\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e\u0026rho; (Spearman\u0026rsquo;s)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ep-value\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eInterpretation\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eCI Build Frequency \u0026harr; Coverage Volatility\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e-0.48\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.006\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eHigher CI frequency significantly reduces volatility\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eCI Build Frequency \u0026harr; Coverage Growth Rate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e+0.42\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.014\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eHigher CI frequency significantly increases growth\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eCoverage Volatility \u0026harr; Coverage Growth Rate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e-0.36\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.033\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eMore volatility is associated with slower growth\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eNote:\u003c/strong\u003e Values in bold are statistically significant (\u003cem\u003ep\u003c/em\u003e \u0026lt; 0.05).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSubgroup Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable V: \u0026nbsp;Coverage Metrics for Agile vs. Traditional Projects with High CI Adoption\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eMetric\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eAgile (High CI) Mean (%)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eTraditional (High CI) Mean (%)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eDifference\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ep-value\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eInterpretation\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eStatement Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e81.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e75.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+5.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.031\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile significantly higher\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eBranch Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e69.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e63.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+6.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.027\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile significantly higher\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003ePath Coverage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e43.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e39.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+4.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.062\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAgile higher, but difference not statistically significant\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eNote:\u003c/strong\u003e High CI adoption defined as \u0026ge; 10 CI builds per developer-month over the study period\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInterpretation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe observed relationship between CI adoption and coverage stability differs markedly between Agile and Traditional projects. In Agile environments, frequent CI builds are tightly coupled with iterative development practices, where new code is accompanied by corresponding tests, and automated test suites are executed on every integration. This approach ensures that coverage metrics are updated incrementally, detecting regressions early and preventing large, destabilizing fluctuations. The result is \u003cstrong\u003esmoother coverage trends\u003c/strong\u003e, as reflected in the strong negative correlation between CI frequency and coverage volatility (\u0026rho; = \u0026ndash;0.62, \u003cem\u003ep\u003c/em\u003e \u0026lt; 0.01).\u003c/p\u003e\n\u003cp\u003eIn Traditional settings, however, CI adoption often occurs in a more constrained form used primarily during later phases or for integration testing rather than as a continuous feedback mechanism throughout development. This limits CI\u0026rsquo;s stabilizing effect because coverage still depends heavily on bulk test execution in dedicated testing phases. Consequently, even projects with relatively high CI frequency may experience \u003cstrong\u003emixed results\u003c/strong\u003e, with stability gains offset by late-phase surges and test maintenance challenges. These findings suggest that while CI can enhance coverage stability across methodologies, its benefits are maximized when embedded within an iterative development cadence rather than appended to a sequential process.\u003c/p\u003e\n\u003ch3\u003eRQ3: Contextual Factors Affecting Coverage Metrics\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;Project Metadata Influence\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable VI: GLM Regression Coefficients for Coverage Levels\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePredictor\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eStatement Coverage (\u0026beta;)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eBranch Coverage (\u0026beta;)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePath Coverage (\u0026beta;)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eSignificance (p \u0026lt; 0.05)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eProject Size (LOC)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026ndash;0.012\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026ndash;0.009\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026ndash;0.007\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eDomain: Embedded\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026ndash;6.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026ndash;5.11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026ndash;4.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eLanguage: JavaScript\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+3.45\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+2.89\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+1.74\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eCI Frequency\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+0.085\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+0.062\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e+0.041\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eNote:\u003c/strong\u003e \u0026beta; values represent the change in coverage percentage points per unit change in the predictor, holding other variables constant.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eQualitative Observations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePerceptions from Industrial Case Studies\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eInterviews with QA leads and senior engineers from the embedded systems projects in our industrial dataset revealed several recurring factors contributing to consistently lower path coverage. First, \u003cstrong\u003ehardware software integration constraints\u003c/strong\u003e limit the extent to which code paths can be exercised in automated test environments. Many functional scenarios require specialized hardware setups, making full path execution impractical during continuous integration. Second, \u003cstrong\u003estringent timing and resource constraints\u003c/strong\u003e in embedded applications often result in code with deeply nested control flows, where achieving complete path coverage would require infeasible combinations of input states. Third, \u003cstrong\u003eregulatory certification processes\u003c/strong\u003e in domains such as automotive and medical devices emphasize requirements-based testing over exhaustive structural coverage, which shifts engineering resources away from maximizing path coverage metrics. Finally, the \u003cstrong\u003ehigh cost of test infrastructure\u003c/strong\u003e including hardware simulators and real-time debugging tools was cited as a barrier to frequent, comprehensive testing across all execution paths. Collectively, these constraints explain why embedded systems projects, even when following Agile or hybrid methodologies, report systematically lower path coverage than software-only domains such as web or enterprise applications.\u003c/p\u003e\n\u003ch3\u003eMachine Learning Classification of Methodology\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eModel Performance\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAlgorithm: Random Forest Classifier\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo assess whether development methodology (Agile vs. Traditional) could be predicted from coverage-related features, we implemented a \u003cstrong\u003eRandom Forest Classifier (RFC)\u003c/strong\u003e. RFC is an ensemble learning method that constructs multiple decision trees during training and outputs the class selected by the majority of the trees. Its inherent ability to model nonlinear feature interactions, combined with built-in measures of feature importance, makes it well-suited for this problem.\u003c/p\u003e\n\u003cp\u003eThe model was trained using the following input features:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e1.\u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eAverage statement coverage\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.\u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eAverage branch coverage\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.\u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eAverage path coverage\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e4. \u003cstrong\u003eCoverage volatility\u003c/strong\u003e (standard deviation over the lifecycle)\u003c/p\u003e\n\u003cp\u003e5. \u003cstrong\u003eCI build frequency\u003c/strong\u003e (per developer-month)\u003c/p\u003e\n\u003cp\u003e6. \u003cstrong\u003eProject size\u003c/strong\u003e (LOC)\u003c/p\u003e\n\u003cp\u003eWe split the dataset into training (80%) and testing (20%) partitions, stratified by methodology to maintain class balance. \u003cstrong\u003e10 fold cross-validation\u003c/strong\u003e was used to validate performance and reduce overfitting risk. The implementation was carried out in Python using the \u003cem\u003escikit-learn\u003c/em\u003e library, with hyperparameters tuned via grid search to optimize accuracy and F1-score. Key parameters included the number of estimators (\u003cem\u003en_estimators\u003c/em\u003e = 200), maximum tree depth (\u003cem\u003emax_depth\u003c/em\u003e = 10), and minimum samples per split (\u003cem\u003emin_samples_split\u003c/em\u003e = 4).\u003c/p\u003e\n\u003cp\u003eThe RFC achieved \u003cstrong\u003e87.3% accuracy\u003c/strong\u003e, \u003cstrong\u003e85.9% precision\u003c/strong\u003e, \u003cstrong\u003e88.6% recall\u003c/strong\u003e, and an \u003cstrong\u003eAUC of 0.92\u003c/strong\u003e on the test set. Feature importance analysis revealed that \u003cstrong\u003ecoverage volatility\u003c/strong\u003e and \u003cstrong\u003eCI build frequency\u003c/strong\u003e were the most influential predictors, suggesting that temporal patterns in coverage are strong indicators of development methodology.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable VII: Random Forest Classifier Performance Metrics\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"410\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eMetric\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eValue\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e87.3 %\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e85.9 %\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e88.6 %\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eF1-score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e87.2 %\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eNote:\u003c/strong\u003e Metrics are averaged over a 10-fold cross-validation on the test dataset, stratified by development methodology.\u003c/p\u003e\n\u003cp\u003eThe Random Forest Classifier achieved an overall accuracy of \u003cstrong\u003e87%\u003c/strong\u003e in distinguishing between Agile and Traditional projects based solely on coverage patterns and CI metadata. This high level of predictive performance indicates that the selected features particularly coverage volatility, CI build frequency, and average branch coverage capture distinctive signatures of each methodology. In practical terms, this suggests that automated monitoring tools could leverage similar feature sets to infer a team\u0026rsquo;s development approach in real time, enabling adaptive process guidance, benchmarking, or early detection of deviations from intended practices. The model\u0026rsquo;s strong recall (88.6%) further implies that it is particularly effective at correctly identifying Agile projects, while maintaining balanced performance across both classes.\u003c/p\u003e\n\u003cp\u003eThe feature importance analysis reveals that \u003cstrong\u003ecoverage volatility\u003c/strong\u003e is the strongest single predictor of development methodology, with higher volatility strongly associated with Traditional projects. This aligns with earlier findings that Traditional teams tend to concentrate testing activities late in the lifecycle, producing abrupt coverage spikes and fluctuations. \u003cstrong\u003eCI build frequency\u003c/strong\u003e emerges as the second most influential factor, with lower frequencies indicating Traditional practices. Agile projects typically exhibit frequent, smaller integrations, leading to steadier coverage growth and reduced volatility. \u003cstrong\u003eAverage branch coverage\u003c/strong\u003e ranks third, with higher values more characteristic of Agile environments, reflecting the emphasis on thorough unit-level testing and incremental test maintenance. Collectively, these top features provide a strong quantitative basis for distinguishing methodologies and suggest that temporal patterns in coverage metrics, coupled with CI activity levels, are critical indicators of development approach.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eModel Validation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe feature importance analysis reveals that \u003cstrong\u003ecoverage volatility\u003c/strong\u003e is the strongest single predictor of development methodology, with higher volatility strongly associated with Traditional projects. This aligns with earlier findings that Traditional teams tend to concentrate testing activities late in the lifecycle, producing abrupt coverage spikes and fluctuations.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCI build frequency\u003c/strong\u003e emerges as the second most influential factor, with lower frequencies indicating Traditional practices. Agile projects typically exhibit frequent, smaller integrations, leading to steadier coverage growth and reduced volatility. \u003cstrong\u003eAverage branch coverage\u003c/strong\u003e ranks third, with higher values more characteristic of Agile environments, reflecting the emphasis on thorough unit-level testing and incremental test maintenance. Collectively, these top features provide a strong quantitative basis for distinguishing methodologies and suggest that temporal patterns in coverage metrics, coupled with CI activity levels, are critical indicators of development approach.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMisclassification Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAlthough the Random Forest Classifier achieved high overall accuracy, a small subset of projects was misclassified. Examination of these cases revealed two recurring patterns. First, several \u003cstrong\u003ehybrid Agile Waterfall teams\u003c/strong\u003e were labeled incorrectly. These teams followed Agile practices in early development phases (iterative sprints, continuous integration) but switched to a more sequential, Traditional testing phase before release. Such process blending produces mixed metric signatures moderate CI frequency, moderate coverage volatility that fall between the two primary methodology profiles. Second, some long-running Agile projects with \u003cstrong\u003elegacy codebases\u003c/strong\u003e exhibited lower-than-typical coverage and higher volatility due to technical debt, refactoring backlog, and inconsistent test maintenance. These anomalies skewed their feature patterns toward the Traditional cluster. Conversely, a small number of Traditional projects with \u003cstrong\u003ehighly automated CI/CD pipelines\u003c/strong\u003e and dedicated QA investment displayed coverage and stability metrics resembling Agile practices, leading to reverse misclassification. These findings highlight that methodology classification from metric patterns, while powerful, must account for \u003cstrong\u003eorganizational context and hybrid process models\u003c/strong\u003e to avoid oversimplification.\u003c/p\u003e"},{"header":"V. DISCUSSION","content":"\u003ch3\u003ePractical Implications for Test Managers\u003c/h3\u003e\n\u003cp\u003eThe findings of this study carry several actionable lessons for test managers. First, the consistently higher and more stable coverage observed in Agile projects suggests that integrating continuous testing and high CI build frequency can yield measurable benefits for code quality and defect detection readiness. Test managers in Traditional settings should consider introducing earlier test execution cycles, even if full Agile adoption is not feasible, to smooth coverage growth and reduce late-phase quality risks. The feature importance analysis further indicates that monitoring \u003cstrong\u003ecoverage volatility\u003c/strong\u003e and \u003cstrong\u003ebranch coverage\u003c/strong\u003e can serve as effective, low-cost indicators of testing health, allowing managers to proactively address potential quality gaps.\u003c/p\u003e\n\u003ch3\u003e1)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Interpretation of Unexpected Findings\u003c/h3\u003e\n\u003cp\u003eOne surprising observation was that some high-CI Traditional projects still exhibited volatile coverage patterns. Interviews revealed that in these cases, CI was primarily used for integration testing rather than as a continuous feedback mechanism during implementation. This suggests that \u003cstrong\u003eCI adoption alone is insufficient\u003c/strong\u003e its placement and integration into the development workflow determine its stabilizing effect. Additionally, certain Agile projects with large legacy codebases displayed lower path coverage than expected, indicating that process advantages can be eroded by technical debt and incomplete test refactoring.\u003c/p\u003e\n\u003ch3\u003e2)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Trade-offs Between Agile and Traditional Coverage\u003c/h3\u003e\n\u003cp\u003eWhile Agile methods clearly promote earlier and steadier coverage gains, they also involve trade-offs. Frequent refactoring and evolving requirements can lead to short-term dips in coverage, which require disciplined test maintenance to avoid long-term erosion. Traditional models, although slower to accumulate coverage, can achieve high end-phase coverage levels, especially in safety-critical domains where structured, requirements-based testing dominates. The key difference is that Agile spreads testing effort over time, reducing the risk of late-stage defect discovery, whereas Traditional approaches consolidate testing into a single, intensive phase, which may be resource-efficient but also risk-prone.\u003c/p\u003e\n\u003ch3\u003e3)\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp; Links to Related Work Trends\u003c/h3\u003e\n\u003cp\u003eThese findings align with recent empirical studies in IEEE Software [1] and IEEE Access [2], which report that Agile projects tend to maintain higher incremental branch coverage and leverage CI for continuous quality monitoring. However, our results extend this work by quantifying \u003cstrong\u003ecoverage volatility\u003c/strong\u003e as a discriminating factor between methodologies and demonstrating its predictive power through machine learning classification. The mixed results of CI adoption in Traditional settings also resonate with earlier work by Herzig and Nagappan [3], who noted that CI benefits are contingent on integration frequency and scope. By situating our results within these broader research trends, we highlight that while coverage metrics remain central to quality assurance in both Agile and Traditional contexts, their interpretation must be methodology-aware and context-sensitive.\u003c/p\u003e"},{"header":"VI. CONCLUSION AND FUTURE WORK","content":"\u003cp\u003eThis study provided a comprehensive, metric-level comparison of test coverage behavior in Agile and Traditional software development projects using a combination of industrial datasets, public repository mining, and controlled experiments. By integrating statistical analysis with machine learning classification, we identified key differentiators such as coverage volatility, CI build frequency, and branch coverage that not only distinguish methodologies but also predict development approach with high accuracy.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMain Contributions:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e1.\u0026nbsp; \u0026nbsp; \u0026nbsp;Empirical evidence that Agile projects generally achieve higher and more stable statement and branch coverage than Traditional projects, with path coverage differences being smaller but still notable.\u003c/p\u003e\n\u003cp\u003e2.\u0026nbsp; \u0026nbsp; \u0026nbsp;Identification of coverage volatility as a strong and interpretable indicator of methodology, supported by both statistical and predictive modeling.\u003c/p\u003e\n\u003cp\u003e3.\u0026nbsp; \u0026nbsp; \u0026nbsp;Contextual analysis showing how factors such as project domain, programming language, and CI usage moderate coverage outcomes.\u003c/p\u003e\n\u003cp\u003e4.\u0026nbsp; \u0026nbsp; \u0026nbsp;A practical framework for test managers to monitor coverage health using lightweight, actionable metrics.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eActionable Recommendations for Industry:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Incorporate frequent CI builds throughout the lifecycle, not just in integration phases, to stabilize coverage trends.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Track coverage volatility alongside absolute coverage values to detect process instability.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;In domains with inherently lower path coverage (embedded systems), focus on requirements-based coverage metrics while optimizing feasible structural coverage.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;For hybrid or transitioning teams, use predictive monitoring to detect process drift and proactively adjust testing strategies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLimitations:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;While diverse, the dataset may not fully represent extremely large-scale systems (\u0026gt;10M LOC) or heavily regulated environments beyond those sampled.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Coverage was limited to statement, branch, and path metrics; stronger criteria (e.g., mutation coverage) were not analyzed.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Some industrial data was aggregated to respect confidentiality, potentially limiting granularity for certain analyses.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFuture Work:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Extend the study to include mutation coverage and fault detection effectiveness to better capture test suite quality.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Conduct longitudinal tracking of hybrid methodology teams to model process transitions and their impact on coverage stability.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Explore integration of real-time predictive monitoring in CI/CD pipelines to provide automated alerts for coverage anomalies.\u003c/p\u003e\n\u003cp\u003e·\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Investigate the relationship between coverage volatility and other process quality indicators, such as defect resolution time or release frequency.\u003c/p\u003e\n\u003cp\u003eBy bridging empirical measurement, contextual analysis, and predictive modeling, this work offers both a methodological contribution to the research community and a set of concrete, data-driven tools for practitioners seeking to optimize testing strategies under different development methodologies.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eN. Humbatova, G. Jahangirova, G. Bavota, V. Riccio, A. Stocco, and P. Tonella, \u0026ldquo;Taxonomy of real faults in deep learning systems,\u0026rdquo; in \u003cem\u003eProc. 42nd Int. Conf. Software Eng.\u003c/em\u003e (ICSE \u0026rsquo;20), ACM, 2020, pp. 12\u0026ndash;22.\u003c/li\u003e\n\u003cli\u003eF. Madeyski and M. Kawalerowicz, \u0026ldquo;Continuous test-driven development \u0026ndash; a novel Agile software development practice and supporting tool,\u0026rdquo; in \u003cem\u003eProc. 8th Int. Conf. Eval. Novel Approaches Software Eng.\u003c/em\u003e (ENASE), 2020.\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;On testing machine learning programs,\u0026rdquo; \u003cem\u003eJ. Systems and Software\u003c/em\u003e, vol. 164, Art. no. 110542, Jun. 2020.\u003c/li\u003e\n\u003cli\u003eK. Tantithamthavorn, J. Jiarpakdee, and J. Grundy, \u0026ldquo;Explainable AI for software engineering,\u0026rdquo; \u003cem\u003earXiv preprint arXiv:2012.01614\u003c/em\u003e, 2020.\u003c/li\u003e\n\u003cli\u003eS. Sharma, M. Kechagia, S. Georgiou, R. Tiwari, I. Vats, H. Moazen, and F. Sarro, \u0026ldquo;A survey on machine learning techniques for source code analysis,\u0026rdquo; \u003cem\u003earXiv preprint arXiv:2110.09610\u003c/em\u003e, 2021.\u003c/li\u003e\n\u003cli\u003eA. Serban, K. van der Blom, H. Hoos, and J. Visser, \u0026ldquo;Adoption and effects of software engineering best practices in machine learning,\u0026rdquo; \u003cem\u003earXiv preprint arXiv:2007.14130\u003c/em\u003e, 2022.\u003c/li\u003e\n\u003cli\u003eP. Shafiq, A. Mashkoor, C. Mayr-Dorn, and A. Egyed, \u0026ldquo;Machine learning for software engineering: a systematic mapping,\u0026rdquo; \u003cem\u003earXiv preprint arXiv:2005.13299\u003c/em\u003e, 2020.\u003c/li\u003e\n\u003cli\u003e\u003cem\u003eContinuous integration\u003c/em\u003e, Wikipedia, Jul. 2025.\u003c/li\u003e\n\u003cli\u003e\u003cem\u003eTestOps\u003c/em\u003e, Wikipedia, Jun. 2025.\u003c/li\u003e\n\u003cli\u003e\u003cem\u003eContinuous testing\u003c/em\u003e, Wikipedia, Jul. 2025.\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;Evolution and implementation of continuous testing in enterprise software engineering,\u0026rdquo; \u003cem\u003eResearchGate Preprint\u003c/em\u003e, 2023.\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;A systematic review of machine learning methods in software testing,\u0026rdquo; \u003cem\u003eApplied Soft Computing\u003c/em\u003e, vol. 162, Art. no. 111805, Sep. 2024.\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;How test coverage changes in quality engineering \u0026ndash; GenQE-AI based quality engineering,\u0026rdquo; GenQE.ai blog, May 2025.\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;Employing machine learning techniques to assess requirement change volatility,\u0026rdquo; \u003cem\u003eResearch in Engineering Design\u003c/em\u003e, 2020.\u003c/li\u003e\n\u003cli\u003eC. L\u0026oacute;pez-Mart\u0026iacute;n, Y. Villuendas-Rey, M. Azzeh, A.B. Nassif, and S. Banitaan, \u0026ldquo;Transformed k-nearest neighborhood output distance minimization for predicting the defect density of software projects,\u0026rdquo; \u003cem\u003eJ. Systems and Software\u003c/em\u003e, vol. 167, Art. no. 110592, 2020.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Software testing, Agile software development, Waterfall model, Test coverage metrics, Continuous integration, Software quality assurance","lastPublishedDoi":"10.21203/rs.3.rs-7470780/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7470780/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTest coverage metrics are broadly used in software engineering to assess the thoroughness of automated testing, up till now little empirical work has directly compared their behavior across Agile and Traditional development methodologies. This research conducts a mixed-method analysis combining industrial case studies, mining of public repositories, controlled experiments, and machine learning modeling to study statement, branch, and path coverage patterns in 30 projects spanning multiple domains and languages. Statistical analysis has x-rayed that Agile projects achieved significantly higher and more stable statement and branch coverage than Traditional projects, with smaller but consistent differences in path coverage. Continuous Integration (CI) occurrence was robustly connected with coverage stability in Agile settings but yields mixed benefits in Traditional contexts. Coverage volatility appeared as a robust predictor of methodology, enabling a Random Forest classifier to differentiate Agile from Traditional projects with 87% accuracy. Contextual features such as project size, domain, and language also moderate coverage results, with embedded systems exhibiting systematically lower path coverage due to hardware and regulatory constraints. These investigation revealed actionable guidance for test managers, including the use of coverage volatility and CI frequency as process health indicators, and highlight the trade-offs between earlier, incremental coverage growth in Agile and concentrated, late-phase coverage in Traditional methods.\u003c/p\u003e","manuscriptTitle":"Comparative Analysis of Test Coverage Metrics in Agile vs. Traditional Software Development","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-01 10:30:30","doi":"10.21203/rs.3.rs-7470780/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6254ff2e-f4df-4f5c-85fe-7fae86909cd0","owner":[],"postedDate":"September 1st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":53779267,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-09-01T10:30:30+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-01 10:30:30","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7470780","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7470780","identity":"rs-7470780","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}