Systematic Literature Review on Observable Indicators of Scrum Ceremony Adherence

Scrum has become one of the most widely adopted frameworks in agile project management, with recent surveys indicating that around 87% of agile teams employ Scrum practice. A defining feature of Scrum is its set of prescribed ceremonies such as sprint planning, daily stand-up meetings, sprint reviews, and sprint retrospectives designed to foster regular communication, stakeholder feedback, and continuous improvement. According to Scrum’s guidelines, each of these ceremonies is essential for the framework’s success, and the method is intended to be implemented in its entirety; however, in practice many teams modify or omit certain Scrum ceremonies (often termed “Scrum But”), potentially limiting the benefits realized from the Scrum framework. This variability in ceremony adherence underscores the importance of understanding how deviations from Scrum’s prescribed events affect team performance and project outcomes. Accordingly, this systematic literature review examines the existing body of research on Scrum ceremony adherence in agile environments, aiming to synthesize current knowledge on the motivations for adherence or deviation and the reported impacts of these practices on agile project success. To achieve these objectives, a rigorous SLR methodology was followed, as detailed in the following Research Methodology section. This study employed a Systematic Literature Review (SLR) approach to identify and organize observable indicators used to assess adherence to Scrum ceremonies. The objective was to move beyond theoretical assumptions by synthesizing evidence-based indicators that reflect how Scrum practices are applied in real project environments. The review was conducted following Kitchenham’s (2007) SLR guidelines and reported in accordance with the PRISMA framework, ensuring methodological transparency, reproducibility, and rigor. Systematic Literature Review on Observable Indicators of Scrum Ceremony Adherence Agha Moiz Fast university Islamabad Scrum has become one of the most widely adopted frameworks in agile project management, with recent surveys indicating that around 87% of agile teams employ Scrum practice. A defining feature of Scrum is its set of prescribed ceremonies such as sprint planning, daily stand-up meetings, sprint reviews, and sprint retrospectives designed to foster regular communication, stakeholder feedback, and continuous improvement. According to Scrum’s guidelines, each of these ceremonies is essential for the framework’s success, and the method is intended to be implemented in its entirety; however, in practice many teams modify or omit certain Scrum ceremonies (often termed “Scrum But”), potentially limiting the benefits realized from the Scrum framework. This variability in ceremony adherence underscores the importance of understanding how deviations from Scrum’s prescribed events affect team performance and project outcomes. Accordingly, this systematic literature review examines the existing body of research on Scrum ceremony adherence in agile environments, aiming to synthesize current knowledge on the motivations for adherence or deviation and the reported impacts of these practices on agile project success. To achieve these objectives, a rigorous SLR methodology was followed, as detailed in the following Research Methodology section. This study employed a Systematic Literature Review (SLR) approach to identify and organize observable indicators used to assess adherence to Scrum ceremonies. The objective was to move beyond theoretical assumptions by synthesizing evidence-based indicators that reflect how Scrum practices are applied in real project environments. The review was conducted following Kitchenham’s (2007) SLR guidelines and reported in accordance with the PRISMA framework, ensuring methodological transparency, reproducibility, and rigor. Research Methodology This study employed a Systematic Literature Review (SLR) approach to identify and organize observable indicators of adherence to Scrum ceremonies. The review followed the established guidelines of Kitchenham (2007) and was reported in accordance with the PRISMA framework to ensure methodological transparency, reproducibility, and rigor. A formal review protocol was defined a priori, specifying the research question, scope, search strategy, and inclusion criteria. The purpose of the SLR was to synthesize evidence-based indicators reflecting how Scrum practices are implemented in real project environments, thereby laying the groundwork for developing a Ceremony Adherence & Quality Index (CAQI) in subsequent research. The scope of the review was limited to the five core Scrum ceremonies Sprint Planning, Backlog Refinement, Daily Scrum (Daily Stand-ups), Sprint Review, and Sprint Retrospective as practiced in industrial or educational contexts. Other Agile frameworks or practices were considered out of scope unless Scrum ceremonies were explicitly analyzed, and quality assurance (QA) outcomes were excluded at this stage (to be addressed in later phases of the thesis). The review was guided by the following primary research question: RQ1: What observable indicators can be used to assess adherence to Scrum ceremonies? To precisely define the focus of the SLR, the PICOC framework (Population, Intervention/Indicator, Comparison, Outcome, Context) was applied as follows: • Population: Scrum teams (in industrial or academic settings). • Intervention/Indicator: Execution and adherence of Scrum ceremonies (e.g., how ceremonies are carried out). • Comparison: High vs. low adherence (as observed across different teams or studies). • Outcome: Identified indicators, rubrics, or frameworks for assessing ceremony adherence. • Context: Empirical Scrum environments (industrial or educational), rather than purely theoretical discussions. Clear inclusion and exclusion criteria were defined to ensure only relevant primary studies were selected: • Inclusion Criteria: • Peer-reviewed journal papers, conference proceedings, or book chapters. • Published between 2008 and 2025 (inclusive). • Explicit focus on Scrum ceremonies (at least one of the five core ceremonies). • Provides measurable indicators, metrics, or rubrics related to ceremony adherence. • Written in English, with full-text available. • Exclusion Criteria: • Studies that do not focus on Scrum ceremonies (e.g., general Agile studies without ceremony-specific analysis). • Studies centering on technical practices only (e.g., continuous integration, pair programming) with no linkage to ceremonies. • Non-peer-reviewed literature such as opinion pieces, blogs, or magazine articles. • Secondary studies (systematic literature reviews or mapping studies) – only primary empirical studies were retained. • Non-English or inaccessible full texts. Notably, any SLRs or other secondary reviews discovered in the search were excluded so that the final pool comprised only primary research evidence on Scrum adherence indicators. The literature search was conducted across seven major scholarly databases: IEEE Xplore, ACM Digital Library, SpringerLink, ScienceDirect, Scopus, Web of Science, and Google Scholar . Google Scholar was used in particular for snowballing (forward and backward reference checking) and to capture relevant open-access publications. The search was conducted using a comprehensive query string that combined keywords related to Scrum ceremonies with terms indicating adherence or measurement. The final search string was: (Scrum AND (ceremony OR ceremonies OR ”sprint planning” OR ”backlog refinement” OR ”daily scrum” OR ”daily stand-up” OR ”sprint review” OR retrospective)) AND (adherence OR compliance OR execution OR assessment OR indicator OR metric* OR measurement OR rubric OR framework OR maturity OR evaluation)*. This query ensured that the search captured studies focusing on Scrum ceremonies and any form of assessing their execution or compliance. The search covered the period 2010–2025 and was not restricted to any specific discipline beyond computing and software engineering, given the context of Scrum in Agile software development. The study selection process was carried out in multiple stages and documented using a PRISMA flow approach. In the identification stage, the database searches yielded approximately 1,400 records. After removing duplicates (e.g., papers appearing in multiple databases), 1,180 unique records remained. These records then underwent a title and abstract screening against the inclusion/exclusion criteria, which filtered out clearly irrelevant studies (for example, studies on Agile in general without ceremony-specific content). This screening reduced the pool to about 100 papers eligible for full-text review. Prior to the full screening, a small pilot screening of roughly 10% of the records was conducted to calibrate and refine the inclusion criteria, ensuring consistency in the selection process. In the eligibility stage, the full texts of these 100 candidate papers were examined in detail. An additional 70 studies were excluded upon full-text review due to reasons such as not actually providing measurable indicators, having only a minimal or tangential focus on Scrum ceremonies, or being secondary reviews rather than primary studies. Secondary SLR or mapping studies were explicitly excluded at this stage to maintain a set of original empirical sources. Ultimately, 30 primary studies met all criteria and were included for data extraction and synthesis. The PRISMA flow diagram (Figure 1) illustrates this selection process, showing the number of studies identified, screened, excluded (with reasons), and the final number of studies included in the review. (Figure 1. PRISMA flow diagram of the study selection process, from 1,400 initial records to 30 included studies.) Each of the 30 included studies was subjected to a quality appraisal to assess the rigor and reliability of its evidence. This appraisal used a scoring rubric based on five criteria : (1) Context clarity (how well the study described its setting and subjects, such as team size or project context) (2) Indicator validity (whether the paper validated or provided rationale for the indicators it identified) (3) Data completeness (the extent and completeness of data or results presented) (4) Research design rigor (the soundness of the study’s methodology or empirical design) (5) Bias mitigation (steps taken to reduce researcher bias or limitations). Each criterion was rated 0, 1, or 2 points, for a maximum quality score of 10 per study. Studies scoring 6 or higher (≥ 6/10) were considered high quality evidence and their findings given more weight during synthesis, whereas studies scoring below 6 were still included (to be comprehensive) but their results were interpreted with caution in the analysis. For each included study, relevant data were extracted systematically. This included bibliographic information (author, year, venue), the study context (industry or academic setting, domain of the project), the specific Scrum ceremony focus (which ceremony or ceremonies were being examined), the research method used (e.g., case study, experiment, survey), any frameworks or models introduced (such as a rubric, checklist, or maturity model for Scrum adherence), the observable indicators of ceremony adherence identified, the key findings or conclusions, and any noted limitations of the study. All extracted indicators were carefully reviewed to ensure they were indeed observable and measurable evidence of Scrum practice (for example, an indicator needed to be something that could be seen or quantified in practice, not just a vague recommendation). The synthesis process then focused on the indicators reported across studies. Indicators identified in each paper were catalogued and grouped by the Scrum ceremony they pertain to: Sprint Planning, Daily Scrum, Sprint Review, Sprint Retrospective, as well as supporting or related activities like Backlog Refinement, Release Planning, or cross-cutting Quality Assurance practices. During this phase, similar or duplicate indicators from different sources were merged and standardized under a common terminology to avoid redundancy. For example, if one study referred to an indicator as ”Definition of Done compliance” and another study used ”DoD criteria met,” these were unified under the indicator Definition of Done . Likewise, terms like ”Commitment Ratio” and ”Say-Do Ratio” were standardized as Commitment Reliability since they both describe the consistency between planned work and completed work in a sprint. After consolidating synonyms and overlapping concepts, the review identified a total of 58 unique indicators that serve as evidence of adherence to one or more Scrum ceremonies. These 58 indicators spanned all the core ceremonies and some cross-cutting areas. In particular, Sprint Planning had the largest number of unique indicators (19 indicators related to planning activities), followed by Sprint Review (17 indicators) and Sprint Retrospective (16 indicators), Daily Scrum (15 indicators), and a smaller number for Backlog Refinement (3) and Release Planning (1). An additional 11 indicators were applicable in a cross-cutting sense (e.g. related to overall team behavior or quality assurance across all ceremonies). Figure 2 illustrates the distribution of these indicators by ceremony, highlighting that Sprint Planning contributes the highest count of distinct adherence measures, while some areas like Backlog Refinement and Release Planning are comparatively underrepresented. While this comprehensive list of 58 indicators provides a broad view, not all indicators are equally prominent or practical for assessment. To distill the findings to the most important measures, the indicators were further prioritized based on their frequency of occurrence across the literature (how many studies reported or utilized them) and their relevance to real-world Scrum practice . Through this synthesis and filtering process, the list was refined to a curated set of 20 high-relevance indicators . These represent the indicators that were most consistently supported by empirical evidence and frequently mentioned across multiple studies, and which practitioners would likely find most meaningful for evaluating Scrum ceremony adherence. The 20 key indicators (with their core focus or rationale) are listed below: 1. Velocity – Reflects the team’s delivery capacity and sprint predictability (amount of work completed per sprint relative to plan). 2. Commitment Reliability – Consistency between what is planned in Sprint Planning and what is actually delivered by the end of the sprint. 3. Definition of Ready – The degree to which backlog items meet clear readiness criteria (fully detailed, estimated, and testable) before being accepted into a sprint. 4. Definition of Done – Verification that all acceptance criteria and quality checks are met for completed items (ensuring work meets a standard definition of completion). 5. Burndown Trend – The pattern of the sprint burndown chart over time, indicating progress against the sprint timeline and workload (steady burn-down vs. spikes). 6. Sprint Goal Achievement Rate – The extent to which the team achieves the sprint goals or objectives they set out at the beginning of the sprint. 7. Stakeholder Participation – The level of engagement and attendance of external stakeholders during Sprint Review and other feedback ceremonies. 8. Impediments Logged & Resolved – The frequency of impediments or blockers raised by the team and the speed or effectiveness with which these issues are resolved. 9. Team Communication Quality – The effectiveness and clarity of information sharing among team members (especially during Daily Scrums), reflecting transparency and collaboration. 10. Attendance Consistency – Regularity of team members’ attendance and active participation in Scrum ceremonies (how often all key members are present and involved). 11. Improvement Action Completion (Improvement KPIs) – The number of actionable improvement items (from Sprint Retrospectives) and the rate at which those action items are completed by the next sprint. 12. Team Satisfaction Index – A measure of team morale or satisfaction (often captured via survey or feedback in Retrospectives) indicating the team’s overall health and sentiment across sprints. 13. Collaboration Effectiveness – The quality of cooperation within the team and with related roles (e.g., developers, testers, product owner), including knowledge sharing and mutual support. 14. Transparency Level – The visibility of progress, obstacles, and decisions to all team members and stakeholders (e.g., use of visible task boards, open communication of issues). 15. Defect Density – The number of defects identified per unit of product output (story, feature, or sprint), linking process adherence to product quality outcomes. 16. Defect Leakage – The occurrence of defects that escape to production or late-stage testing, indicating effectiveness of the team’s quality practices during the sprint. 17. Test Automation Coverage – The ratio or percentage of test cases that are automated (part of overall test coverage), reflecting the maturity of QA practices in the process. 18. Backlog Health – The overall state of the product backlog, including its size appropriateness, prioritization clarity, and the currency or refinement level of items (ensuring a ready pipeline of work). 19. Release Readiness Index – A measure of how prepared the team is for upcoming release milestones, considering factors like completed integration testing, documentation, and backlog items aligned with the release. 20. Team Autonomy – The team’s ability to self-organize, make decisions, and resolve issues independently without external micromanagement (an indicator of Agile maturity and empowerment). These twenty indicators collectively capture a balanced view of process efficiency, team dynamics, and product quality in the context of Scrum. They are strongly grounded in the literature and appear across multiple studies, suggesting they are robust measures of how well Scrum ceremonies are being executed. Together, these key indicators form an evidence-driven foundation for further empirical validation and will inform the construction of the envisioned Ceremony Adherence and Quality Index (CAQI) in later phases of this research. Visual Analysis and Diagrams To make the data easier to interpret, I visualized my results in five figures that show the trend, frequency, and relationship of indicators across ceremonies. These figures will be included in my Word document after this section. Figure 1: Publication trend by year (2008 to 2024) The figure shows a steady increase in studies on Scrum adherence indicators over time, with the highest number of publications appearing in 2023 and 2024. Figure 2: Number of unique observable indicators per ceremony The figure highlights that Sprint Planning has the highest number of unique adherence indicators, reflecting its central role in defining and managing Scrum execution. Cross-cutting indicators apply across ceremonies and represent shared practices like transparency, team communication, and definition of done. Figure 3: Distribution of Studies by Source Database Most studies were retrieved from Google Scholar, IEEE Xplore, and SpringerLink, which collectively formed the majority of the 1,400 records screened during the systematic literature review. Figure 4: Figure 4: Most Frequent Indicators Identified Across Studies Velocity and Definition of Done emerged as the most widely used adherence indicators across studies. They consistently appeared as part of both planning and review ceremonies, followed by indicators focused on team well-being, backlog health, and retrospective improvements. Figure 5. Overall Distribution of Indicator Categories. Across all reviewed studies, process-oriented indicators remain dominant but people-centric and product-quality measures now account for nearly half of all indicators identified. • Process indicators: Measure sprint efficiency and adherence (e.g., velocity, cycle time, backlog health). • People indicators: Reflect team collaboration, morale, and communication quality. • Product indicators: Capture output quality and reliability (e.g., defect density, test coverage, delivery rate). Figure 6. Indicator Ceremony Linkage (Condensed View). Each cluster groups the key indicators most frequently associated with a specific Scrum ceremony. The chart shows how Sprint Planning stresses commitment and readiness, Daily Scrum emphasizes communication and impediment handling, Sprint Review focuses on product quality and stakeholder participation, Retrospective captures improvement and team sentiment, Backlog Refinement ensures prioritization health, and QA activities monitor overall technical quality. Together, these figures summarize how research around Scrum adherence has evolved, where measurement focus lies, and which ceremonies are underrepresented (such as backlog refinement and release planning). The synthesized findings from the SLR reveal several important insights about Scrum adherence in practice. First, Sprint Planning was associated with the greatest number of unique indicators among the ceremonies. This suggests that planning is a critical phase influencing overall Scrum success – it directly impacts sprint scope, team commitment, and predictability, which in turn generate many measurable facets (e.g., commitment reliability, planning accuracy, etc.). Daily Scrums and Sprint Retrospectives, while having slightly fewer distinct metrics, consistently emphasized communication, transparency, and continuous improvement . For instance, regular impediment logging and resolution, open information sharing, and follow-through on retrospective action items were common themes, highlighting the importance of team interaction and learning. Sprint Reviews tended to focus on validation and feedback loops – indicators here centered on product quality (such as meeting the Definition of Done, defect counts) and stakeholder engagement (attendance and feedback integration), reflecting the review’s role in ensuring the increment meets expectations. Additionally, several cross-cutting indicators (like test automation coverage and defect rates) connect the proper execution of Scrum ceremonies to tangible quality outcomes. This linkage suggests that teams with higher ceremony adherence (e.g., diligently following Scrum rituals and norms) often correlate with better product quality and technical performance. In summary, the research indicates that strengthening the fidelity of Scrum ceremony practices can have broad positive effects, from improved team dynamics to higher-quality deliverables. With an evidence-based set of indicators now established from the literature, the next phase of this research will focus on empirical validation of these indicators in an industry context. In this upcoming phase, a survey instrument will be developed to gather expert feedback on the final list of 20 indicators . Each indicator will be formulated as a survey item, and Agile practitioners (such as Scrum Masters, Quality Assurance leads, Product Owners, and Agile coaches) will be asked to rate each item on a Likert scale (for example, from “Strongly Disagree (1)” to “Strongly Agree (5)” in response to statements about the indicator’s relevance or observability). The goal of this survey is to confirm whether the indicators identified from the literature are perceived as relevant, clear, and practically observable in real-world Scrum teams. The questionnaire will be distributed to professionals with substantial Scrum experience. Their collective responses will provide insights into which indicators are considered most crucial and whether any important practical indicators were overlooked. After data collection, the consistency and reliability of the survey responses will be evaluated using Cronbach’s alpha, a statistical measure of internal consistency. A Cronbach’s alpha coefficient of 0.70 or higher will be regarded as an acceptable level of reliability for the set of survey items. If the analysis yields a reliability below this threshold, it will indicate that certain indicators may not align well with the others, in which case those indicators will be reconsidered or reworded for clarity. Based on the survey feedback and the reliability analysis, the list of indicators will be refined. The outcome will be a finalized set of industry-validated indicators of Scrum ceremony adherence. These validated indicators will then serve as the foundation for developing the Ceremony Adherence and Quality Index (CAQI) in the subsequent research phase. The CAQI is envisioned as a comprehensive framework or tool for quantitatively assessing how well Scrum ceremonies are executed within teams, ultimately helping organizations to diagnose areas of strength or needed improvement in their Agile practices. By incorporating practitioner validation, this next step will strengthen the rigor of the research findings, ensuring that the proposed indicators are not only grounded in literature but also endorsed by those with real-world Scrum experience. This approach will enhance the credibility and applicability of the final CAQI framework, bridging the gap between academic research and practical Agile team assessment. In conclusion, this systematic literature review has produced an evidence-based catalogue of indicators for measuring adherence to Scrum ceremonies in a structured, quantifiable manner. By translating the abstract practices of Scrum into concrete, observable metrics, the study supports both rigorous research and practical application in Agile teams. The work shifts the focus beyond merely asking whether teams are following Scrum, to evaluating how well they adhere to Scrum practices and providing objective data to support that evaluation. In doing so, it addresses a critical need in both academia and industry: a clearer understanding of Scrum implementation quality. The set of identified indicators especially the curated high-relevance subset offers a foundation for further empirical validation and for the development of the CAQI measurement framework. Ultimately, this contributes to the broader goal of improving Agile process quality by enabling teams and organizations to monitor and enhance their Scrum practice adherence with empirically grounded metrics. Information & Authors Information Version history Copyright This work is licensed under a Non Exclusive No Reuse License.

Authors Metrics & Citations Metrics Article Usage 256views 146downloads Citations Download citation Agha Moiz. Systematic Literature Review on Observable Indicators of Scrum Ceremony Adherence. Authorea. 30 October 2025. DOI: https://doi.org/10.22541/au.176183673.34271999/v1 DOI: https://doi.org/10.22541/au.176183673.34271999/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu.