Inference for the treatment effect in staircase designs with continuous outcomes: a simulation study

Inference for the treatment effect in staircase designs with continuous outcomes: a simulation study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Inference for the treatment effect in staircase designs with continuous outcomes: a simulation study Ehsan Rezaei-Darzi, Kelsey L Grantham, Andrew B Forbes, Jessica Kasza This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5657090/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 10 May, 2025 Read the published version in BMC Medical Research Methodology → Version 1 posted 6 You are reading this latest preprint version Abstract Background Staircase designs are incomplete stepped wedge designs that, unlike standard stepped wedge designs, require clusters to contribute data for only a limited number of trial periods. Previous work has provided formulae based on asymptotic results for the calculation of the power of staircase designs to detect treatment effects of interest. Methods We conduct a simulation study to assess the finite sample performance of these formulae, and the impact of misspecifying the correlation structure when analysing data from staircase designs on inference for the treatment effect, under a range of realistic trial settings. This study focuses on basic staircase designs with one control period followed by one intervention period in each sequence. We simulate staircase trial datasets with continuous outcomes and a repeated cross-sectional measurement scheme under exchangeable and block-exchangeable intracluster correlation structures, and then fit linear mixed models with linear and categorical time period effects. For settings with a small number of clusters, Kenward-Roger and Satterthwaite small-sample corrections are applied. Comparisons are made between nominal and observed Type I error rates, and theoretically-derived study power and empirical power. The impact on inference for the treatment effect when misspecifying the intracluster correlation structure is assessed through considering performance metrics including bias and 95% confidence interval coverage. Results Data analysis assuming an exchangeable correlation structure and application of the Satterthwaite correction controls Type I error well when the correlation structure is correctly specified, and there are a sufficient number of clusters. For the true block-exchangeable model, when fitting the correct model with the Satterthwaite correction, the observed Type I error (empirical power) can be higher (lower) than the nominal (i.e., theoretical) value when there is only 1 cluster per sequence, but otherwise, it aligns well with the nominal (theoretical) value. Misspecification of the correlation structure (fitting an exchangeable model when the true structure is block-exchangeable) can lead to inflated Type I error and poor confidence interval coverage. Conclusions Staircase designs with one cluster per sequence should be used with caution. Additionally, using a correlation structure that allows for decay is preferable for making valid inferences for the estimation of the treatment effect. Cluster randomised trials Incomplete design Intracluster correlation Stepped wedge Full Text Additional Declarations No competing interests reported. Supplementary Files Additionalfile1.pdf Cite Share Download PDF Status: Published Journal Publication published 10 May, 2025 Read the published version in BMC Medical Research Methodology → Version 1 posted Editorial decision: Accepted 15 Apr, 2025 Reviews received at journal 02 Apr, 2025 Reviewers agreed at journal 26 Mar, 2025 Reviewers invited by journal 24 Mar, 2025 Submission checks completed at journal 24 Mar, 2025 First submitted to journal 21 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5657090","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":433226723,"identity":"5a0e5498-4fc8-4e75-a9ef-bf4a306313fb","order_by":0,"name":"Ehsan Rezaei-Darzi","email":"","orcid":"","institution":"School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia","correspondingAuthor":false,"prefix":"","firstName":"Ehsan","middleName":"","lastName":"Rezaei-Darzi","suffix":""},{"id":433226724,"identity":"b0cb85f4-1145-4024-950c-5a24e236e908","order_by":1,"name":"Kelsey L Grantham","email":"","orcid":"","institution":"School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia","correspondingAuthor":false,"prefix":"","firstName":"Kelsey","middleName":"L","lastName":"Grantham","suffix":""},{"id":433226725,"identity":"024514bf-8933-4b2b-8da9-e61522ef8d2d","order_by":2,"name":"Andrew B Forbes","email":"","orcid":"","institution":"School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia","correspondingAuthor":false,"prefix":"","firstName":"Andrew","middleName":"B","lastName":"Forbes","suffix":""},{"id":433226726,"identity":"65396b2f-90f8-4062-977d-8f89b387c11f","order_by":3,"name":"Jessica Kasza","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABBklEQVRIiWNgGAWjYBACxgYgkQDEEmBuBZhkA2JmYrWcIUILHIC1MLYRoYW5vf3xhwc1dgyS7ccff/g5b5ucOfvxZw8YKqwTG3A5rOeMmUTCsWQGaZ4cA8PebbeNLXtyzA0YzqTj1jIjh40hsYGZQY4hhyGBd9vtxA0HctgkGNsO49Yy//njD4kN9Qxy/M8fHPw7B6jl/PNnEoz/8GiZwWAgkdhwmEFaIsGwmbcBqOVGgpkEYwMeLT05IL8c55Gc8caYWebYbWODG29AIunGuLQYAgPq44+aajmJ8+mPP76puS1ncD79mcSHGmtZnFqgEjyowgk4lIOAPB65UTAKRsEoGAUQAAD7iVwYpmMPCAAAAABJRU5ErkJggg==","orcid":"","institution":"School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia","correspondingAuthor":true,"prefix":"","firstName":"Jessica","middleName":"","lastName":"Kasza","suffix":""}],"badges":[],"createdAt":"2024-12-17 00:23:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5657090/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5657090/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12874-025-02567-5","type":"published","date":"2025-05-10T15:57:37+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":82538169,"identity":"e5272c3b-c466-413a-8419-920222c268b5","added_by":"auto","created_at":"2025-05-12 16:10:38","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1485880,"visible":true,"origin":"","legend":"","description":"","filename":"Revised.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5657090/v1_covered_b3e2cc14-5c68-462a-912e-152ffc8013b8.pdf"},{"id":79179606,"identity":"bea6dbc8-2245-49ee-bb80-f7d79df7771a","added_by":"auto","created_at":"2025-03-25 10:16:01","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":3946619,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5657090/v1/4bdef4bcd8744ef4278ee4e1.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Inference for the treatment effect in staircase designs with continuous outcomes: a simulation study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Cluster randomised trials, Incomplete design, Intracluster correlation, Stepped wedge","lastPublishedDoi":"10.21203/rs.3.rs-5657090/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5657090/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eStaircase designs are incomplete stepped wedge designs that, unlike standard stepped wedge designs, require clusters to contribute data for only a limited number of trial periods. Previous work has provided formulae based on asymptotic results for the calculation of the power of staircase designs to detect treatment effects of interest.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe conduct a simulation study to assess the finite sample performance of these formulae, and the impact of misspecifying the correlation structure when analysing data from staircase designs on inference for the treatment effect, under a range of realistic trial settings. This study focuses on basic staircase designs with one control period followed by one intervention period in each sequence. We simulate staircase trial datasets with continuous outcomes and a repeated cross-sectional measurement scheme under exchangeable and block-exchangeable intracluster correlation structures, and then fit linear mixed models with linear and categorical time period effects. For settings with a small number of clusters, Kenward-Roger and Satterthwaite small-sample corrections are applied. Comparisons are made between nominal and observed Type I error rates, and theoretically-derived study power and empirical power. The impact on inference for the treatment effect when misspecifying the intracluster correlation structure is assessed through considering performance metrics including bias and 95% confidence interval coverage.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData analysis assuming an exchangeable correlation structure and application of the Satterthwaite correction controls Type I error well when the correlation structure is correctly specified, and there are a sufficient number of clusters. For the true block-exchangeable model, when fitting the correct model with the Satterthwaite correction, the observed Type I error (empirical power) can be higher (lower) than the nominal (i.e., theoretical) value when there is only 1 cluster per sequence, but otherwise, it aligns well with the nominal (theoretical) value. Misspecification of the correlation structure (fitting an exchangeable model when the true structure is block-exchangeable) can lead to inflated Type I error and poor confidence interval coverage.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eStaircase designs with one cluster per sequence should be used with caution. Additionally, using a correlation structure that allows for decay is preferable for making valid inferences for the estimation of the treatment effect.\u003c/p\u003e","manuscriptTitle":"Inference for the treatment effect in staircase designs with continuous outcomes: a simulation study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-25 09:59:55","doi":"10.21203/rs.3.rs-5657090/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Accepted","date":"2025-04-15T06:47:13+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-02T15:33:36+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"262722638805059261488975853365644370229","date":"2025-03-26T05:43:07+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-24T13:53:12+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-03-24T11:35:48+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Research Methodology","date":"2025-03-21T05:14:29+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"48d1b343-deb8-47d9-b796-6c2681666cb6","owner":[],"postedDate":"March 25th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-05-12T16:07:45+00:00","versionOfRecord":{"articleIdentity":"rs-5657090","link":"https://doi.org/10.1186/s12874-025-02567-5","journal":{"identity":"bmc-medical-research-methodology","isVorOnly":false,"title":"BMC Medical Research Methodology"},"publishedOn":"2025-05-10 15:57:37","publishedOnDateReadable":"May 10th, 2025"},"versionCreatedAt":"2025-03-25 09:59:55","video":"","vorDoi":"10.1186/s12874-025-02567-5","vorDoiUrl":"https://doi.org/10.1186/s12874-025-02567-5","workflowStages":[]},"version":"v1","identity":"rs-5657090","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5657090","identity":"rs-5657090","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}