Detecting departures from the conditional independence assumption in diagnostic latent class models: A simulation study

Detecting departures from the conditional independence assumption in diagnostic latent class models: A simulation study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Detecting departures from the conditional independence assumption in diagnostic latent class models: A simulation study Yasin Okkaoglu, Nicky J Welton, Hayley E Jones This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4764167/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 05 Dec, 2024 Read the published version in BMC Medical Research Methodology → Version 1 posted 13 You are reading this latest preprint version Abstract Background Latent class models can be used to estimate diagnostic accuracy without a gold standard test. Early studies often assumed independence between tests given the true disease state, however this can lead to biased estimates when there are inter-test dependencies. Residual correlation plots and chi-squared statistics have been commonly utilized to assess the validity of the conditional independence assumption and, when it does not hold, identify which test pairs are conditionally dependent. We aimed to assess the performance of these tools with a simulation study covering a wide range of scenarios. Methods We generated data sets from a model with four tests and a dependence between tests 1 and 2 within the diseased group. We varied sample size, prevalence, covariance, sensitivity and specificity, with 504 combinations of these in total, and 1000 data sets for each combination. We fitted the conditional independence model in a Bayesian framework, and reported absolute bias, coverage, and how often the residual correlation plots, G 2 and χ 2 statistics indicated lack-of-fit globally or for each test pair. Results Across all settings, residual correlation plots, pairwise G 2 and χ 2 detected the correct correlated pair of tests only 12.1%, 10.3%, and 10.3% of the time, respectively, but incorrectly suggested dependence between tests 3 and 4 64.9%, 49.7%, and 49.5% of the time. We observed some variation in this across parameter settings, with these tools appearing to perform more as intended when tests 3 and 4 were both much more accurate than tests 1 and 2. Residual correlation plots, G 2 and χ 2 statistics identified a lack of overall fit in 74.3%, 64.5% and 67.5% of models, respectively. The conditional independence model tended to overestimate the sensitivities of the correlated tests (median bias across all scenarios 0.094, 2.5th and 97.5th percentiles − 0.003, 0.397) and underestimate prevalence and the specificities of the uncorrelated tests. Conclusions Residual correlation plots and chi-squared statistics cannot be relied upon to identify which tests are conditionally dependent, and also have relatively low power to detect lack of overall fit. This is important since failure to account for conditional dependence can lead to highly biased parameter estimates. Latent class model diagnostic accuracy conditional independence model selection goodness of fit residual correlation plots Full Text Additional Declarations No competing interests reported. Supplementary Files Additionalfile1.pdf Additionalfile2.pdf Additionalfile3.pdf Cite Share Download PDF Status: Published Journal Publication published 05 Dec, 2024 Read the published version in BMC Medical Research Methodology → Version 1 posted Editorial decision: Revision requested 25 Sep, 2024 Reviews received at journal 24 Sep, 2024 Reviewers agreed at journal 16 Sep, 2024 Reviews received at journal 15 Sep, 2024 Reviewers agreed at journal 05 Sep, 2024 Reviewers agreed at journal 03 Sep, 2024 Reviewers agreed at journal 29 Jul, 2024 Reviewers agreed at journal 22 Jul, 2024 Reviewers invited by journal 20 Jul, 2024 Editor invited by journal 19 Jul, 2024 Editor assigned by journal 19 Jul, 2024 Submission checks completed at journal 19 Jul, 2024 First submitted to journal 18 Jul, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4764167","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":336574977,"identity":"263942d7-320d-4582-950e-5df782eb4f6c","order_by":0,"name":"Yasin Okkaoglu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABJUlEQVRIie3RMWrDMBQG4GcMySLqVSal6RFkBG4Ht72KjEFdTAgYSkaXgD12VW9R6AUMGryUeA10cSg0HTIIeoFIydAMSroWqh+EHuh94gkBuLj80TR6EWyqHhIGvqmmuxP/d8KAG+KVQE4TOCCS6e00IW27klNIaFhL+ZVW3eSqHq57RWAclIgSG3nLiRTA4xHi/Dqt3otziaJHQSASDaLMRpocJAKZXEAeE01S4SNvjgh4L4BoYyPdut+TYGPIQpPhypC7o2TJiCHxCOe0T6tGE4gMSQ2xDRYuN5oQTkPxyYEtsgL7KHoWBGdCDgrb88+6+49vNEsi0WVSqYfbCQ7aXqlZcvNUz1+xhVzupt1fNjhswEc/clz+1L6y97i4uLj892wBzt9hb1DXUkwAAAAASUVORK5CYII=","orcid":"","institution":"University of Bristol","correspondingAuthor":true,"prefix":"","firstName":"Yasin","middleName":"","lastName":"Okkaoglu","suffix":""},{"id":336574978,"identity":"8d9bb410-e17c-4083-b938-692f583e8798","order_by":1,"name":"Nicky J Welton","email":"","orcid":"","institution":"University of Bristol","correspondingAuthor":false,"prefix":"","firstName":"Nicky","middleName":"J","lastName":"Welton","suffix":""},{"id":336574979,"identity":"6113a430-fa43-4ebd-b86d-2e5663fd4577","order_by":2,"name":"Hayley E Jones","email":"","orcid":"","institution":"University of Bristol","correspondingAuthor":false,"prefix":"","firstName":"Hayley","middleName":"E","lastName":"Jones","suffix":""}],"badges":[],"createdAt":"2024-07-18 17:06:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4764167/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4764167/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12874-024-02432-x","type":"published","date":"2024-12-05T15:57:54+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":70964850,"identity":"8ac03c64-88d5-499c-ab5c-a5f160a35462","added_by":"auto","created_at":"2024-12-09 16:16:44","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":807871,"visible":true,"origin":"","legend":"","description":"","filename":"DetectingdeparturesfromtheconditionalindependenceassumptionindiagnosticlatentclassmodelsAsimulationstudy.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4764167/v1_covered_a8044c92-fd8f-470d-abef-65f06260dc9c.pdf"},{"id":62359355,"identity":"15a09517-e874-4e3d-be47-5dda2ef932d6","added_by":"auto","created_at":"2024-08-13 09:52:18","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":143608,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4764167/v1/410b104bc339535a92471eac.pdf"},{"id":62359354,"identity":"ff722150-b2e8-4822-a1a6-d0596b6603e5","added_by":"auto","created_at":"2024-08-13 09:52:18","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":284576,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4764167/v1/500f8cad59169228838f6987.pdf"},{"id":62359356,"identity":"4776f5ae-8bd3-4966-8ed0-f03386f7543c","added_by":"auto","created_at":"2024-08-13 09:52:18","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":4926965,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile3.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4764167/v1/5146546e105fd813aa688fe3.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Detecting departures from the conditional independence assumption in diagnostic latent class models: A simulation study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Latent class model, diagnostic accuracy, conditional independence, model selection, goodness of fit, residual correlation plots","lastPublishedDoi":"10.21203/rs.3.rs-4764167/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4764167/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eLatent class models can be used to estimate diagnostic accuracy without a gold standard test. Early studies often assumed independence between tests given the true disease state, however this can lead to biased estimates when there are inter-test dependencies. Residual correlation plots and chi-squared statistics have been commonly utilized to assess the validity of the conditional independence assumption and, when it does not hold, identify which test pairs are conditionally dependent. We aimed to assess the performance of these tools with a simulation study covering a wide range of scenarios.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe generated data sets from a model with four tests and a dependence between tests 1 and 2 within the diseased group. We varied sample size, prevalence, covariance, sensitivity and specificity, with 504 combinations of these in total, and 1000 data sets for each combination. We fitted the conditional independence model in a Bayesian framework, and reported absolute bias, coverage, and how often the residual correlation plots, G\u003csup\u003e2\u003c/sup\u003e and \u003cem\u003eχ\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e statistics indicated lack-of-fit globally or for each test pair.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAcross all settings, residual correlation plots, pairwise G\u003csup\u003e2\u003c/sup\u003e and \u003cem\u003eχ\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e detected the correct correlated pair of tests only 12.1%, 10.3%, and 10.3% of the time, respectively, but incorrectly suggested dependence between tests 3 and 4 64.9%, 49.7%, and 49.5% of the time. We observed some variation in this across parameter settings, with these tools appearing to perform more as intended when tests 3 and 4 were both much more accurate than tests 1 and 2. Residual correlation plots, G\u003csup\u003e2\u003c/sup\u003e and \u003cem\u003eχ\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e statistics identified a lack of \u003cem\u003eoverall\u003c/em\u003e fit in 74.3%, 64.5% and 67.5% of models, respectively. The conditional independence model tended to overestimate the sensitivities of the correlated tests (median bias across all scenarios 0.094, 2.5th and 97.5th percentiles − 0.003, 0.397) and underestimate prevalence and the specificities of the uncorrelated tests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eResidual correlation plots and chi-squared statistics cannot be relied upon to identify which tests are conditionally dependent, and also have relatively low power to detect lack of overall fit. This is important since failure to account for conditional dependence can lead to highly biased parameter estimates.\u003c/p\u003e","manuscriptTitle":"Detecting departures from the conditional independence assumption in diagnostic latent class models: A simulation study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-13 09:52:12","doi":"10.21203/rs.3.rs-4764167/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-09-25T15:24:29+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-09-24T17:47:15+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"215743761174921016135627703873400885105","date":"2024-09-16T13:18:09+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-09-16T03:17:40+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"191490615919345599365754965654782942488","date":"2024-09-05T17:46:31+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"15280460281049683241682709043550495118","date":"2024-09-03T14:16:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"51677619567520652179457413066929574613","date":"2024-07-29T08:37:58+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"55410808601412632924271544788162307235","date":"2024-07-22T13:29:47+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-07-20T08:29:12+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-07-19T08:04:06+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-07-19T08:01:16+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-07-19T08:01:01+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Research Methodology","date":"2024-07-18T17:03:13+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"718b6deb-5d3b-4add-ae3a-59a87d8e0363","owner":[],"postedDate":"August 13th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-12-09T16:06:23+00:00","versionOfRecord":{"articleIdentity":"rs-4764167","link":"https://doi.org/10.1186/s12874-024-02432-x","journal":{"identity":"bmc-medical-research-methodology","isVorOnly":false,"title":"BMC Medical Research Methodology"},"publishedOn":"2024-12-05 15:57:54","publishedOnDateReadable":"December 5th, 2024"},"versionCreatedAt":"2024-08-13 09:52:12","video":"","vorDoi":"10.1186/s12874-024-02432-x","vorDoiUrl":"https://doi.org/10.1186/s12874-024-02432-x","workflowStages":[]},"version":"v1","identity":"rs-4764167","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4764167","identity":"rs-4764167","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}