Reliability and Validity of Two Surgical Prioritization Systems for Reinstating Non-Emergent Benign Gynecologic Surgery During the COVID-19 Pandemic

Background: The COVID-19 pandemic necessitated cancellation of non-emergent surgeries worldwide. As local resources and infection rates allow, systems face an unprecedented backlog of patients, some of whom have incurred suffering and morbidity as a consequence of their delay. Clinically and ethically appropriate triaging is paramount to maximize patient outcomes, allocate resources, and minimize COVID-19 exposure risk. Methods: We performed a retrospective cohort study at a single academic tertiary care center in the US. The inter-rater reliability and validity of 2 novel surgical prioritization systems (Gyn-MeNTS and mESAS) were assessed. For benign gynecologic procedures postponed between 3/16/2020 and 4/30/2020, surgeons self-prioritized patients per institutional guidelines to minimize patient morbidity. Gyn-MeNTS scores were calculated by 3 raters and analyzed as continuous variables, with a lower score indicating more urgency/priority. The mESAS score was calculated by 2 raters and analyzed as a 3-level ordinal variable with a higher score indicating more urgency/priority. All 5 raters were blinded to reduce bias. Gyn-MeNTS inter-rater reliability was tested using Spearman r and paired t-tests were used to detect systematic differences between raters. Weighted kappa indicated mESAS reliability. Concurrent validity with mESAS and surgeon self-prioritization (SSP) was examined with Spearman r and logistic regression. Findings: 93 patients were included. Spearman r’s for all Gyn-MeNTS rater pairs were above 0·80 (0·84 for 1 vs. 2, 0·82 for 1 vs. 3, 0·82 for 2 vs. 3, all p<·0001) indicating strong agreement. The weighted kappa for the 2 mESAS raters was 0·57 (95% CI 0·40-0·73) indicating moderate agreement. When used together, both scores were significantly independently associated with SSP, with strong discrimination (AUC 0.89). Interpretation: Inter-rater reliability is acceptable for both scoring systems, and concurrent validity of each is moderate for predicting SSP, but discrimination improves to a high level when they are used together. Funding Statement: No funding was received for this study.Declaration of Interests: CQM, JSK, CZW, WAB, CMCB, and RLA have no conflicts of interest to disclose.Ethics Approval Statement: IRB exemption was obtained. (IRB#NCR202525)