Increasing test specificity without impairing sensitivity - lessons learned from SARS-CoV-2 serology

41

42 Serological tests are widely used in various medical disciplines for diagnostic and 43 monitoring purposes. Unfortunately, the sensitivity and specificity of test systems is often 44 poor, leaving room for false positive and false negative results. However, conventional 45

used to increase specificity decrease sensitivity and vice versa. Using SARS-46 CoV-2 serology as an example, we propose here a novel testing strategy: the 47 "Sensitivity Improved Two-Test" or "SIT²" algorithm. 48

49 SIT² involves confirmatory re-testing of samples with results falling in a predefined 50 retesting-zone of an initial screening test, with adjusted cut-offs to increase sensitivity. 51 We verified and compared the performance of SIT² to single tests and orthogonal testing 52 (OTA) in an Austrian cohort (1,117 negative, 64 post-COVID positive samples) and 53 validated the algorithm in an independent British cohort (976 negatives, 536 positives). 54

55 The specificity of SIT² was superior to single tests and non-inferior to OTA. The 56 sensitivity was maintained or even improved using SIT² when compared to single tests 57 or OTA. SIT² allowed correct identification of infected individuals even when a live virus 58 neutralization assay could not detect antibodies. Compared to single testing or OTA, 59 SIT² significantly reduced total test errors to 0.46% (0.24-0.65) or 1.60% (0.94-2.38) at 60 both 5% or 20% seroprevalence. 61

62 For SARS-CoV-2 serology, SIT² proved to be the best diagnostic choice at both 5% and 63 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 4 20% seroprevalence in all tested scenarios. It is an easy to apply algorithm and can 64 potentially be helpful for the serology of other infectious diseases. 65 Running Head 66 Sensitivity-improved two-test serology 67 Abbreviations 68 Confidence interval (CI) 69 Coronavirus disease 2019 (COVID-19) 70 Enzyme-linked immunosorbent assay (ELISA) 71 Intensive Care Unit (ICU) 72 Neutralization titer (NT) 73 Nucleocapsid (NC) 74 Nucleoprotein (NP) 75 Orthogonal testing algorithms (OTA) 76 Polymerase chain reaction (PCR) 77 Receptor binding domain (RBD) 78 Sensitivity-Improved Two-Test (SIT2) 79 Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) 80 Spike protein (S) 81 Virus neutralization test (VNT) 82 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 5

83 Serological tests are commonly used diagnostic tools in a broad medical field, spanning 84 from infectiology (1) to autoimmunity (2), oncology (3) and transplantation medicine (4). 85 They also play a critical role in animal disease surveillance (5). However, many 86 serological tests come with acceptable but imperfect sensitivities and specificities. Tests 87 with specificities slightly above 90% are considered good (6) or even highly specific (3). 88 However, at low seroprevalence rates, every single percent counts: if the frequency of a 89 given disease in the tested population is only 5%, a specificity of 90% would mean that - 90 even at a sensitivity of 100% - 5 true positives would be matched by ten false positives. 91 Thus, the probability of an individual with a positive test (positive predictive value, PPV) 92 to be a true positive would be only 33%. 93 During the early phase of the SARS-CoV-2 pandemic, seroprevalences were far below 94 1% (7). Therefore, highly specific test systems were necessary (>99,5%) to provide 95 good positive predictive values (8). Sensitivity and specificity can be seen as 96 communicating vessels – the improvement of one is usually at the expense of the other 97 (9). Consequently, test systems adjusted by the manufacturers to very high specificities 98 (>99%) showed moderate sensitivity. This problem was particularly evident when non-99 hospitalized patients were included in the cohorts studied (10-12). To further increase 100 specificity at very low seroprevalence levels, various methods have been proposed, e.g., 101 raising the thresholds for positivity or confirming a positive result with a second test 102 (orthogonal testing) (9, 13, 14). Unfortunately, these specificity improvement strategies 103 inevitably lead to a further reduction of the previously moderate sensitivities. 104 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 6 As the pandemic progressed, the problem became more pronounced as antibodies 105 declined, and sophisticated statistical models were required to compensate for the 106 waning sensitivity (15). In the case of SARS-CoV-2, as with any evolving pandemic, 107 increasing seroprevalence rates worldwide have attenuated the need for higher 108 specificity. 109 However, the problem persists in non-epidemic diseases where seroprevalence remains 110 low. Moreover, each new pandemic begins with extremely low seroprevalence rates as 111 well, and in the future, we should have better diagnostic strategies in infectious serology 112 ready here. 113 In the present work, we propose for the first time an orthogonal test algorithm based on 114 real-life data for the SARS-CoV-2 antibody tests of Roche, Abbott, DiaSorin, and two 115 commercial SARS-CoV-2 ELISAs (16) intending to maximize both specificity and 116 sensitivity at the same time. Although our algorithm follows a general principle, it was 117 developed based on SARS-CoV-2 antibody tests. The SARS-CoV-2 pandemic provided 118 a unique opportunity in this regard, as historical samples from before the pandemic are 119 negative by definition (thus allowing accurate specificity testing). In addition, sufficient 120 PCR-confirmed positive cases were available quickly, ensuring a sophisticated and 121 reliable sensitivity verification. Thus, for SARS-CoV-2 - in contrast to most other 122 circulating microorganisms - a realistic and unusually accurate estimation of specificities 123 and sensitivities of serological tests was possible. We benefited from this advantage to 124 develop our sophisticated diagnostic algorithm. 125 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 7

126 Study design and cohorts 127 Sera used in this non-blinded prospective cross-sectional study were either residual 128 clinical specimens or samples stored in the MedUni Wien Biobank (n=1,181), a facility 129 specialized in the preservation and storage of human biomaterial, which operates within 130 a certified quality management system (ISO 9001:2015) (17). 131 For derivation of the SIT2 algorithm, sample sets from individuals known to be negative 132 and positive were established for testing. As previously described (18), samples 133 collected before 01.01.2020 (i.e., assumed SARS-CoV-2 negative) were used as a 134 specificity cohort (n=1117): a cross-section of the Viennese population (the LEAD 135 study)(19), preselected for samples collected between November and April to enrich for 136 seasonal infections (n=494); a collection of healthy voluntary donors (n= 265); a 137 disease-specific collection of samples from patients with rheumatic diseases (n=358); 138 (see also Tables S1 and S2). 139 The SARS-CoV-2 positive cohort (n=64 samples from n=64 individuals) included 140 patients testing positive with RT-PCR during the first wave and their close, symptomatic 141 contacts. Of this cohort five individuals were asymptomatic, 42 had mild-moderate 142 symptoms, four reported severe symptoms, and 13 were admitted to the Intensive Care 143 Unit (ICU). The timing of symptom onset was determined by a questionnaire for 144 convalescent donors and by reviewing individual health records for patients and was in 145 median 41 [26,25-49] days. For asymptomatic donors (n=5), SARS-CoV-2 RT-PCR 146 confirmation time was used instead (for more details, see Tables S1 and S3). All 147 included participants gave written informed consent to donate their samples for research 148 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 8 purposes. The overall evaluation plan conformed with the Declaration of Helsinki as well 149 as relevant regulatory requirements. It was reviewed and approved by the ethics 150 committee of the Medical University of Vienna (1424/2020). 151 For validation of the SIT² algorithm, we used data from an independent United Kingdom 152 cohort (20), including 1,512 serum/plasma samples (536 PCR confirmed SARS-CoV-2 153 positive cases; 976 negative cases, collected earlier than 2017). 154 Antibody testing 155 For the derivation analyses, SARS-CoV-2 antibodies were either measured according to 156 the manufacturers' instructions on three different commercially available automated 157 platforms (Roche Elecsys® SARS-CoV-2 [nuclecapsid total antibody assay, further 158 referred to as Roche NC], Abbott SARS-CoV-2-IgG assay [nucleocapsid IgG assay, 159 Abbott NC], DiaSorin LIASION® SARS-CoV-2 S1/S2 assay [S1/S2 combination antigen 160 IgG assay, DiaSorin S1/S2]) or using 96-well enzyme-linked immunosorbent assays 161 (ELISAs) (Technoclone Technozym® RBD and Technozym® NP) yielding quantitative 162 results(16) (for details see Supplement, Supplemental Methods). The antibody assays 163 used in the validation cohort were Abbott NC, DiaSorin S1/S2, Roche NC, Siemens RBD 164 total antibody, and a novel 384-well trimeric spike protein ELISA (Oxford Immunoassay) 165 (20), resulting in 20 evaluable combinations. All samples from the Austrian SARS-CoV-166 2-positive cohort also underwent live virus neutralization testing (VNT), and 167 neutralization titers (NT) were calculated, as is described in detail in the Supplemental 168 Methods. 169 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 9 Sensitivity improved two-test method (SIT²) 170 Our newly developed sensitivity improved two-test (SIT²) method consists of the 171 following key components: i) sensitivity improvement by cut-off modification and ii) 172 specificity rescue by a second, confirmatory test (Fig. 1A). 173 For the first component of the SIT² algorithm, positivity thresholds were optimized for 174 sensitivity according to the first published alternative thresholds for the respective 175 assays, calculated e.g. by ROC-analysis (21-23). Additionally, a high cut-off, above 176 which a result can be reliably regarded as true positive without the need for further 177 confirmation, was defined. These levels were based on in-house observations(18) and 178 represent those values (including a safety margin) above which no more false positives 179 were found. The highest results seen in false-positives were 1.800 COI, 2.86 Index, and 180 104.0 AU/mL, respectively. Hence, we defined the high cut-off for Roche and Abbott as 181 3.00 COI/Index and for DiaSorin as 150.0 AU/mL. The lowering of positivity thresholds 182 improves sensitivity; the high cut-off prevents unnecessary re-testing of clearly positive 183 samples. Moreover, the high cut-off avoids possible erroneous exclusion by the 184 confirmatory test. The newly defined interval between the reduced threshold for positivity 185 and the high cut-off is the re-testing zone (Fig. 1A). The initial antibody test (screening 186 test) is then followed by a confirmatory test, whereby positive samples from the re-187 testing zone of the screening test are re-tested. Also, for the confirmatory test, 188 sensitivity-adapted assay thresholds are needed (Figs.1A, 1B). As false-positive 189 samples are usually only positive in one test system (Fig. S1), false positives can be 190 identified, and specificity markedly restored with minimal additional testing as most 191 samples do not fall within the re-testing zone (18, 24). A flowchart of the testing strategy 192 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 10 and the applied cut-off levels and their associated quality criteria are presented in Figs. 193 1B, 1C. 194 Test strategy evaluation 195 On the derivation cohort, we compared the overall performance of the following SARS-196 CoV-2 antibody testing strategies: i) testing using single assays; ii) simple lowering of 197 thresholds; iii) classical orthogonal testing (OTA), and iv) our newly developed SIT2 198 algorithm at assumed seroprevalences of 5% and 20%. As part of the derivation, we 199 then compared the performance of OTAs and SIT2 against the results of a virus 200 neutralization assay. On the validation cohort, we then compared the performance of 201 OTAs and SIT2. Finally, we used data from this cohort to evaluate the performance of 202 SIT2 versus single tests at seroprevalences of 5%, 10%, 20%, and 50% if the Abbott and 203 DiaSorin assays (i.e., assays with varying degrees of discrepancies in sensitivity and 204 specificity) were used. 205 Statistical analysis 206 Unless otherwise indicated, categorical data are given as counts (percentages), and 207 continuous data are presented as median (interquartile range). Total test errors were 208 compared by Mann-Whitney tests or, in case they were paired, by Wilcoxon tests. 95% 209 confidence intervals (CI) for sensitivities and specificities were calculated according to 210 Wilson, 95% CI for predictive values were computed according to Mercaldo-Wald unless 211 otherwise indicated. Sensitivities and specificities were compared using z-scores. P 212 values <0.05 were considered statistically significant. All calculations were performed 213 using Analyse-it 5.66 (Analyse-it Software, Leeds, UK) and MedCalc 19.6 (MedCalc 214 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 11 bvba, Ostend, Belgium). Graphs were drawn using Microsoft Visio (Armonk, USA) and 215 GraphPad Prism 7.0 (La Jolla, USA). 216 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 12

217 In the derivation cohort of 1,117 pre-pandemic sera and 64 sera from convalescent 218 COVID-19 patients (80% non-hospitalized, 20% hospitalized), the Roche NC, Abbott 219 NC, and DiaSorin S1/S2 antibody assays gave rise to 7/64, 10/64, and 11/64 false-220 negative, as well as to 3/1,117, 9/1,117, and 20/1,117 false-positive results. Assuming a 221 seroprevalence of 20%, this led to 2180, 3120, and 3440 false-negative results per 222 100,000 tests, and 240, 650 and 1,440 false-positive results per 100,000 tests 223 respectively (Fig. 2A, right panel). 224 Effects of threshold lowering on Sensitivity and Specificity 225 Lowering the positivity thresholds for the Roche NC, Abbott NC, and Diasorin S1/S2 to 226 0.165 COI, 0.55 Index and 9 AU/mL increased the sensitivity significantly and reduced 227 false-negative results to 63/64, 62/64, and 57/64 (320, 620, and 2,180 per 100,000 tests 228 at a seroprevalence of 20%), but substantially increased false-positive results to 229 18/1,117, 27/1,117, and 31/1,117, respectively (1,280, 1,920 and 2,240 per 100,000 230 tests, an assumed seroprevalence of 20%; Table S4, Fig. 2A, right panel). 231 Classical OTA compared to SIT2 232 Subsequently, we evaluated 12 OTA combinations using the fully automated SARS-233 CoV-2 antibody tests from Roche NC, Abbott NC, and DiaSorin S1/S2 as screening 234 tests, each combined with one of the other fully automated assays or a commercially 235 available NC or RBD-specific ELISA as a confirmation test. Combining these tests as 236 classical OTAs significantly increased specificity and reduced false positives to 0 (0-237 1)/1,117. However, the rate of false negatives was 14 (12-16)/64 (1,095 [955-1,230] per 238 100,000 tests at 20% seroprevalence), and therefore considerably higher than for single 239 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 13 testing strategies. In contrast, the SIT2 algorithm minimized false positives to 0 (0-240 2)/1,117 (0 [0-140) per 100,000 tests at 20% seroprevalence) while also reducing false 241 negatives to 5 (3-8)/64 (1,560 [940-2420] per 100,000 tests at 20% seroprevalence, Fig. 242 2A right panel; Table S5). 243 Reduction of total error rates by the Sensitivity-Improved Two-Test 244 Of all the methods assessed, SIT2 reached the lowest total error rates per 100,000 tests 245 under both 5% and 20% assumed seroprevalence (455 [235-685] and 1,600 [940-2,490] 246 per 100,000 tests) (Fig. 2B). At a seroprevalence of 5 %, OTA on average performed 247 better than individual tests, and the total error rates of the single tests were higher for 248 the Abbott NC and DiaSorin S1/S2 assay (OTA 1,095 [955-1,325] vs. 830 [Roche NC], 249 1,540 [Abbott NC] and 2,570 [DiaSorin S1/S2] per 100,000 tests). But with a 250 seroprevalence of 20 %, performance of OTAs, worsened compared to single tests 251 (OTA 4,380 [3,820-5,000] vs 1,600 [Roche], 2,540 [Abbott] and 4,420 [DiaSorin] per 252 100,000 tests) (Fig. 2B). Therefore, at both 5% and 20% seroprevalence, SIT2 resulted 253 in the lowest overall errors. Compared to OTAs, SIT2 yielded a similar improvement in 254 specificity while not suffering from the significant sensitivity reduction (Fig. S2). Since the 255 better overall performance of SIT2 compared to OTAs was not due to increased 256 specificity but improved sensitivity, we subsequently set out to examine these 257 differences in more detail. 258 Sensitivities of single tests, OTA and SIT2 in relation to Neutralization Testing 259 Next, we compared the sensitivities of the three screening tests as single tests and in 260 both two-test methods (OTA and SIT2), benchmarking them using the Austrian 261 sensitivity cohort (n=64) simultaneously evaluated with an authentic SARS-CoV-2 virus 262 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 14 neutralization test (VNT). Regardless of the screening test used (Roche NC, Abbott NC, 263 or DiaSorin S1/S2), OTAs had lower sensitivities than single tests (80.5% [78.5-83.6], 264 78.1% [75.8-82.8], or 75.8% [71.5-78.9] vs. 89.1%, 84.4%, or 82.8% respectively), and 265 SIT2 showed the best sensitivities of all methods (95.3% [93.0-96.5], 93.8% [92.2-96.5], 266 or 87.5% [85.1-88.7]) (Fig. 3). SIT2 algorithms, including the Roche NC and Abbott NC 267 assays, achieved similar or even higher sensitivities than VNT (Fig. 3, VNT reference 268 line), made possible by the unique re-testing zone of SIT2 (Fig. S3). 269 Validation of the Sensitivity-Improved Two-Test using an independent cohort 270 To confirm the improved sensitivity of SIT 2 compared to OTA, we analyzed the 271 sensitivities of OTAs and SIT2 in an independent validation cohort of 976 pre-pandemic 272 samples and 536 post-COVID samples. Out of 20 combinations using the assays Roche 273 NC (total antibody), Abbott NC (IgG), DiaSorin S1/S2 (IgG), Siemens RBD (total 274 antibody), and Oxford trimeric-S (IgG), a statistically significant improvement in 275 sensitivities over OTAs was shown for SIT2 in 18 combinations (Fig. 4). The 276 performance was comparable for the remaining two combinations (Siemens RBD with 277 Oxford trimeric-S and vice versa). Still, no statistically significant improvement could be 278 achieved due to the high pre-existing sensitivities of these assays on this particular 279 sample cohort. 280 To further illustrate the effect of SIT2 on the outcome of SARS-CoV-2 antibody testing, 281 we compared single testing versus SIT2 with the Abbott and DiaSorin assays at varying 282 assumed seroprevalences (5, 10, 20, and 50%), given that the Abbott NC assay is a 283 highly specific (99.9%), but moderately sensitive test (92.7%), and the DiaSorin S1/S2 284 assay has the most limited specificity (98.7%) of all evaluated assays but an acceptable 285 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 15 sensitivity (96.3%). Regardless of whether a lack of specificity (DiaSorin S1/S2) or 286 sensitivity (Abbott NC) had to be compensated for, SIT2 improved the overall error rate 287 compared to the individual tests in all four combinations and at all four assumed 288 seroprevalence levels (Fig. 5). 289 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 16

290 Serology is a commonly used, multi-purpose analytical method (1-4). However, not all 291 serologic assays have appropriate sensitivities and specificities, especially in low-292 prevalence settings. The SARS-CoV-2 pandemic prompted the simultaneous 293 development of several antibody tests and, which is rare otherwise, allowed to evaluate 294 these tests with both confirmed positive and negative cases, the latter derived from 295 biobank collections established before the virus emerged. In the case of SARS-CoV-2, 296 false-positive samples are usually not simultaneously reactive in different test systems 297 (14, 25). This led to the hypothesis that reducing the threshold for positivity in screening 298 and confirmation tests would increase the specificity without impairing the sensitivity. A 299 further improvement in sensitivity was possible by defining a high cut-off for the 300 screening test, above which, due to the excellent reliability of high test results, no further 301 confirmation (and, thereby, a possible false-negative result in the confirmation test) was 302 necessary. 303 In the early waves of the SARS-CoV-2 pandemic, many commercially available SARS-304 CoV-2 antibody tests did not provide sufficient specificity to achieve acceptable positive 305 predictive values (PPVs), for example, at a seroprevalence rate of 1-5% (13, 25). 306 Lowering positivity thresholds might improve test sensitivity (21-23) and conventional 307 orthogonal testing can maximize specificity (26-28). The latter might increase the 308 positive predictive value, but PPV will only be relevantly increased at low 309 seroprevalences. However, since seroprevalence is often neither known and varies 310 widely from region to region, it is difficult to judge whether a less specific or less 311 sensitive test is the lesser of two evils. 312 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 17 However, the reported imperfect specificities and sensitivities of SARS-CoV-2 antibody 313 assays are still considerably higher than those of many other serological tests used in 314 clinical routine. Based on actual data related to SARS-CoV-2, which are in principle 315 generalizable for all types of serological tests, we, therefore, we propose a new, 316 universally adaptable two-test system that could, in the case of SARS-CoV-2, perform 317 better than any other known approach regardless of the actual seroprevalence: the 318 sensitivity-improved Two-Test or SIT2. Its generalizability can be inferred from the 319 following features: i) the adapted cut-offs used to optimize sensitivity were determined in 320 various independent studies and were not explicitly calculated for our cohort (21-23); ii) 321 SIT2 was effective, albeit with different efficiencies, in a total of 32 different test 322 combinations; and iii) SIT2 was successfully validated in an independent cohort which 323 was profoundly different from the derivation cohort. Therefore, SIT2 does not require a 324 particular infrastructure or the availability of high-performance individual test systems but 325 achieves the best performance out of an available test. 326 Our SIT2 strategy can rescue the specificity with minimal repeat testing required (see 327 Table S6). For example, when applying the Roche NC as a screening test to our cohort, 328 only 27 out of 1,181 samples needed confirmation testing with the Abbott NC test to 329 correctly identify 62/64 true positives. Simultaneously, all false-positive results were 330 eliminated, including those added by lowering the cut-offs (Table S4 and Fig. S1). 331 Additionally, it was more sensitive than virus neutralization testing, which identified only 332 60/64 clinical positives (Fig. 3). This result is not completely surprising as it is known that 333 not all patients who recovered from COVID-19 show detectable levels of neutralizing 334 antibodies (29). Nevertheless, it should be noted that although antibody binding assays 335 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 18 may have a higher sensitivity than neutralization assays, they only partially reflect the 336 functional activity of SARS-CoV-2-reactive antibodies(30). 337 Our study has both strengths and limitations. One strength is the size of the cohorts 338 examined, both in deriving the SIT2 algorithm (N=1,181) and validating it (N=1,512). The 339 composition of our specificity cohort is also unique: it consists of three sub-cohorts with 340 selection criteria to further challenge analytical specificity. The lower cut-offs used to 341 increase sensitivity were not modeled within our datasets but were derived from ROC-342 analyses data of independent studies (21-23). Furthermore, we were able to test the 343 performance of the two-test systems in a total of 32 combinations, 12 in the derivation 344 cohort and another 20 combinations in the validation cohort. As a limitation, in the 345 Austrian cohort, only samples ≥ 14 days after symptom onset were included. Therefore, 346 no conclusions on the sensitivity of the early seroconversion phase can be made from 347 these data. Furthermore, mild and asymptomatic cases were underrepresented in the 348 British cohort, perhaps leading to an observed higher sensitivity of the test systems. 349 In conclusion, we describe the novel two-test algorithm SIT2, which makes it possible to 350 maintain or even significantly improve sensitivity while approaching 100% specificity. 351 Acknowledgments 352 We sincerely thank Marika Gerdov, Susanne Keim, Karin Mildner, Elisabeth Ponweiser, 353 Manuela Repl, Ilse Steiner, Christine Thun, and Martina Trella for excellent technical 354 assistance. Finally, we want to thank all the donors of the various study cohorts. The 355 MedUni Wien Biobank is funded to participate in the biobank consortium BBMRI.at 356 (www.bbmri.at) by the Austrian Federal Ministry of Science, Research and Technology. 357 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 19 There was no external funding received for the work presented. However, test kits for 358 the Technoclone ELISAs were kindly provided by the manufacturer. All authors have 359 read the journal’s policy on disclosure of potential conflicts of interest and the following 360 has to be disclosed: NP-N received a travel grant from DiaSorin. DWE reports lecture 361 fees from Gilead outside the submitted work. OCB reports grants from GSK, grants from 362 Menarini, grants from Boehringer Ingelheim, grants from Astra, grants from MSD, grants 363 from Pfizer, and grants from Chiesi, outside the submitted work. SH does receive 364 unrestricted research grants (GSK, Boehringer, Menarini, Chiesi, Astra Zeneca, MSD, 365 Novartis, Air Liquide, Vivisol, Pfizer, TEVA) for the Ludwig Boltzmann Institute of COPD 366 and Respiratory Epidemiology, and is on advisory boards for G. SK, Boehringer 367 Ingelheim, Novartis, Menarini, Chiesi, Astra Zeneca, MSD, Roche, Abbvie, Takeda, and 368 TEVA for respiratory oncology and COPD. PQ is an advisory board member for Roche 369 Austria and reports personal fees from Takeda outside the submitted work. The Dept. of 370 Laboratory Medicine (Head: OWF) received compensations for advertisement on 371 scientific symposia from Roche, DiaSorin, and Abbott and holds a grant for evaluating 372 an in-vitro diagnostic device from Roche. CJB is a Board Member of Technoclone. HH 373 receives compensations for biobank services from Glock Health Science and Research 374 and BlueSky immunotherapies. 375

376 1. Lass-Flörl C, Samardzic E, Knoll M. Serology anno 2021-fungal infections: from 377 invasive to chronic. Clin Microbiol Infect. 2021;27(9):1230-41. 378 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 20 2. Popp A, Kivelä L, Fuchs V, Kurppa K. Diagnosing Celiac Disease: Towards Wide-379 Scale Screening and Serology-Based Criteria? Gastroenterol Res Pract. 380 2019;2019:2916024. 381 3. Hibbert J, Halec G, Baaken D, Waterboer T, Brenner N. Sensitivity and Specificity 382 of Human Papillomavirus (HPV) 16 Early Antigen Serology for HPV-Driven 383 Oropharyngeal Cancer: A Systematic Literature Review and Meta-Analysis. Cancers 384 (Basel). 2021;13(12). 385 4. Smatti MK, Al-Sadeq DW, Ali NH, Pintus G, Abou-Saleh H, Nasrallah GK. 386 Epstein-Barr Virus Epidemiology, Serology, and Genetic Variability of LMP-1 Oncogene 387 Among Healthy Population: An Update. Front Oncol. 2018;8:211. 388 5. Gaudino M, Moreno A, Snoeck CJ, Zohari S, Saegerman C, O'Donovan T, et al. 389 Emerging Influenza D virus infection in European livestock as determined in serology 390 studies: Are we underestimating its spread over the continent? Transbound Emerg Dis. 391 2021;68(3):1125-35. 392 6. Shibata Y, Horita N, Yamamoto M, Tsukahara T, Nagakura H, Tashiro K, et al. 393 Diagnostic test accuracy of anti-glycopeptidolipid-core IgA antibodies for Mycobacterium 394 avium complex pulmonary disease: systematic review and meta-analysis. Sci Rep. 395 2016;6:29325. 396 7. Arora RK, Joseph A, Van Wyk J, Rocco S, Atmaja A, May E, et al. SeroTracker: a 397 global SARS-CoV-2 seroprevalence dashboard. Lancet Infect Dis. 2020. 398 8. Bailey D, Konforte D, Barakauskas VE, Yip PM, Kulasingam V, Abou El Hassan 399 M, et al. Canadian society of clinical chemists (CSCC) interim consensus guidance for 400 testing and reporting of SARS-CoV-2 serology. Clin Biochem. 2020. 401 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 21 9. Turbett SE, Anahtar M, Dighe AS, Garcia Beltran W, Miller T, Scott H, et al. 402 Evaluation of Three Commercial SARS-CoV-2 Serologic Assays and their Performance 403 in Two-Test Algorithms. J Clin Microbiol. 2020. 404 10. Gattinger P, Borochova K, Dorofeeva Y, Henning R, Kiss R, Kratzer B, et al. 405 Antibodies in serum of convalescent patients following mild COVID-19 do not always 406 prevent virus-receptor binding. Allergy. 2020. 407 11. Long QX, Tang XJ, Shi QL, Li Q, Deng HJ, Yuan J, et al. Clinical and 408 immunological assessment of asymptomatic SARS-CoV-2 infections. Nat Med. 409 2020;26(8):1200-4. 410 12. Baron RC, Risch L, Weber M, Thiel S, Grossmann K, Wohlwend N, et al. 411 Frequency of serological non-responders and false-negative RT-PCR results in SARS-412 CoV-2 testing: a population-based study. Clin Chem Lab Med. 2020. 413 13. Manthei DM, Whalen JF, Schroeder LF, Sinay AM, Li SH, Valdez R, et al. 414 Differences in Performance Characteristics Among Four High-Throughput Assays for the 415 Detection of Antibodies Against SARS-CoV-2 Using a Common Set of Patient Samples. 416 Am J Clin Pathol. 2020. 417 14. Pfluger LS, Bannasch JH, Brehm TT, Pfefferle S, Hoffmann A, Norz D, et al. 418 Clinical evaluation of five different automated SARS-CoV-2 serology assays in a cohort 419 of hospitalized COVID-19 patients. J Clin Virol. 2020;130:104549. 420 15. Buss LF, Prete CA, Jr., Abrahim CMM, Mendrone A, Jr., Salomon T, de Almeida-421 Neto C, et al. Three-quarters attack rate of SARS-CoV-2 in the Brazilian Amazon during 422 a largely unmitigated epidemic. Science. 2021;371(6526):288-92. 423 16. Klausberger M, Duerkop M, Haslacher H, Wozniak-Knopp G, Cserjan-424 Puschmann M, Perkmann T, et al. A comprehensive antigen production and 425 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 22 characterisation study for easy-to-implement, specific and quantitative SARS-CoV-2 426 serotests. EBioMedicine. 2021;67:103348. 427 17. Eskandary F, Regele H, Baumann L, Bond G, Kozakowski N, Wahrmann M, et al. 428 A Randomized Trial of Bortezomib in Late Antibody-Mediated Kidney Transplant 429 Rejection. J Am Soc Nephrol. 2018;29(2):591-605. 430 18. Perkmann T, Perkmann-Nagele N, Breyer MK, Breyer-Kohansal R, Burghuber 431 OC, Hartl S, et al. Side-by-Side Comparison of Three Fully Automated SARS-CoV-2 432 Antibody Assays with a Focus on Specificity. Clin Chem. 2020;66(11):1405-13. 433 19. Breyer-Kohansal R, Hartl S, Burghuber OC, Urban M, Schrott A, Agusti A, et al. 434 The LEAD (Lung, Heart, Social, Body) Study: Objectives, Methodology, and External 435 Validity of the Population-Based Cohort Study. J Epidemiol. 2019;29(8):315-24. 436 20. National S-C-SAEG. Performance characteristics of five immunoassays for 437 SARS-CoV-2: a head-to-head benchmark comparison. Lancet Infect Dis. 438 2020;20(12):1390-400. 439 21. Favresse J, Eucher C, Elsen M, Tre-Hardy M, Dogne JM, Douxfils J. Clinical 440 Performance of the Elecsys Electrochemiluminescent Immunoassay for the Detection of 441 SARS-CoV-2 Total Antibodies. Clin Chem. 2020;66(8):1104-6. 442 22. Lau CS, Oh HML, Hoo SP, Liang YL, Phua SK, Aw TC. Performance of an 443 automated chemiluminescence SARS-CoV-2 IG-G assay. Clin Chim Acta. 444 2020;510:760-6. 445 23. Bonelli F, Sarasini A, Zierold C, Calleri M, Bonetti A, Vismara C, et al. Clinical and 446 Analytical Performance of an Automated Serological Test That Identifies S1/S2-447 Neutralizing IgG in COVID-19 Patients Semiquantitatively. J Clin Microbiol. 2020;58(9). 448 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 23 24. Pfluger LS, Bannasch JH, Brehm TT, Pfefferle S, Hoffmann A, Norz D, et al. 449 Clinical evaluation of five different automated SARS-CoV-2 serology assays in a cohort 450 of hospitalized COVID-19 patients. J Clin Virol. 2020;130:104549. 451 25. Perkmann T, Perkmann-Nagele N, Breyer MK, Breyer-Kohansal R, Burghuber 452 OC, Hartl S, et al. Side by side comparison of three fully automated SARS-CoV-2 453 antibody assays with a focus on specificity. Clin Chem. 2020. 454 26. Turbett SE, Anahtar M, Dighe AS, Garcia Beltran W, Miller T, Scott H, et al. 455 Evaluation of Three Commercial SARS-CoV-2 Serologic Assays and Their Performance 456 in Two-Test Algorithms. J Clin Microbiol. 2020;59(1). 457 27. Bolotin S, Tran V, Osman S, Brown KA, Buchan SA, Joh E, et al. SARS-CoV-2 458 seroprevalence survey estimates are affected by anti-nucleocapsid antibody decline. J 459 Infect Dis. 2021. 460 28. Irsara C, Egger AE, Prokop W, Nairz M, Loacker L, Sahanic S, et al. Evaluation of 461 four commercial, fully automated SARS-CoV-2 antibody tests suggests a revision of the 462 Siemens SARS-CoV-2 IgG assay. Clin Chem Lab Med. 2021;0(0). 463 29. Kalkan Yazici M, Koc MM, Cetin NS, Karaaslan E, Okay G, Durdu B, et al. 464 Discordance between Serum Neutralizing Antibody Titers and the Recovery from 465 COVID-19. J Immunol. 2020;205(10):2719-25. 466 30. Padoan A, Bonfante F, Pagliari M, Bortolami A, Negrini D, Zuin S, et al. Analytical 467 and clinical performances of five immunoassays for the detection of SARS-CoV-2 468 antibodies in comparison with neutralization activity. EBioMedicine. 2020;62:103101. 469 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 24 Figure Legends 470 Fig. 1. A) The Sensitivity Improved Two-Test (SIT2) algorithm includes sensitivity 471 improvement by adapted cut-offs and a subsequent specificity rescue by re-testing all 472 samples within the re-testing zone of the screening test by a confirmatory test. B) 473 Testing algorithm for SIT2 utilizing a screening test on an automated platform 474 (ECLIA/Roche, CMIA/Abbott, CLIA/DiaSorin) and a confirmation test, either on one of 475 the remaining platforms or tested by means of ELISA (Technozym RBD, NP). C) All test 476

between a reduced cut-off suggested by the literature, and a higher cut-off, 477 above which no more false-positives were observed, were subject to confirmation 478 testing. **… results between 12.0 and 15.0, which are according to the manufacturer 479 considered borderline, were treated as positives; ***… suggested as a cut-off for 480 seroprevalence testing; ****… determined by in-house modeling; 1… see (21); 2… see 481 (22); 3… see (23). 482 Fig. 2. False-positives (FP)/false-negatives (FN) (A) and total error (B) of single tests, 483 tests with reduced thresholds according to (21-23), orthogonal testing algorithms (OTAs) 484 and the Sensitivity Improved Two-Test (SIT2) algorithm at 5 and 20% estimated 485 seroprevalence. Data in (B) were compared by Mann-Whitney tests (unpaired) or 486 Wicoxon tests (paired). *… P<0.05; **...P<0.01; ***…P<0.001. 487 Fig. 3. Sensitivities of single tests, orthogonal testing algorithms (OTAs) and the 488 Sensitivity Improved Two-Test (SIT2) algorithm. The dotted line indicates the sensitivity 489 of virus neutralization test (VNT). 490 Fig. 4. Differences in sensitivity and specificity (mean±95% confidence interval) between 491 the Sensitivity Improved Two-Test (SIT2) algorithm and standard orthogonal testing 492 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint 25 algorithms (OTAs) within the UK validation cohort. *… P<0.05; **...P<0.01; 493 ***…P<0.001; ****...P<0.0001 494 Fig. 5. Comparing false-positives (FP), false-negatives (FN), and total error (TE) for two 495 selected test systems, A) Abbott, B) DiaSorin, between different Sensitivity Improved 496 Two-Test (SIT2) combinations and the respective single test within the UK validation 497 cohort for different estimated seroprevalences. 498 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted January 12, 2022. ; https://doi.org/10.1101/2020.11.05.20226449doi: medRxiv preprint