Computed Tomography Severity Index vs. Other Indices in the Prediction of Severity and Mortality in Acute Pancreatitis: A Predictive Accuracy Meta-analysis

Acute pancreatitis (AP) is an inflammatory disease of the pancreas, one of the most common causes of hospitalization among gastrointestinal diseases (Lankisch et al., 2015 ). Based on the revised Atlanta classification, the severity of AP may be mild, moderate, or severe (Banks et al., 2013 ). Most cases of AP are mild, but the management of the moderate and severe forms of the disease with necrosis and multiorgan failure remains a challenge. The prognosis of the severe form is poor, it occurs in 8.8% of AP (Parniczky et al., 2016 ) and the mortality of severe AP (SAP) may reach 28% (Parniczky et al., 2016 ). Therefore, it is necessary to predict the severity of the disease because the early escalation of care and aggressive therapy may prevent complications and adverse outcomes of AP in high-risk patients. Unfortunately, research on pancreatitis is in danger, therefore, attempts to obtain clinically relevant data has very high importance (Szentesi et al., 2016 ). Currently, there are various scoring systems used for the early prediction of SAP. First, the Ranson score was used (Ranson and Pasternack, 1977 ), but later the acute physiology and chronic health examination II (APACHE II) scoring system seemed to be more accurate (Yeung et al., 2006 ). Moreover, several inflammatory parameters such as C-reactive protein (CRP) and interleukin-6 were documented to be clinically relevant in the differentiation of mild and non-mild AP (Sternby et al., 2017 ). In 2008, a new, easy-to-implement bedside index of severity in acute pancreatitis (BISAP) was proposed for use within 24 h of hospitalization to predict in-hospital mortality (Wu et al., 2008 ). However, with the improvements of the imaging techniques, contrast-enhanced computed tomography (CT) gets an important place in the diagnosis of AP and its complications. In the early ‘90s, Balthazar and his coworkers developed a numerical scoring system, the CT severity index (CTSI), for the estimation of the severity of AP (Balthazar et al., 1990 ). It combines the quantification of pancreatic and peripancreatic inflammation with the extent of pancreatic parenchymal necrosis. In 2004, Mortele et al., formulated the modified CTSI (mCTSI) including a simplified evaluation of peripancreatic inflammation and extent of pancreatic parenchymal necrosis and incorporated the extrapancreatic complications (vascular, gastrointestinal, and extrapancreatic parenchymal complications as well as the presence of pleural effusion and/or ascites) in the assessment. This modified index correlated more with the outcome of AP (Mortele et al., 2004 ). Tables 1A,B show the components of CTSI and mCTSI. Components of the CTSI and mCTSI. It is still not clear, which scoring system has the highest predictive accuracy for severity and mortality of AP. The aim of this meta-analysis was to investigate, how accurate CT-based severity indices are in the prediction of the severity and mortality of AP in comparison with other widely accepted and used scoring systems.

ÉV and PH designed the research and the study concept. ÉV and AM performed the acquisition of data. PM and AS analyzed and interpreted the data. ÉV, AM, PM, AS, and PH wrote the article. AG, MB, BE, and LC supervised the study. BM, BE, JT, PS, and IR made a critical revision of the manuscript for important intellectual content. All of the authors gave their final approval of the version of the article to be published. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The systematic search was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement (Moher et al., 2009 ) ( Supplementary Document 1 ). We included studies on AP that used CTSI or mCTSI (index test) and compared them to any of the other scoring systems (Ranson, BISAP, APACHE II) or CRP (reference standards) in terms of their predictive value. A systematic search was performed in Pubmed, Embase, and the Cochrane Library (CENTRAL), using the following search query: “ acute pancreatitis” AND (“computed tomography severity index” OR CTSI OR “modified computed tomography severity index” OR MCTSI) . English language filter was used. Search key in Embase were: (‘acute pancreatitis’/exp OR ‘acute pancreatitis’) AND (ctsi OR mctsi OR ‘computed tomography severity index’/exp OR ‘computed tomography severity index’ OR ‘modified computed tomography severity index’) AND [english]/lim . The search was conducted on 11th March 2018. Duplicates were removed using the (EndNote X7.4, Clarivate Analytics, Philadelphia, PA, US) reference manager software. Two independent investigators (EV, AM) selected the studies, and disagreements were resolved by a third reviewer (PH). The records were selected for meta-analysis if (1) AP patients of any severity were enrolled consecutively; (2) if CTSI or mCTSI were used for the prediction of the severity or mortality of AP; and (3) if sensitivity and specificity values, the absolute numbers of true positive (TP), false negative (FN), false positive (FP) and true negative (TN), and/or area under the curve (AUC) were reported (for CTSI/mCTSI regarding AP severity and/or mortality). If other prognostic scores (Ranson, BISAP, APACHE II) or CRP values were also assessed in the selected articles, those results were extracted as well. Only full-text articles were included. Studies, which met the inclusion criteria were assessed for full-text evaluation. The following data were extracted from the articles: first author; year of the publication; study period; study design; the AP scoring systems used; evaluation time of the scores; the used definition for SAP; sample size based on severity; mean age; male/female ratio; cut-off value, clinical end-points, and several data about the statistical analysis were reviewed for the risk of bias assessment. To construct 2 × 2 contingency tables, true positive, false positive, false negative, and true negative values were abstracted. These served as input to fit Hierarchial Summary Receiver Operating Characteristics (HSROC) curves and estimate summary sensitivity, specificity, and diagnostic odds ratio (DOR) with 95% confidence intervals (CI). For each method and outcome, we collected the AUC values and their CIs as well and performed a meta-analysis using the random effect model to gain pooled AUC estimates with 95% CI. The statistical analysis was performed with Stata 14 software using the metandi module 1 . Heterogeneity was assessed using the I 2 measure and the corresponding chi 2 test, p < 0.1 indicates significant heterogeneity. Based on the Cochrane Handbook, I 2 = 100% × (Q–df)/Q represents the magnitude of the heterogeneity (moderate: 30–60%, substantial: 50–90%, considerable: 75–100%) (Higgins and Green, 2011 ). The Prediction model Risk Of Bias ASsessment Tool (PROBAST) (Wolff et al., 2019 ) was used to assess the risk of bias and applicability of primary studies in accordance with the recommendation of the Cochrane Collaboration. This tool is able to assess the risk of bias based on the following four domains: participants, predictors, outcome, and analysis. It includes also concerns regarding applicability in three domains: participants, predictors and outcome.

The systematic search strategy identified 319 articles after removal of duplicates. The search provided 123 articles in Pubmed, 279 studies in Embase and 25 articles in Cochrane library databases. The flow chart ( Figure 1 ) shows the study selection process: 148 articles were screened by abstract and after excluding 86 studies, 62 full-text articles were assessed for eligibility. A further 32 articles were excluded, because of the lack of reporting of the outcome of interest or inappropriate patient population. In our meta-analysis 30 articles were included (Simchuk et al., 2000 ; Chatzicostas et al., 2003 ; Gurleyik et al., 2005 ; Vriens et al., 2005 ; Huang et al., 2010 ; Papachristou et al., 2010 ; Bollen et al., 2011 , 2012 ; Fabre et al., 2012 ; Lautz et al., 2012 ; Jakchairoongruang and Arjhansiri, 2013 ; Khanna et al., 2013 ; Park et al., 2013 ; Xu et al., 2014 ; Banday et al., 2015 ; Cho et al., 2015 ; Qiu et al., 2015 ; Sharma et al., 2015 ; Yue et al., 2015 ; Alper et al., 2016 ; Hashimoto et al., 2016 ; Lee et al., 2016 ; Raghuwanshi et al., 2016 ; Yadav et al., 2016 ; Yang et al., 2016 ; Zhao et al., 2016 ; Biberoglu et al., 2017 ; Kumar et al., 2017 ; Sahu et al., 2017 ; Fei et al., 2018 ), which contained the data of 5,988 patients. The main characteristics of the included studies are shown in Table 2 . Flowchart of study selection. Basic characteristics of the included studies. CTSI, computed tomography severity index; MCTSI, modified computed tomography severity index; BISAP, bedside index of severity in acute pancreatitis; APACHE II, Acute Physiology And Chronic Health Examination II; N/A, not applicable; mean; median; standard difference; standard error of mean; range . From the 30 articles, 11 contained data on AUC for the prediction of mortality ( Figure 2 ). Table 3 summarizes the study numbers, sample sizes, AUC, and heterogeneity data of the different severity scores based on the outcome of AP. For CTSI based on data from 10 articles, the pooled AUC for mortality was 0.79 (CI 0.73–0.86; heterogeneity I 2 = 83%, p < 0.001). Eight articles included AUC data for mortality for BISAP, the pooled AUC was 0.87 (CI 0.83–0.90; heterogeneity I 2 = 0%, p = 0.578). The pooled AUC for mCTSI was 0.80 (CI 0.72–0.89; heterogeneity I 2 = 79.4%, p = 0.001) according to five studies. Only two studies reported AUC data predicting mortality for CRP level, and the pooled AUC was 0.73 (CI 0.66–0.81; heterogeneity I 2 = 0%, p = 0.708) Six articles included AUC data for mortality for Ranson score with a pooled AUC of 0.87 (CI 0.81–0.92; heterogeneity I 2 = 65.6%, p = 0.013) and also for APACHE II score with a pooled AUC of 0.91 (CI 0.88–0.93; heterogeneity I 2 = 4.8%, p = 0.386). Area under the curve (AUC) summarizing the predictive performance of scoring systems regarding mortality in acute pancreatitis. Size of squares for effect size reflects weight of studies in pooled analysis. Horizontal bars represent 95% confidence intervals (CI). CTSI, computed tomography severity index; BISAP, bedside index of severity in acute pancreatitis; mCTSI, modified computed tomography severity index; CRP, C-reactive protein; APACHE II, Acute Physiology And Chronic Health Examination II. The vertical line represents the line of no effect. Summary table of mortality and severity data based on the forest plots. Based on the above results of the meta-analytical calculations the APACHE II scoring system had significantly higher predictive accuracy for mortality than CTSI or CRP level ( p = 0.001; p < 0.001, respectively). However, CTSI was not different from the BISAP, mCTSI, CRP or Ranson criteria in the prediction of mortality of AP, and these scores can be classified as good and fair. AUC data for severity were included in 19 studies ( Figure 3 ). Area under the curve (AUC) summarizing the predictive performance of scoring systems regarding severity in acute pancreatitis. Size of squares for effect size reflects weight of trial in pooled analysis. Horizontal bars represent 95% confidence intervals (CI). CTSI, computed tomography severity index; BISAP, bedside index of severity in acute pancreatitis; mCTSI, modified computed tomography severity index; CRP, C-reactive protein; APACHE II, Acute Physiology And Chronic Health Examination II. The vertical line represents the line of no effect. The pooled AUC for severity based on data from 18 articles was 0.80 (CI 0.76–0.85) for CTSI with high heterogeneity (I 2 = 86.2%; p < 0.001). The pooled AUC for BISAP was 0.79 (CI 0.72–0.86, heterogeneity I 2 = 89.7%, p < 0.001). The pooled AUC for mCTSI from three studies was 0.83 (CI 0.75–0.91) with substantial heterogeneity ( I 2 = 68.1%; p = 0.043). The pooled AUC was 0.73 (CI 0.64–0.83) for CRP level (heterogeneity: I 2 = 77%, p = 0.001), 0.81 (CI 0.75–0.87) for Ranson score (heterogeneity I 2 = 87.5%, p < 0.001), and 0.80 (CI 0.77–0.83) for APACHE II score (heterogeneity I 2 = 36.8%, p = 0.105). There was no statistical difference between the severity predicting values of the different scoring systems. The heterogeneity values were I 2 = 86.2%, p < 0.001; I 2 = 89.7%, p < 0.001; I 2 = 68.1%, p = 0.043; I 2 = 77%, p = 0.001; I 2 = 87.5%, p < 0.001; I 2 = 36.8%, p = 0.105 for CTSI, BISAP, mCTSI, CRP, Ranson, and APACHE II scores, respectively. The heterogeneity across the studies was significant in all scoring systems or predicting values, except for the APACHE II score. Based on the results of the meta-analytical calculations, the severity prediction values of the included scoring systems are not different. Figure 4A shows the application of CTSI in the form of HSROC curves for mortality [sensitivity: 0.88 (CI 0.69–0.97); specificity: 0.61 (CI 0.52–0.70); DOR: 12.84 (CI 4.19–39.41)] and Figure 4B for severity, respectively [sensitivity: 0.81 (CI 0.73–0.87); specificity: 0.82 (CI 0.73–0.88); DOR: 19.1 (CI 10.29–35.45)]. Figure 5A illustrates the bivariate HSROC curve of BISAP score for mortality [sensitivity: 0.88 (CI 0.71–0.96); specificity: 0.77 (CI 0.70–0.83); DOR: 24.74 (CI 9.44–64.81)] and Figure 5B for severity, respectively [sensitivity: 0.73 (CI 0.53–0.87); specificity: 0.80 (CI 0.72–0.88); DOR: 11.71 (CI 4.49–30.61)]. Figure 6A shows the application of mCTSI in the form of HSROC curves for mortality [sensitivity: 0.95 (CI 0.76–0.99); specificity: 0.36 (CI 0.16–0.63); DOR: 10.32 (CI 2.11–50.53)] and Figure 6B for severity, respectively [sensitivity: 0.88 (CI 0.47–0.98); specificity: 0.80 (CI 0.56–0.92); DOR: 29.07 (CI 3.36–251.91)]. Figure 7 illustrates the bivariate HSROC curve of CRP score for severity [sensitivity: 0.71 (CI 0.59–0.81); specificity: 0.87 (CI 0.66–0.96); DOR: 16.75 (CI 3.49–80.48)]. Figure 8A illustrates the bivariate HSROC curve of Ranson score for mortality [sensitivity: 0.91 (CI 0.70–0.98); specificity: 0.72 (CI 0.66–0.79); DOR: 28.72 (CI 7.57–109.05)] and Figure 8B for severity, respectively [sensitivity: 0.79 (CI 0.69–0.86); specificity: 0.78 (CI 0.71–0.84); DOR: 13.32 (CI 7.33–24.24)]. Figure 9A illustrates the bivariate HSROC curve of APACHE II score for mortality [sensitivity: 0.92 (CI 0.70–0.98); specificity: 0.79 (CI 0.66–0.88); DOR: 45.08 (CI 11.4–178.2)] and Figure 9B for severity, respectively [sensitivity: 0.71 (CI 0.60–0.79); specificity: 0.80 (CI 0.71–0.88); DOR: 9.94 (CI 6.45–15.30)]. (A) Hierarchial summary receiver operating characteristic curves (HSROC) for computed tomography severity index (CTSI) for predicting mortality of acute pancreatitis. (B) HSROC for CTSI for predicting severity of acute pancreatitis. (A) Hierarchial summary receiver operating characteristic curves (HSROC) for bedside index of severity in acute pancreatitis (BISAP) for predicting mortality. (B) HSROC for BISAP for predicting severity of acute pancreatitis. (A) Hierarchial summary receiver operating characteristic curves (HSROC) for modified computed tomography severity index (mCTSI) for predicting mortality of acute pancreatitis. (B) HSROC for mCTSI for predicting severity of acute pancreatitis. Hierarchial summary receiver operating characteristic curves (HSROC) for C-reactive protein (CRP) for predicting severity of acute pancreatitis. (A) Hierarchial summary receiver operating characteristic curves (HSROC) for Ranson score for predicting mortality of acute pancreatitis. (B) HSROC for Ranson score for predicting severity of acute pancreatitis. (A) Hierarchial summary receiver operating characteristic curves (HSROC) for Acute Physiology And Chronic Health Examination II (APACHE II) score for predicting mortality of acute pancreatitis. (B) HSROC for APACHE II score for predicting severity of acute pancreatitis. Sensitivity, specificity, and DOR data of all scores predicting mortality and severity are summarized in Table 4 . In summary, the sensitivity for the prediction of mortality of AP was the highest for mCTSI, Ranson, and APACHE II scores. While the specificity for prediction of mortality was the best for APACHE II, BISAP, and Ranson scores. The sensitivity for the prediction of severity of CTSI, mCTSI, and Ranson scores were the highest, while the specificity for prediction of severity were the highest for CRP and CTSI. Summary table of mortality and severity data based on the HSROC curves. In this study, the two main outcomes were mortality and severity of the disease, therefore, we included the data in two different tables. The majority of the studies included in our analysis met the predefined criteria of the definition of AP and contained all grades of severities, therefore the risk of bias regarding the included populations was deemed as low. The data on CTSI from all studies were significantly limited by the timeframe the CT was done from either the admission or the onset of the symptoms and the diagnosis of AP. Therefore, this is the main limitation of the applicability of our results on CTSI. The result of the risk of bias and applicability assessment is showed in Tables 5A , B . The PROBAST Tool for mortality. The PROBAST Tool for severity. red, high risk of bias or concern; yellow, unclear risk of bias or concern; green, low risk of bias or concern; +, high risk of bias or concern; ?, unclear risk of bias or concern; −, low risk of bias or concern .

Severe acute pancreatitis is a serious state with high mortality and it requires high costs of the health care system. By more accurate prediction of the severity on admission, the risk of mortality can be reduced with the immediate optimal therapy. Acute pancreatitis is diagnosed on the basis of the presence of two or more of the following three criteria: abdominal pain consistent with the diagnosis elevated pancreatic enzymes to a level of more than three times the upper normal value, and characteristic findings on abdominal imaging. Different radiological modalities (ultrasound, CT) are not only necessary to make the diagnosis of AP, but by the visualization of the gallbladder and biliary tract, they can reveal its etiology as biliary or non-biliary. Furthermore, by using morphological scoring systems e.g., CTSI or mCTSI, obtaining a CT scan can be useful for assessing the severity of AP. This is the first meta-analysis, which quantifies the accuracy of CTSI and mCTSI scores for the prediction of the severity and mortality of AP, and compares them with other commonly used scoring systems. Two previous meta-analyses (Gao et al., 2015 ; Yang and Li, 2016 ) assessed the predictive accuracy of the BISAP score, but these articles did not contain CTSI nor mCTSI. Yang and Li ( 2016 ) found that the pooled sensitivity and specificity of the BISAP for the prediction of SAP were 0.65 (CI: 0.54–0.74) and 0.84 (CI: 0.70–0.92), respectively, the pooled AUC was 0.77 (CI: 0.73–0.80). Gao et al. ( 2015 ) calculated the pooled sensitivity as 0.51 (CI: 0.43–0.60), the specificity as 0.91 (CI: 0.89–0.92), the AUC was 0.87. Based on our results, we calculated 0.73 (CI: 0.53–0.87) for sensitivity, 0.80 (CI: 0.72–0.88) for specificity, and our pooled AUC was 0.79. The results are similar, the difference in specificity between our results and those of Gao et al. ( 2015 ) may be explained by the higher numbers of articles included in our analysis. In our meta-analysis, APACHE II proved to be the most accurate scoring system for the prediction of mortality. It is the most widely used mortality prediction score in critically ill patients, however, it contains 12 points, including numerous clinical parameters, hence its application can be cumbersome and it limits its widespread use. In addition, APACHE II is designed for patients admitted to the intensive care unit, therefore it is not suitable for the early prediction of severity of AP. The AUC's of BISAP, mCTSI, and Ranson scores overlapped with APACHE II, while those of CTSI and CRP were mildly weaker. Computed tomography severity index is accurate to predict severity, and its accuracy did not differ from the other scoring systems. However, the Ranson, APACHE II, and BISAP scores include several clinical parameters. There is a good correlation between morphological severity according to CT scoring systems and clinical scoring systems using clinical data and laboratory parameters. The most recent guidelines of AP recommend a CT scan 72–96 h after the onset of the symptoms (Working Group IAP/APA Acute Pancreatitis Guidelines., 2013 ; Hritz et al., 2015 ), because pancreatic parenchymal necrosis in contrast-enhanced CT rarely appears within 48 h (Ryu, 2009 ). The guidelines allow an earlier CT scan in case of diagnostic uncertainty. However, the contrast-enhanced CT examination cannot always be performed in every patient. In extreme obese patients, body weight, and size preclude the CT investigation. The contrast-enhanced CT assessment requires an intravenous injection of iodinated contrast medium for the detection of hypoperfused areas in pancreas parenchyma, therefore intravenous contrast media allergy, impaired renal function, and hyperthyroidism are contraindications. Because of the risk of radiation exposure, repeated CT scans should be avoided, and should be reserved for patients who fail to improve clinically. CT examination had shown an advantage in evaluation of local complications (Ju et al., 2006 ), which can modify therapeutical strategy. Contrary to the evaluation of Ranson, BISAP, and APACHE II scores, contrast-enhanced CT assessment, and CTSI calculation require radiological expertise. Three of the articles contained data of children (Fabre et al., 2012 ; Lautz et al., 2012 ; Hashimoto et al., 2016 ), in 2 from these articles the DeBanto score was used for evaluating the severity of AP, which is a specific score for pediatric pancreatitis. The AP population of the studies is not necessarily representative for the whole AP population, because the CT scan is mostly performed in the more severe cases. In several studies not all etiologies of AP were included, Yang et al. included only patients with hyperlipidemic etiology while Alper et his coworkers included only biliary AP cases. Fifteen of the included studies were retrospectively designed, and these might have caused selection bias. The time of CTSI and mCTSI was not the same in the studies, in several studies it included a longer delay, while in others it was carried out on admission, leading to higher heterogeneity. While APACHE II, Ranson, BISAP and CRP values were established mostly on admission or within the first 48 h, the optimal timing of the CT examination 72–96 h after the onset of the symptoms and in several studies it was performed later than the other prognostic scores. This can limitate the prognostic score of the CTSI because the other scores can predict earlier severity or mortality with similar accuracy. It is also a good question if the radiologist can judge the time point of necrosis development. In populations with previous necrotizing pancreatitis, the severity cannot be accurately assessed. For the value of predicting mortality, a considerable heterogeneity for CTSI and a substantial heterogeneity for Ranson score can be observed, while for the severity predicting value of AP a considerable heterogeneity for CTSI, BISAP, CRP, and Ranson scores and substantial heterogeneity for mCTSI can be noticed. We suspect that the confounder factors, that cause high heterogeneity among the studies are because of different population in terms of ethnicity, BMI, age (etc.), and etiology, the different timing and interpretation of imaging modalities, and potential inter observer variability between the radiologists interpreting the CT images. Because of the long delay characterizing the studies, the severity was assessed according to several Atlanta classifications and definitions.

In the prediction of mortality in AP, CTSI was revealed as equally valuable as BISAP, mCTSI, CRP, or Ranson score, only APACHE II score overcame its predicting ability. Considering severity, there was no difference in the prediction value of the scores. If CT scans are performed, CTSI and mCTSI can be easily calculated and should be used in addition to the other scoring systems. Further research is warranted for the assessment of the effect of early CT and its predictive value in AP.

The datasets generated for this study are available on request to the corresponding author.