Towards Reliable Measurement of Cerebellar Morphology : A comparative assessment of segmentation pipelines

Towards Reliable Measurement of Cerebellar Morphology : A comparative assessment of segmentation pipelines | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Short Report Towards Reliable Measurement of Cerebellar Morphology : A comparative assessment of segmentation pipelines Katia Chardon, Ting Xu, Marie Chupin, Edouard Duchesnay, Davide Boido, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8724066/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 9 You are reading this latest preprint version Abstract Background. Characterizing cerebellar morphology is fundamental for accurately mapping its structure and function across individuals, yet remains challenging due to its densely foliated architecture. Although multiple automated segmentation pipelines exist, the measurement reproducibility of these tools has not been benchmarked. Methods. We conducted a systematic assessment of robustness for cerebellar morphology estimates using four commonly used pipelines: one classic parcellation method (CERES), two deep-learning methods (ACAPULCO, DeepCERES), and one voxel-based morphometry toolbox (SUIT). Leveraging the HNU test-retest dataset, which provides MRI scans for ten timepoints per individual over a month, we evaluated the test-retest reliability for each of four pipelines using ReX, an integrative tool for quantifying and optimizing measurement reliability and individual differences. We quantified intra- and inter-individual variability, as well as the Intraclass Correlation Coefficient (ICC), for each pipeline at both global and region-of-interest levels. Results. Overall, all pipelines yielded highly reliable segmentation volumes (ICC > 0.8). Across pipelines, DeepCERES demonstrated the strongest performance, exhibiting high inter-individual consistency and low intra-individual variability. Importantly, our analysis highlighted substantial heterogeneity in reliability across lobules for each method. Lobule X consistently showed reduced reliability whereas lobules I-V were reliably estimated across all pipelines. Conclusion. Our work evaluated the robustness of cerebellar segmentation pipelines. While DeepCERES offers a robust global performance, substantial lobule-specific variability underscores the need for reliability-aware pipeline selection to optimize morphology estimation in research. Cerebellum Test-Retest Reliability Magnetic Resonance Imaging Segmentation Figures Figure 1 Figure 2 Figure 3 Introduction The cerebellum is crucial for motor and cognitive functions. It plays an essential role in various neurological and psychiatric disorders [ 1 , 2 ]. Its complex anatomy, characterized by a foliated structure, presents significant challenges for accurate segmentation. The lobules exhibit pronounced variability with distinct differences in size among lobules within a single cerebellum, asymmetries between hemispheres, and considerable inter-individual variation [ 3 , 4 ]. The cerebellum's involvement in conditions such as schizophrenia, where volume reductions are observed [ 5 – 8 ], underscores the importance of precise parcellation for clinical studies. Moreover, increasing evidence suggests that cerebellar dysfunction contributes to the pathophysiology of schizophrenia, particularly to its cognitive and negative symptoms, making the cerebellum a promising therapeutic target for non-invasive brain stimulation. Accurate segmentation is crucial for identifying specific subregions affected by disorders and for comparing results across studies using different regions of interest. Manual segmentation by experts is time-consuming and tedious, which has led to the development of automatic methods. With the rise of artificial intelligence and increased data availability, numerous automatic segmentation techniques have emerged. These methods include both atlas-based approaches, such as SUIT [ 4 , 9 , 10 ] and CERES [ 11 ], and deep learning-based methods, such as ACAPULCO [ 12 ] and DeepCERES [ 13 ]. These methods are typically evaluated with classic image segmentation metrics [ 14 ]. Their test-retest reliability, which is crucial for longitudinal and individual difference studies, has not yet been comprehensively compared. A previous study [ 15 ] assessed the test-retest reliability of cerebellar morphometry but was limited to only two runs per subject, which may not fully capture the variability in segmentation performance for robust ICC estimation. While they evaluated three different datasets, all used the same MRI sequence, potentially limiting generalizability across acquisition protocols. Our work extends this effort by providing a detailed analysis of reliability based on multiple metrics, including the Intraclass Correlation Coefficient (ICC) and both between- and within-subject variation, assessed at global and lobule-specific levels. Additionally, we evaluate a broader range of segmentation approaches, including recent deep learning-based methods and a voxel-based morphometry (VBM) approach, and we quantify segmentation failure rates across methods. Finally, the dataset used contains 10 runs per subject acquired over a month with acquisition parameters that differ from prior work. Together, these analyses offer new insights into the challenges and limitations of current cerebellar segmentation techniques, paving the way for more reliable morphometric studies and clinical applications. Materials and Methods MRI Data High-resolution T1-weighted anatomical images were obtained from the Hangzhou Normal University open dataset of the Consortium for Reliability and Reproducibility (HNU) [ 16 ]. The data were acquired using a GE Discovery MR750 3.0T MR system (RRID:SCR_025460) with an 8-channel head coil and a 3D SPGR sequence. The imaging parameters were as follows : TR = 8.06 ms, TE = minimum full, flip angle = 8°, TI = 450 ms, voxel size = 1 mm isotropic, FOV = 250 × 250 × 180 mm. The dataset includes ten sessions of scans conducted over the course of a month from 30 young healthy adults (15 females, 15 males, between 20 and 30 years old, mean age 24 ± 2.4 years) with no history of neurological or psychiatric disorders, head injuries, or substance abuse. The study was approved by the ethics committee of the Center for Cognition and Brain Disorders at Hangzhou Normal University, and all participants provided written consent before data collection. The dataset was downloaded from the International Neuroimaging Data-Sharing Initiative. Parcellation Methods SUIT and ACAPULCO were run on the high-performance computer cluster at NeuroSpin, CEA, Université Paris-Saclay, France. CERES and DeepCERES were run through the online MRI Brain Volumetry System volBrain [ 17 ]. Systematic visual quality control was performed on each method's outputs and only results without major segmentation errors were kept for further analyses; this made it possible to compute the failure rate for each method. Each method produced different segmentation outputs. For example, some methods segmented the vermis and hemispheres separately, while others grouped anterior lobules differently (e.g., I-III, I-IV, or I-V). To allow for consistent comparisons across methods, we selected a set of common regions: lobules I-V, VI, Crus I, Crus II, VIIB, VIIIA, VIIIB, IX, and X, separately for the left and right hemispheres. For methods that provided combined segmentations of the vermis and hemispheres, we used these combined outputs. For methods that outputted separate segmentations for the vermis and hemispheres, we only used the hemisphere segmentations. For the anterior lobules, we combined lobules I to V. SUIT The SUIT (Spatially Unbiased Infratentorial Template) toolbox is an SPM (RRID:SCR_007037) based software in MATLAB (The MathWorks Inc., RRID:SCR_001622) environment that uses a 1-mm isotropic atlas template and probabilistic atlas of the cerebellum. The template was created with 20 T1-weighted images of healthy subjects (11 females, 9 males, mean age 27.25 years) and the probabilistic atlas by averaging the T1-weighted images of 20 healthy individuals (10 females, 10 males, mean age 23.7 years). The parcellation process begins with the automatic isolation of cerebellar and brainstem structures from the cerebral cortex. The pipeline then achieves anatomical normalization of these structures into the SUIT atlas space using the DARTEL algorithm [ 18 ]. Following normalization, the pipeline uses the SUIT probabilistic atlas of cerebellar anatomy to assign locations to different cerebellar lobules (Fig. 1 d). The pipeline outputs probabilistic maps of gray matter content. We ran SUIT with MATLAB and SPM12 as described on the SUIT website ( https://www.diedrichsenlab.org/imaging/suit_function.htm ) and used the VBM outputs to compute the mean gray-matter density of each region using Python (RRID:SCR_008394) and FSL (RRID:SCR_002823) [ 19 ]. In VBM, grey matter density represents an approximation of local grey matter volume; for simplicity, we will refer to this measure as grey matter volumetry throughout the remainder of the article. CERES CERES (CEREbellum Segmentation) is a multi-atlas patch-based segmentation framework [ 20 , 21 ] with a non-local label fusion technique. It uses a library of 0.6-mm isotropic T1-weighted images (reconstructed to 0.3-mm isotropic voxels using ZIP filters) manually labeled from 5 healthy individuals (3 females, 2 males, aged 29–57 years) [ 22 ]. The pipeline begins with a preprocessing including denoising [ 23 , 24 ], N4 bias field correction [ 25 ] in the native space, linear registration with affine transform to the 1-mm isotropic MNI152 space [ 26 , 27 ], N4 correction in the MNI space, cerebellar cropping, non-linear registration to the cropped MNI152 atlas, and intensity normalization. Segmentation is then carried out via non-local patch-based label fusion, accelerated by the Optimized PatchMatch (OPAL) algorithm [ 28 , 29 ] and the volumes are computed (Fig. 1 b). ACAPULCO ACAPULCO (Automatic Cerebellum Anatomical Parcellation using U-Net with Locally Constrained Optimization) is a convolutional neural network-based method processing T1-weighted images, preferably acquired with an MPRAGE sequence. The pipeline begins by estimating a brain mask with ROBEX (RRID:SCR_002534) [ 30 ] which is then used for N4 bias field correction. Next, the images are rigidly registered to the 1-mm isotropic ICBM 2009c nonlinear symmetric template. The cerebellum of the MNI-registered image is then parcellated with two three-dimensional convolutional neural networks. First, a locating network is used to predict a bounding box around the cerebellum. The cerebellum is then cropped using this bounding box, and a parcellating network is used to parcellate the cerebellum inside the bounding box. The parcellation is finally transformed back into the original image space using nearest-neighbor interpolation and the volume of each parcellated region is calculated (Fig. 1 c). The neural networks were trained on a cohort containing 15 adult subjects of which 6 are healthy controls and 9 have cerebellum atrophy [ 31 ]. We first cropped all the images with the robustfov command provided by FSL to remove the neck, as suggested by the developers. We then ran ACAPULCO with Singularity. DeepCERES DeepCERES is based on deep convolutional architectures with multi‑atlas priors using ultra‑high resolution multimodal MRI. It was trained on an ultra‑high resolution dataset (0.7-mm isotropic) of the Human Connectome Project (HCP) database [ 32 ] consisting of T1 and T2-weighted images semi‑automatically labeled from 75 healthy subjects (41 females, 34 males, aged 22–36 years). A second dataset consisting of 4857 T1-weighted MRI subjects from early infancy to old age was used for data augmentation [ 13 ]. DeepCERES uses a novel architecture based on Deep Pyramidal Networks (DPN) and a classical U‑Net network with prior spatial knowledge from multi‑atlas segmentation methods. The pipeline includes denoising [ 23 ], registration to the 1-mm isotropic MNI152 space, N4 bias correction, intensity normalization [ 33 , 17 ], brain extraction [ 34 ], a second N4 bias correction, super-resolution of the T1-weighted images to 0.125-mm 3 [ 35 ], crop of the cerebellar region, atlas generation, T2-weighted images synthesis [ 36 ], segmentation and volume extraction (Fig. 1 a). CerebNet CerebNet [ 37 ] is a deep-learning approach leveraging the 2.5D FastSurfer approach [ 38 ]. The pipeline begins with the localization of the cerebellum, where a bounding box is computed using FastSurfer. Subsequently, three independent 2D U-Net-based fully convolutional networks are applied in the axial, coronal, and sagittal orientations to generate label probability maps. These probabilistic segmentations are then combined through view aggregation to produce the final 3D parcellation. CerebNet was trained on a cohort of 30 subjects consisting of 20 spinocerebellar ataxia type 3 mutation carriers and 10 age-matched healthy controls (16 females, 14 males, aged 20–63 years), using 1-mm isotropic T1-weighted MRI scans. However, CerebNet could not be included in our analysis because it consistently failed on the HNU dataset (see Supplementary Fig. 1). Statistical Analyses We used the Reliability eXplorer (ReX) toolbox [ 39 ], an open-source method for assessing measurement reliability and individual variation. ReX is an integrative framework for quantifying intra-subject variability (within-subject measurement error), inter-subject variability (true between-subject differences, adjusted for measurement error), and the ICC. The ICC reflects the proportion of variance attributable to true differences between individuals rather than differences due to noise (within individuals) [ 40 ]. Here we used the two-way mixed-effects, absolute agreement, single measurement model, ICC(A,1) or ICC(2,1) [ 41 – 43 ], which is appropriate for test-retest reliability [ 44 ]. For each segmentation method, we extracted the volumes of all cerebellar lobules, normalized them by the intracranial volume (ICV) to account for head size, and scaled them across lobules to reduce bias from lobular size difference. All statistical analyses were performed to assess differences between segmentation methods across the metrics. For each metric, the normality of the data distribution was evaluated using the Shapiro-Wilk test. When the normality assumption was met, we applied a repeated-measures ANOVA to test for overall differences between methods, followed by pairwise paired t-tests for post hoc comparisons. To control for multiple comparisons, False Discovery Rate (FDR) correction was applied, with a significance threshold of α = 0.05. When the normality assumption was violated, we used the non-parametric Friedman test to assess global differences between methods, followed by Nemenyi post hoc tests for pairwise comparisons. Notice, this test has an integrated multiple-comparison correction. All statistical tests were two-sided, and results were considered significant for p < 0.05. Results Data Selection Among the 30 subjects of HNU, only 22 had a good coverage of the cerebellum (see Supplementary Fig. 2 for examples of poor cerebellar coverage). From this group, we included only subjects whose ten sessions showed less than 10% segmentation error for each lobule, manually estimated with ITK-SNAP (RRID:SCR_017341 ) [ 45 ] (see Supplementary Fig. 3 for examples of poor segmentations) and selected nine subjects (2 females, 7 males, mean age 26 ± 2.7 years). We then computed the segmentation failure rate for each method, except SUIT, which is a VBM method. The failure rates among all datasets with good cerebellar coverage are 31.8% for ACAPULCO, 14.1% for CERES and 5.9% for DeepCERES (see complete statistics in Supplementary Table 1). Statistical Analysis Intraclass Correlation Coefficient Analysis The ICCs of the cerebellar pipelines were, on average, above 0.8 for all methods (Fig. 2 , 3 a). DeepCERES demonstrated the best performance with the highest mean ICC of 0.895, followed by CERES with 0.87, SUIT with 0.869, and ACAPULCO with 0.82. A Friedman test showed significant differences between the methods ( χ² = 9.27, p = 0.026). Nemenyi post-hoc tests resulted in a significant difference only between DeepCERES and ACAPULCO ( p = 0.023). All other comparisons were not statistically significant (Supplementary Table 2). For a more comprehensive view, Supplementary Table 3 presents ICC and within/between-subject variation values computed using all subjects with no failures for each method. These values are not directly comparable between methods due to the different number of subjects. Between-Subject Variation Analysis DeepCERES showed the highest between-subject variation with a mean of 0.069, followed by CERES and SUIT with means of 0.062 and 0.061, respectively. ACAPULCO showed the lowest between-subject variation with a mean of 0.047 (Fig. 3 b). A repeated-measures ANOVA showed significant differences between the methods ( F (3,51) = 4.41, p = 0.008). The comparisons between DeepCERES and ACAPULCO, CERES and ACAPULCO and SUIT and ACAPULCO remained significant with post hoc paired t -tests after correction (respectively p corrected = 0.009, p corrected = 0.03 and p corrected = 0.009). All other comparisons were not statistically significant after correction (Supplementary Table 4). Within-Subject Variation Analysis DeepCERES exhibited the lowest within-subject variation with a mean of 0.0053, followed by CERES with 0.0067, SUIT with 0.0073, and ACAPULCO with 0.0082 (Fig. 3 c). A Friedman test showed significant differences between the methods ( χ² = 9.6, p = 0.022). Nemenyi post-hoc tests showed that the comparisons between DeepCERES and ACAPULCO and CERES and ACAPULCO were significant (both with p = 0.048). All other comparisons were not statistically significant after correction (Supplementary Table 5). Lobule-Level Analysis At the lobule level, most methods demonstrated good test-retest reliability, generally exceeding 0.8 (Fig. 3 d). However, there were notable differences across the methods for certain lobules. Lobule X exhibited the lowest reliability independently of the method. The most reliable lobules overall were I.V and IX. DeepCERES and SUIT were the most reliable for lobules VIIB and VIIIA, DeepCERES and CERES were the most reliable for Crus I and all three methods performed best for lobule VI and Crus II. DeepCERES, while being the best overall, showed some baseline reliability issues for VIIIB compared to the other methods. (Fig. 3 d, 3 e, 3 f) Discussion In this study, we evaluated the test-retest reliability of four leading cerebellar segmentation pipelines: SUIT, CERES, ACAPULCO, and DeepCERES. Our findings demonstrate that all methods exhibited strong test-retest reliability. The trend across all considered metrics consistently shows DeepCERES as the best-performing method, followed by CERES and SUIT, which have similar performance. ACAPULCO exhibited the lowest reliability overall. Our analysis highlighted different reliability estimate for each method across different lobules. Additionally, some lobules are consistently less reliable across all methods. These results are consistent with the study by Sörös et al. [ 15 ], where CERES achieved better reliability than ACAPULCO. However, we obtained lower ICC values than their study, which may reflect differences in MRI sequences or the increased precision of our ICC estimates derived from ten runs per subject compared to their two-run design. Our results also correspond with the segmentation precision evaluation [ 13 ], where DeepCERES achieved the best Dice score and the lobules with the highest segmentation accuracy were lobule VI, Crus I and II. Our study presents several notable strengths. To our knowledge, it is only the second work to systematically assess the test-retest reliability of cerebellar segmentation, and the first to evaluate recently developed methods using a robust test-retest dataset with ten sessions per subject, enabling more precise ICC estimation. Segmentation quality was visually assessed for each subject and each session across segmentation methods by an expert rater (KC). Furthermore, by evaluating reliability metrics for each cerebellar lobule separately, we were able to identify region-specific differences in segmentation performance that are often overlooked in broader analyses. In addition to ICC, we analyzed both within-subject and between-subject variability, offering a more comprehensive assessment of segmentation consistency. Notably, the study was conducted entirely independently from the groups that developed the segmentation algorithms, using a dataset not involved in their training or testing. One limitation of this study is the relatively small and homogeneous sample. Given our focus on the cerebellum, some data were cropped, and the number of subjects was further decreased by failure rates, particularly with ACAPULCO, which had the highest failure rate. The homogeneity of the sample may not capture the full range of interindividual variability or all potential segmentation challenges across lobules. Future research should therefore include a more diverse cohort, encompassing different age groups and clinical populations, such as patients with cerebellar atrophy. This expansion would enhance the generalizability of the findings and their applicability to various clinical settings. It is also worth noting that the different methods do not produce the same segmentation outputs. For example, ACAPULCO segments the hemispheres and vermis separately, groups anterior lobules together, and includes white matter in lobule segmentation. These differences underscore the importance of selecting the appropriate method based on the specific research questions and anatomical features of interest. Furthermore, CerebNet, which was also compared in a previous article [ 13 ], could not be included in our analysis because it consistently failed on the HNU dataset. This suggests that the method may encounter difficulties when applied to specific image characteristics or acquisition protocols, highlighting the importance of validating cerebellar segmentation tools across multiple datasets. Conclusion This study provides a comprehensive assessment of the test-retest reliability of leading cerebellar segmentation pipelines. DeepCERES demonstrated superior reliability, offering high ICC values, high between-subject variation and low within-subject variation, with a significant difference from ACAPULCO across all metrics. However, while DeepCERES may be the best overall method, specific lobules such as lobules VIIIB and X might require alternative approaches for optimal segmentation. Researchers should consider these nuances when selecting a segmentation method for their studies and interpret results for lobules with lower segmentation reliability with caution. Declarations Acknowledgement None. Funding The Ph.D. fellowship of Katia Chardon has been funded by a NeuroSpin-CEA CFR grant. Competing interests All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. Ethics approval This study is a re-analysis of a previously published dataset. For the ethical aspects, refer to [16]. Consent to participate This study is a re-analysis of a previously published dataset. All the information about the involved subjects can be found in [16]. Code and Data Availability The used dataset is available through the Consortium for Reliability and Reproducibility [16] (https://fcon_1000.projects.nitrc.org/indi/CoRR/html/hnu_1.html). The codes to process the data and the ReX outputs are available at https://github.com/kchardon/reliability-cerebellar-volumetry. Authors’ Contributions Statement Conceptualization: Charles Laidi, Edouard Duchesnay; Methodology: Charles Laidi, Davide Boido, Katia Chardon; Formal analysis and investigation: Katia Chardon; Writing - original draft preparation: Katia Chardon; Writing - review and editing: Charles Laidi, Davide Boido, Katia Chardon, Edouard Duchesnay, Ting Xu, Marie Chupin. References Hoppenbrouwers SS, Schutter DJLG, Fitzgerald PB, Chen R, Daskalakis ZJ. The role of the cerebellum in the pathophysiology and treatment of neuropsychiatric disorders: A review. Brain Research Reviews. 2008 Nov;59(1):185–200. Available from: http://dx.doi.org/10.1016/j.brainresrev.2008.07.005 Phillips JR, Hewedi DH, Eissa AM, Moustafa AA. The Cerebellum and Psychiatric Disorders. Front Public Health. 2015 May 5;3. Available from: http://dx.doi.org/10.3389/fpubh.2015.00066 Steele CJ, Chakravarty MM. Gray-matter structural variability in the human cerebellum: Lobule-specific differences across sex and hemisphere. NeuroImage. 2018 Apr;170:164–73. Available from: http://dx.doi.org/10.1016/j.neuroimage.2017.04.066 Diedrichsen J, Balsters JH, Flavell J, Cussans E, Ramnani N. A probabilistic MR atlas of the human cerebellum. NeuroImage. 2009 May 15;46(1):39–46. Available from: http://dx.doi.org/10.1016/j.neuroimage.2009.01.045 Nopoulos PC, Ceilley JW, Gailis EA, Andreasen NC. An MRI study of cerebellar vermis morphology in patients with schizophrenia: evidence in support of the cognitive dysmetria concept. Biological Psychiatry. 1999 Sept;46(5):703–11. Available from: http://dx.doi.org/10.1016/s0006-3223(99)00093-1 Gur RE, Turetsky BI, Bilker WB, Gur RC. Reduced Gray Matter Volume in Schizophrenia. Arch Gen Psychiatry. 1999 Oct 1;56(10):905. Available from: http://dx.doi.org/10.1001/archpsyc.56.10.905 Laidi C, d’Albis M ‐A., Wessa M, Linke J, Phillips ML, Delavest M, et al. Cerebellar volume in schizophrenia and bipolar I disorder with and without psychotic features. Acta Psychiatr Scand. 2014 Nov 28;131(3):223–33. Available from: http://dx.doi.org/10.1111/acps.12363 Moberget T, Doan NT, Alnæs D, Kaufmann T, Córdova-Palomera A, et al. Cerebellar volume and cerebellocerebral structural covariance in schizophrenia: a multisite mega-analysis of 983 patients and 1349 healthy controls. Mol Psychiatry. 2017 May 16;23(6):1512–20. Available from: http://dx.doi.org/10.1038/mp.2017.106 Diedrichsen J. A spatially unbiased atlas template of the human cerebellum. NeuroImage. 2006 Oct;33(1):127–38. Available from: http://dx.doi.org/10.1016/j.neuroimage.2006.05.056 Diedrichsen J, Maderwald S, Küper M, Thürling M, Rabe K, Gizewski ER, et al. Imaging the deep cerebellar nuclei: A probabilistic atlas and normalization procedure. NeuroImage. 2011 Feb;54(3):1786–94. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.10.035 Romero JE, Coupé P, Giraud R, Ta VT, Fonov V, Park MTM, et al. CERES: A new cerebellum lobule segmentation method. NeuroImage. 2017 Feb;147:916–24. Available from: http://dx.doi.org/10.1016/j.neuroimage.2016.11.003 Han S, Carass A, He Y, Prince JL. Automatic cerebellum anatomical parcellation using U-Net with locally constrained optimization. NeuroImage. 2020 Sept;218:116819. Available from: http://dx.doi.org/10.1016/j.neuroimage.2020.116819 Morell-Ortega S, Ruiz-Perez M, Gadea M, Vivo-Hernando R, Rubio G, Aparici F, et al. DeepCERES: A deep learning method for cerebellar lobule segmentation using ultra-high resolution multimodal MRI. NeuroImage. 2025 Mar;308:121063. Available from: http://dx.doi.org/10.1016/j.neuroimage.2025.121063 Carass A, Cuzzocreo JL, Han S, Hernandez-Castillo CR, Rasser PE, Ganz M, et al. Comparing fully automated state-of-the-art cerebellum parcellation from magnetic resonance images. NeuroImage. 2018 Dec;183:150–72. Available from: http://dx.doi.org/10.1016/j.neuroimage.2018.08.003 Sörös P, Wölk L, Bantel C, Bräuer A, Klawonn F, Witt K. Replicability, Repeatability, and Long-term Reproducibility of Cerebellar Morphometry. Cerebellum. 2021 Jan 9;20(3):439–53. Available from: http://dx.doi.org/10.1007/s12311-020-01227-2 Zuo XN, Anderson JS, Bellec P, Birn RM, Biswal BB, Blautzik J, et al. An open science resource for establishing reliability and reproducibility in functional connectomics. Sci Data. 2014 Dec 9;1(1). Available from: http://dx.doi.org/10.1038/sdata.2014.49 Manjón JV, Coupé P. volBrain: An Online MRI Brain Volumetry System. Front Neuroinform. 2016 July 27;10. Available from: http://dx.doi.org/10.3389/fninf.2016.00030 Ashburner J. A fast diffeomorphic image registration algorithm. NeuroImage. 2007 Oct;38(1):95–113. Available from: http://dx.doi.org/10.1016/j.neuroimage.2007.07.007 Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TEJ, Johansen-Berg H, et al. Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage. 2004 Jan;23:S208–19. Available from: http://dx.doi.org/10.1016/j.neuroimage.2004.07.051 Coupé P, Manjón JV, Fonov V, Pruessner J, Robles M, Collins DL. Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation. NeuroImage. 2011 Jan;54(2):940–54. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.09.018 Rousseau F, Habas PA, Studholme C. A Supervised Patch-Based Approach for Human Brain Labeling. IEEE Trans Med Imaging. 2011 Oct;30(10):1852–62. Available from: http://dx.doi.org/10.1109/tmi.2011.2156806 Park MTM, Pipitone J, Baer LH, Winterburn JL, Shah Y, Chavez S, et al. Derivation of high-resolution MRI atlases of the human cerebellum at 3T and segmentation using multiple automatically generated templates. NeuroImage. 2014 July;95:217–31. Available from: http://dx.doi.org/10.1016/j.neuroimage.2014.03.037 Manjón JV, Coupé P, Martí‐Bonmatí L, Collins DL, Robles M. Adaptive non‐local means denoising of MR images with spatially varying noise levels. Magnetic Resonance Imaging. 2009 Dec 20;31(1):192–203. Available from: http://dx.doi.org/10.1002/jmri.22003 Manjón JV, Tohka J, Robles M. Improved estimates of partial volume coefficients from noisy brain MRI using spatial context. NeuroImage. 2010 Nov 1;53(2):480–90. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.06.046 Tustison NJ, Avants BB, Cook PA, Yuanjie Zheng, Egan A, Yushkevich PA, et al. N4ITK: Improved N3 Bias Correction. IEEE Trans Med Imaging. 2010 June;29(6):1310–20. Available from: http://dx.doi.org/10.1109/tmi.2010.2046908 Fonov V, Evans AC, Botteron K, Almli CR, McKinstry RC, Collins DL. Unbiased average age-appropriate atlases for pediatric studies. NeuroImage. 2011 Jan;54(1):313–27. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.07.033 Fonov V, Evans A, McKinstry R, Almli C, Collins D. Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage. 2009 July;47:S102. Available from: http://dx.doi.org/10.1016/s1053-8119(09)70884-5 Ta VT, Giraud R, Collins DL, Coupé P. Optimized PatchMatch for Near Real Time and Accurate Label Fusion. Lecture Notes in Computer Science. Springer International Publishing; 2014. p. 105–12. Available from: http://dx.doi.org/10.1007/978-3-319-10443-0_14 Giraud R, Ta VT, Papadakis N, Manjón JV, Collins DL, Coupé P. An Optimized PatchMatch for multi-scale and multi-feature label fusion. NeuroImage. 2016 Jan;124:770–82. Available from: http://dx.doi.org/10.1016/j.neuroimage.2015.07.076 Iglesias JE, Cheng-Yi Liu, Thompson PM, Zhuowen Tu. Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods. IEEE Trans Med Imaging. 2011 Sept;30(9):1617–34. Available from: http://dx.doi.org/10.1109/tmi.2011.2138152 Carass A, Cuzzocreo JL, Han S, Hernandez-Castillo CR, Rasser PE, Ganz M, et al. Comparing fully automated state-of-the-art cerebellum parcellation from magnetic resonance images. NeuroImage. 2018 Dec;183:150–72. Available from: http://dx.doi.org/10.1016/j.neuroimage.2018.08.003 Van Essen DC, Ugurbil K, Auerbach E, Barch D, Behrens TEJ, Bucholz R, et al. The Human Connectome Project: A data acquisition perspective. NeuroImage. 2012 Oct;62(4):2222–31. Available from: http://dx.doi.org/10.1016/j.neuroimage.2012.02.018 Manjón JV, Tohka J, García‐Martí G, Carbonell‐Caballero J, Lull JJ, Martí‐Bonmatí L, et al. Robust MRI brain tissue parameter estimation by multistage outlier rejection. Magnetic Resonance in Med. 2008 Mar 27;59(4):866–73. Available from: http://dx.doi.org/10.1002/mrm.21521 Manjón JV, Romero JE, Vivo-Hernando R, Rubio-Navarro G, De la Iglesia-Vaya M, Aparici-Robles F, et al. Deep ICE: A Deep learning approach for MRI Intracranial Cavity Extraction. arXiv; 2020. Available from: https://arxiv.org/abs/2001.05720 He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. arXiv; 2015. Available from: https://arxiv.org/abs/1512.03385 Manjón JV, Romero JE, Coupe P. Deep learning based MRI contrast synthesis using full volume prediction using full volume prediction. Biomed Phys Eng Express. 2021 Dec 3;8(1):015013. Available from: http://dx.doi.org/10.1088/2057-1976/ac3c64 Faber J, Kügler D, Bahrami E, Heinz LS, Timmann D, Ernst TM, et al. CerebNet: A fast and reliable deep-learning pipeline for detailed cerebellum sub-segmentation. NeuroImage. 2022 Dec;264:119703. Available from: http://dx.doi.org/10.1016/j.neuroimage.2022.119703 Henschel L, Conjeti S, Estrada S, Diers K, Fischl B, Reuter M. FastSurfer - A fast and accurate deep learning based neuroimaging pipeline. NeuroImage. 2020 Oct;219:117012. Available from: http://dx.doi.org/10.1016/j.neuroimage.2020.117012 Xu T, Kiar G, Cho JW, Bridgeford EW, Nikolaidis A, Vogelstein JT, et al. ReX: an integrative tool for quantifying and optimizing measurement reliability for the study of individual differences. Nat Methods. 2023 June 1;20(7):1025–8. Available from: http://dx.doi.org/10.1038/s41592-023-01901-3 Bartko JJ. The Intraclass Correlation Coefficient as a Measure of Reliability. Psychol Rep. 1966 Aug;19(1):3–11. Available from: http://dx.doi.org/10.2466/pr0.1966.19.1.3 McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996 Mar;1(1):30–46. Available from: http://dx.doi.org/10.1037/1082-989x.1.1.30 Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86(2):420–8. Available from: http://dx.doi.org/10.1037/0033-2909.86.2.420 Liljequist D, Elfving B, Skavberg Roaldsen K. Intraclass correlation – A discussion and demonstration of basic features. Chiacchio F, editor. PLoS ONE. 2019 July 22;14(7):e0219854. Available from: http://dx.doi.org/10.1371/journal.pone.0219854 Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine. 2016 June;15(2):155–63. Available from: http://dx.doi.org/10.1016/j.jcm.2016.02.012 Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., & Gerig, G. (2006). User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability. NeuroImage , 31 (3), 1116–1128. https://doi.org/10.1016/j.neuroimage.2006.01.015 Additional Declarations No competing interests reported. Supplementary Files CerebellumSegSUPMATChardonSUBMITTEDTOTHECEREBELLUM20260128.docx Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 25 Feb, 2026 Reviews received at journal 24 Feb, 2026 Reviews received at journal 17 Feb, 2026 Reviewers agreed at journal 04 Feb, 2026 Reviewers agreed at journal 30 Jan, 2026 Reviewers invited by journal 30 Jan, 2026 Editor assigned by journal 29 Jan, 2026 Submission checks completed at journal 29 Jan, 2026 First submitted to journal 28 Jan, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8724066","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Short Report","associatedPublications":[],"authors":[{"id":582894022,"identity":"506ba1cb-c6cd-4f4b-b64b-95e2a79f9bc5","order_by":0,"name":"Katia Chardon","email":"","orcid":"","institution":"BAOBAB, NeuroSpin, CEA, Université Paris-Saclay, CNRS","correspondingAuthor":false,"prefix":"","firstName":"Katia","middleName":"","lastName":"Chardon","suffix":""},{"id":582894023,"identity":"11b21195-e75b-43a8-90ea-59288f683ed8","order_by":1,"name":"Ting Xu","email":"","orcid":"","institution":"Child Mind Institute","correspondingAuthor":false,"prefix":"","firstName":"Ting","middleName":"","lastName":"Xu","suffix":""},{"id":582894024,"identity":"41508205-45b6-478d-8fa2-ac72d94f6214","order_by":2,"name":"Marie Chupin","email":"","orcid":"","institution":"Centre d'acquisition et de traitement des images (CATI), Sorbonne Université, CNRS, INSERM, CEA, AP-HP","correspondingAuthor":false,"prefix":"","firstName":"Marie","middleName":"","lastName":"Chupin","suffix":""},{"id":582894025,"identity":"b95e121e-fbf3-4261-a90e-8eca2fd68d07","order_by":3,"name":"Edouard Duchesnay","email":"","orcid":"","institution":"BAOBAB, NeuroSpin, CEA, Université Paris-Saclay, CNRS","correspondingAuthor":false,"prefix":"","firstName":"Edouard","middleName":"","lastName":"Duchesnay","suffix":""},{"id":582894026,"identity":"569540ea-2894-4fb4-8ffc-4fe047544dc4","order_by":4,"name":"Davide Boido","email":"","orcid":"","institution":"BAOBAB, NeuroSpin, CEA, Université Paris-Saclay, CNRS","correspondingAuthor":false,"prefix":"","firstName":"Davide","middleName":"","lastName":"Boido","suffix":""},{"id":582894027,"identity":"c8aeefb9-0f1b-4bc8-9640-22f019c7506f","order_by":5,"name":"Charles Laidi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYDACCSB+AMR8DIwNEgkVICHGB4S1JAAxG1jLGZAQswGxWoBsxjYitPDPbn72IaGijoGN/XDjjYfz7OT4G5jZPuC15M4x4xkJZw4zsPEkNlskbks2ljjAzDwDnxYDiQRjhsS2A0CHJbZJJG47kNhwgP8wXocZSKR/Zkj8B3QY/0OgljkH6ucDbSGgJQdoSwMzA5sEyJaGAwkGhLRI3MgpZkg4dpiHTeJhs0XCsWTDjYcJaOGfkb6Z4UNNnRw/f/rDmz9q7OTljjfj1wIDPAgmcRpGwSgYBaNgFOADANAjQjJO+FCvAAAAAElFTkSuQmCC","orcid":"","institution":"Institut Mondor de Recherche Biomédicale (IMRB), INSERM, Université Paris-Est Créteil","correspondingAuthor":true,"prefix":"","firstName":"Charles","middleName":"","lastName":"Laidi","suffix":""}],"badges":[],"createdAt":"2026-01-28 17:38:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8724066/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8724066/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101881395,"identity":"6d9e55e9-7e42-4d0d-95df-3ef0c65b654a","added_by":"auto","created_at":"2026-02-04 15:11:52","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":473713,"visible":true,"origin":"","legend":"\u003cp\u003eExamples of cerebellar segmentation with the DeepCERES pipeline (a), the CERES pipeline (b), the ACAPULCO pipeline (c) and the SUIT pipeline (d) - coronal view. Panels a, b and c are shown on anatomical data, while panel d is shown on a gray matter concentrattion map\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-8724066/v1/19f8f564f6a5ceeaffa25741.png"},{"id":101826635,"identity":"09e7f607-a046-470f-8e59-dd897f1a73de","added_by":"auto","created_at":"2026-02-04 05:17:27","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":95043,"visible":true,"origin":"","legend":"\u003cp\u003eKernel density estimate of between-subject variation and within-subject variation for each method. The filled contours indicate the density of data points. Dashed gray lines correspond to constant ICC values. The ICC of the different methods are, on average, superior to 0.8, indicating good reliability\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-8724066/v1/06a35ff40a3c4a9e5161ef0f.png"},{"id":101881567,"identity":"56b52728-619f-4e53-9172-e300e7470f00","added_by":"auto","created_at":"2026-02-04 15:13:18","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":135000,"visible":true,"origin":"","legend":"\u003cp\u003eICC, between-subject variation and within-subject variation for each method (a, b, c) and each lobule (d, e, f). In a, b and c, the dots represent each lobule, the central lines indicate mean values and the whiskers indicate the minimum and maximum values. The trend across all metrics consistently shows DeepCERES as the best-performing method, followed by CERES and SUIT, having similar performances and finally ACAPULCO (a, b, c). Lobule X exhibited the lowest reliability across parcellation methods and metrics. The most reliable lobules overall were I-V and IX. DeepCERES and SUIT were the most reliable for lobules VIIB and VIIIA, DeepCERES and CERES were the most reliable for Crus I and all three methods performed best for lobule VI and Crus II. DeepCERES showed some reliability issues for VIIIB compared to the other methods (d, e, f)\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-8724066/v1/5b7a6575bf33e279dc8932b4.png"},{"id":101883989,"identity":"3a66effd-3d11-4078-a16f-367ab95cfcc8","added_by":"auto","created_at":"2026-02-04 15:30:04","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1247235,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8724066/v1/d1a8b785-5654-48bc-9341-0dc628ca3cee.pdf"},{"id":101826637,"identity":"7a67e166-46bd-4397-96fe-de57c2fb018c","added_by":"auto","created_at":"2026-02-04 05:17:28","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":717991,"visible":true,"origin":"","legend":"","description":"","filename":"CerebellumSegSUPMATChardonSUBMITTEDTOTHECEREBELLUM20260128.docx","url":"https://assets-eu.researchsquare.com/files/rs-8724066/v1/9df8a827c4ff2948509bcf07.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Towards Reliable Measurement of Cerebellar Morphology : A comparative assessment of segmentation pipelines","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe cerebellum is crucial for motor and cognitive functions. It plays an essential role in various neurological and psychiatric disorders [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Its complex anatomy, characterized by a foliated structure, presents significant challenges for accurate segmentation. The lobules exhibit pronounced variability with distinct differences in size among lobules within a single cerebellum, asymmetries between hemispheres, and considerable inter-individual variation [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. The cerebellum's involvement in conditions such as schizophrenia, where volume reductions are observed [\u003cspan additionalcitationids=\"CR6 CR7\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e], underscores the importance of precise parcellation for clinical studies. Moreover, increasing evidence suggests that cerebellar dysfunction contributes to the pathophysiology of schizophrenia, particularly to its cognitive and negative symptoms, making the cerebellum a promising therapeutic target for non-invasive brain stimulation. Accurate segmentation is crucial for identifying specific subregions affected by disorders and for comparing results across studies using different regions of interest.\u003c/p\u003e \u003cp\u003eManual segmentation by experts is time-consuming and tedious, which has led to the development of automatic methods. With the rise of artificial intelligence and increased data availability, numerous automatic segmentation techniques have emerged. These methods include both atlas-based approaches, such as SUIT [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] and CERES [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e], and deep learning-based methods, such as ACAPULCO [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] and DeepCERES [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. These methods are typically evaluated with classic image segmentation metrics [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. Their test-retest reliability, which is crucial for longitudinal and individual difference studies, has not yet been comprehensively compared. A previous study [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] assessed the test-retest reliability of cerebellar morphometry but was limited to only two runs per subject, which may not fully capture the variability in segmentation performance for robust ICC estimation. While they evaluated three different datasets, all used the same MRI sequence, potentially limiting generalizability across acquisition protocols. Our work extends this effort by providing a detailed analysis of reliability based on multiple metrics, including the Intraclass Correlation Coefficient (ICC) and both between- and within-subject variation, assessed at global and lobule-specific levels. Additionally, we evaluate a broader range of segmentation approaches, including recent deep learning-based methods and a voxel-based morphometry (VBM) approach, and we quantify segmentation failure rates across methods. Finally, the dataset used contains 10 runs per subject acquired over a month with acquisition parameters that differ from prior work. Together, these analyses offer new insights into the challenges and limitations of current cerebellar segmentation techniques, paving the way for more reliable morphometric studies and clinical applications.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eMRI Data\u003c/h2\u003e \u003cp\u003eHigh-resolution T1-weighted anatomical images were obtained from the Hangzhou Normal University open dataset of the Consortium for Reliability and Reproducibility (HNU) [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. The data were acquired using a GE Discovery MR750 3.0T MR system (RRID:SCR_025460) with an 8-channel head coil and a 3D SPGR sequence. The imaging parameters were as follows : TR\u0026thinsp;=\u0026thinsp;8.06 ms, TE\u0026thinsp;=\u0026thinsp;minimum full, flip angle\u0026thinsp;=\u0026thinsp;8\u0026deg;, TI\u0026thinsp;=\u0026thinsp;450 ms, voxel size\u0026thinsp;=\u0026thinsp;1 mm isotropic, FOV\u0026thinsp;=\u0026thinsp;250 \u0026times; 250 \u0026times; 180 mm. The dataset includes ten sessions of scans conducted over the course of a month from 30 young healthy adults (15 females, 15 males, between 20 and 30 years old, mean age 24\u0026thinsp;\u0026plusmn;\u0026thinsp;2.4 years) with no history of neurological or psychiatric disorders, head injuries, or substance abuse. The study was approved by the ethics committee of the Center for Cognition and Brain Disorders at Hangzhou Normal University, and all participants provided written consent before data collection. The dataset was downloaded from the International Neuroimaging Data-Sharing Initiative.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eParcellation Methods\u003c/h3\u003e\n\u003cp\u003eSUIT and ACAPULCO were run on the high-performance computer cluster at NeuroSpin, CEA, Universit\u0026eacute; Paris-Saclay, France. CERES and DeepCERES were run through the online MRI Brain Volumetry System volBrain [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Systematic visual quality control was performed on each method's outputs and only results without major segmentation errors were kept for further analyses; this made it possible to compute the failure rate for each method.\u003c/p\u003e \u003cp\u003eEach method produced different segmentation outputs. For example, some methods segmented the vermis and hemispheres separately, while others grouped anterior lobules differently (e.g., I-III, I-IV, or I-V). To allow for consistent comparisons across methods, we selected a set of common regions: lobules I-V, VI, Crus I, Crus II, VIIB, VIIIA, VIIIB, IX, and X, separately for the left and right hemispheres. For methods that provided combined segmentations of the vermis and hemispheres, we used these combined outputs. For methods that outputted separate segmentations for the vermis and hemispheres, we only used the hemisphere segmentations. For the anterior lobules, we combined lobules I to V.\u003c/p\u003e\n\u003ch3\u003eSUIT\u003c/h3\u003e\n\u003cp\u003eThe SUIT (Spatially Unbiased Infratentorial Template) toolbox is an SPM (RRID:SCR_007037) based software in MATLAB (The MathWorks Inc., RRID:SCR_001622) environment that uses a 1-mm isotropic atlas template and probabilistic atlas of the cerebellum. The template was created with 20 T1-weighted images of healthy subjects (11 females, 9 males, mean age 27.25 years) and the probabilistic atlas by averaging the T1-weighted images of 20 healthy individuals (10 females, 10 males, mean age 23.7 years). The parcellation process begins with the automatic isolation of cerebellar and brainstem structures from the cerebral cortex. The pipeline then achieves anatomical normalization of these structures into the SUIT atlas space using the DARTEL algorithm [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. Following normalization, the pipeline uses the SUIT probabilistic atlas of cerebellar anatomy to assign locations to different cerebellar lobules (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ed). The pipeline outputs probabilistic maps of gray matter content. We ran SUIT with MATLAB and SPM12 as described on the SUIT website (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.diedrichsenlab.org/imaging/suit_function.htm\u003c/span\u003e\u003cspan address=\"https://www.diedrichsenlab.org/imaging/suit_function.htm\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e) and used the VBM outputs to compute the mean gray-matter density of each region using Python (RRID:SCR_008394) and FSL (RRID:SCR_002823) [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. In VBM, grey matter density represents an approximation of local grey matter volume; for simplicity, we will refer to this measure as grey matter volumetry throughout the remainder of the article.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e\n\u003ch3\u003eCERES\u003c/h3\u003e\n\u003cp\u003eCERES (CEREbellum Segmentation) is a multi-atlas patch-based segmentation framework [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] with a non-local label fusion technique. It uses a library of 0.6-mm isotropic T1-weighted images (reconstructed to 0.3-mm isotropic voxels using ZIP filters) manually labeled from 5 healthy individuals (3 females, 2 males, aged 29\u0026ndash;57 years) [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. The pipeline begins with a preprocessing including denoising [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], N4 bias field correction [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] in the native space, linear registration with affine transform to the 1-mm isotropic MNI152 space [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e], N4 correction in the MNI space, cerebellar cropping, non-linear registration to the cropped MNI152 atlas, and intensity normalization. Segmentation is then carried out via non-local patch-based label fusion, accelerated by the Optimized PatchMatch (OPAL) algorithm [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] and the volumes are computed (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb).\u003c/p\u003e\n\u003ch3\u003eACAPULCO\u003c/h3\u003e\n\u003cp\u003eACAPULCO (Automatic Cerebellum Anatomical Parcellation using U-Net with Locally Constrained Optimization) is a convolutional neural network-based method processing T1-weighted images, preferably acquired with an MPRAGE sequence. The pipeline begins by estimating a brain mask with ROBEX (RRID:SCR_002534) [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] which is then used for N4 bias field correction. Next, the images are rigidly registered to the 1-mm isotropic ICBM 2009c nonlinear symmetric template. The cerebellum of the MNI-registered image is then parcellated with two three-dimensional convolutional neural networks. First, a locating network is used to predict a bounding box around the cerebellum. The cerebellum is then cropped using this bounding box, and a parcellating network is used to parcellate the cerebellum inside the bounding box. The parcellation is finally transformed back into the original image space using nearest-neighbor interpolation and the volume of each parcellated region is calculated (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ec). The neural networks were trained on a cohort containing 15 adult subjects of which 6 are healthy controls and 9 have cerebellum atrophy [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWe first cropped all the images with the \u003cem\u003erobustfov\u003c/em\u003e command provided by FSL to remove the neck, as suggested by the developers. We then ran ACAPULCO with Singularity.\u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eDeepCERES\u003c/h2\u003e \u003cp\u003eDeepCERES is based on deep convolutional architectures with multi‑atlas priors using ultra‑high resolution multimodal MRI. It was trained on an ultra‑high resolution dataset (0.7-mm isotropic) of the Human Connectome Project (HCP) database [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] consisting of T1 and T2-weighted images semi‑automatically labeled from 75 healthy subjects (41 females, 34 males, aged 22\u0026ndash;36 years). A second dataset consisting of 4857 T1-weighted MRI subjects from early infancy to old age was used for data augmentation [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. DeepCERES uses a novel architecture based on Deep Pyramidal Networks (DPN) and a classical U‑Net network with prior spatial knowledge from multi‑atlas segmentation methods. The pipeline includes denoising [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], registration to the 1-mm isotropic MNI152 space, N4 bias correction, intensity normalization [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e], brain extraction [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e], a second N4 bias correction, super-resolution of the T1-weighted images to 0.125-mm\u003csup\u003e3\u003c/sup\u003e [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e], crop of the cerebellar region, atlas generation, T2-weighted images synthesis [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e], segmentation and volume extraction (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea).\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eCerebNet\u003c/h3\u003e\n\u003cp\u003eCerebNet [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] is a deep-learning approach leveraging the 2.5D FastSurfer approach [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. The pipeline begins with the localization of the cerebellum, where a bounding box is computed using FastSurfer. Subsequently, three independent 2D U-Net-based fully convolutional networks are applied in the axial, coronal, and sagittal orientations to generate label probability maps. These probabilistic segmentations are then combined through view aggregation to produce the final 3D parcellation. CerebNet was trained on a cohort of 30 subjects consisting of 20 spinocerebellar ataxia type 3 mutation carriers and 10 age-matched healthy controls (16 females, 14 males, aged 20\u0026ndash;63 years), using 1-mm isotropic T1-weighted MRI scans.\u003c/p\u003e \u003cp\u003eHowever, CerebNet could not be included in our analysis because it consistently failed on the HNU dataset (see Supplementary Fig.\u0026nbsp;1).\u003c/p\u003e\n\u003ch3\u003eStatistical Analyses\u003c/h3\u003e\n\u003cp\u003eWe used the Reliability eXplorer (ReX) toolbox [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e], an open-source method for assessing measurement reliability and individual variation. ReX is an integrative framework for quantifying intra-subject variability (within-subject measurement error), inter-subject variability (true between-subject differences, adjusted for measurement error), and the ICC. The ICC reflects the proportion of variance attributable to true differences between individuals rather than differences due to noise (within individuals) [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Here we used the two-way mixed-effects, absolute agreement, single measurement model, ICC(A,1) or ICC(2,1) [\u003cspan additionalcitationids=\"CR42\" citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e], which is appropriate for test-retest reliability [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFor each segmentation method, we extracted the volumes of all cerebellar lobules, normalized them by the intracranial volume (ICV) to account for head size, and scaled them across lobules to reduce bias from lobular size difference.\u003c/p\u003e \u003cp\u003eAll statistical analyses were performed to assess differences between segmentation methods across the metrics. For each metric, the normality of the data distribution was evaluated using the Shapiro-Wilk test. When the normality assumption was met, we applied a repeated-measures ANOVA to test for overall differences between methods, followed by pairwise paired t-tests for post hoc comparisons. To control for multiple comparisons, False Discovery Rate (FDR) correction was applied, with a significance threshold of α\u0026thinsp;=\u0026thinsp;0.05.\u003c/p\u003e \u003cp\u003eWhen the normality assumption was violated, we used the non-parametric Friedman test to assess global differences between methods, followed by Nemenyi post hoc tests for pairwise comparisons. Notice, this test has an integrated multiple-comparison correction.\u003c/p\u003e \u003cp\u003eAll statistical tests were two-sided, and results were considered significant for p\u0026thinsp;\u0026lt;\u0026thinsp;0.05.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eData Selection\u003c/h2\u003e \u003cp\u003eAmong the 30 subjects of HNU, only 22 had a good coverage of the cerebellum (see Supplementary Fig.\u0026nbsp;2 for examples of poor cerebellar coverage). From this group, we included only subjects whose ten sessions showed less than 10% segmentation error for each lobule, manually estimated with ITK-SNAP (RRID:SCR_017341 ) [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e] (see Supplementary Fig.\u0026nbsp;3 for examples of poor segmentations) and selected nine subjects (2 females, 7 males, mean age 26\u0026thinsp;\u0026plusmn;\u0026thinsp;2.7 years). We then computed the segmentation failure rate for each method, except SUIT, which is a VBM method. The failure rates among all datasets with good cerebellar coverage are 31.8% for ACAPULCO, 14.1% for CERES and 5.9% for DeepCERES (see complete statistics in Supplementary Table\u0026nbsp;1).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003eStatistical Analysis\u003c/h2\u003e \u003cdiv id=\"Sec14\" class=\"Section3\"\u003e \u003ch2\u003eIntraclass Correlation Coefficient Analysis\u003c/h2\u003e \u003cp\u003eThe ICCs of the cerebellar pipelines were, on average, above 0.8 for all methods (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea). DeepCERES demonstrated the best performance with the highest mean ICC of 0.895, followed by CERES with 0.87, SUIT with 0.869, and ACAPULCO with 0.82.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eA Friedman test showed significant differences between the methods (\u003cem\u003eχ\u0026sup2;\u003c/em\u003e = 9.27, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.026). Nemenyi post-hoc tests resulted in a significant difference only between DeepCERES and ACAPULCO (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.023). All other comparisons were not statistically significant (Supplementary Table\u0026nbsp;2).\u003c/p\u003e \u003cp\u003eFor a more comprehensive view, Supplementary Table\u0026nbsp;3 presents ICC and within/between-subject variation values computed using all subjects with no failures for each method. These values are not directly comparable between methods due to the different number of subjects.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003eBetween-Subject Variation Analysis\u003c/h2\u003e \u003cp\u003eDeepCERES showed the highest between-subject variation with a mean of 0.069, followed by CERES and SUIT with means of 0.062 and 0.061, respectively. ACAPULCO showed the lowest between-subject variation with a mean of 0.047 (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb).\u003c/p\u003e \u003cp\u003eA repeated-measures ANOVA showed significant differences between the methods (\u003cem\u003eF\u003c/em\u003e(3,51)\u0026thinsp;=\u0026thinsp;4.41, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.008). The comparisons between DeepCERES and ACAPULCO, CERES and ACAPULCO and SUIT and ACAPULCO remained significant with post hoc paired \u003cem\u003et\u003c/em\u003e-tests after correction (respectively \u003cem\u003ep\u003c/em\u003e\u003csub\u003ecorrected\u003c/sub\u003e = 0.009, \u003cem\u003ep\u003c/em\u003e\u003csub\u003ecorrected\u003c/sub\u003e = 0.03 and \u003cem\u003ep\u003c/em\u003e\u003csub\u003ecorrected\u003c/sub\u003e = 0.009). All other comparisons were not statistically significant after correction (Supplementary Table\u0026nbsp;4).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003eWithin-Subject Variation Analysis\u003c/h2\u003e \u003cp\u003eDeepCERES exhibited the lowest within-subject variation with a mean of 0.0053, followed by CERES with 0.0067, SUIT with 0.0073, and ACAPULCO with 0.0082 (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec).\u003c/p\u003e \u003cp\u003eA Friedman test showed significant differences between the methods (\u003cem\u003eχ\u0026sup2;\u003c/em\u003e = 9.6, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.022). Nemenyi post-hoc tests showed that the comparisons between DeepCERES and ACAPULCO and CERES and ACAPULCO were significant (both with \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.048). All other comparisons were not statistically significant after correction (Supplementary Table\u0026nbsp;5).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section2\"\u003e \u003ch2\u003eLobule-Level Analysis\u003c/h2\u003e \u003cp\u003eAt the lobule level, most methods demonstrated good test-retest reliability, generally exceeding 0.8 (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed). However, there were notable differences across the methods for certain lobules. Lobule X exhibited the lowest reliability independently of the method. The most reliable lobules overall were I.V and IX. DeepCERES and SUIT were the most reliable for lobules VIIB and VIIIA, DeepCERES and CERES were the most reliable for Crus I and all three methods performed best for lobule VI and Crus II. DeepCERES, while being the best overall, showed some baseline reliability issues for VIIIB compared to the other methods. (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed, \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ee, \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ef)\u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn this study, we evaluated the test-retest reliability of four leading cerebellar segmentation pipelines: SUIT, CERES, ACAPULCO, and DeepCERES. Our findings demonstrate that all methods exhibited strong test-retest reliability. The trend across all considered metrics consistently shows DeepCERES as the best-performing method, followed by CERES and SUIT, which have similar performance. ACAPULCO exhibited the lowest reliability overall. Our analysis highlighted different reliability estimate for each method across different lobules. Additionally, some lobules are consistently less reliable across all methods.\u003c/p\u003e \u003cp\u003eThese results are consistent with the study by S\u0026ouml;r\u0026ouml;s et al. [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], where CERES achieved better reliability than ACAPULCO. However, we obtained lower ICC values than their study, which may reflect differences in MRI sequences or the increased precision of our ICC estimates derived from ten runs per subject compared to their two-run design. Our results also correspond with the segmentation precision evaluation [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e], where DeepCERES achieved the best Dice score and the lobules with the highest segmentation accuracy were lobule VI, Crus I and II.\u003c/p\u003e \u003cp\u003eOur study presents several notable strengths. To our knowledge, it is only the second work to systematically assess the test-retest reliability of cerebellar segmentation, and the first to evaluate recently developed methods using a robust test-retest dataset with ten sessions per subject, enabling more precise ICC estimation. Segmentation quality was visually assessed for each subject and each session across segmentation methods by an expert rater (KC). Furthermore, by evaluating reliability metrics for each cerebellar lobule separately, we were able to identify region-specific differences in segmentation performance that are often overlooked in broader analyses. In addition to ICC, we analyzed both within-subject and between-subject variability, offering a more comprehensive assessment of segmentation consistency. Notably, the study was conducted entirely independently from the groups that developed the segmentation algorithms, using a dataset not involved in their training or testing.\u003c/p\u003e \u003cp\u003eOne limitation of this study is the relatively small and homogeneous sample. Given our focus on the cerebellum, some data were cropped, and the number of subjects was further decreased by failure rates, particularly with ACAPULCO, which had the highest failure rate. The homogeneity of the sample may not capture the full range of interindividual variability or all potential segmentation challenges across lobules. Future research should therefore include a more diverse cohort, encompassing different age groups and clinical populations, such as patients with cerebellar atrophy. This expansion would enhance the generalizability of the findings and their applicability to various clinical settings. It is also worth noting that the different methods do not produce the same segmentation outputs. For example, ACAPULCO segments the hemispheres and vermis separately, groups anterior lobules together, and includes white matter in lobule segmentation. These differences underscore the importance of selecting the appropriate method based on the specific research questions and anatomical features of interest. Furthermore, CerebNet, which was also compared in a previous article [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e], could not be included in our analysis because it consistently failed on the HNU dataset. This suggests that the method may encounter difficulties when applied to specific image characteristics or acquisition protocols, highlighting the importance of validating cerebellar segmentation tools across multiple datasets.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study provides a comprehensive assessment of the test-retest reliability of leading cerebellar segmentation pipelines. DeepCERES demonstrated superior reliability, offering high ICC values, high between-subject variation and low within-subject variation, with a significant difference from ACAPULCO across all metrics. However, while DeepCERES may be the best overall method, specific lobules such as lobules VIIIB and X might require alternative approaches for optimal segmentation. Researchers should consider these nuances when selecting a segmentation method for their studies and interpret results for lobules with lower segmentation reliability with caution.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNone.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Ph.D. fellowship of Katia Chardon has been funded by a NeuroSpin-CEA CFR grant.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study is a re-analysis of a previously published dataset. For the ethical aspects, refer to [16].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study is a re-analysis of a previously published dataset. All the information about the involved subjects can be found in [16].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCode and Data Availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe used dataset is available through the Consortium for Reliability and Reproducibility [16] (https://fcon_1000.projects.nitrc.org/indi/CoRR/html/hnu_1.html).\u003c/p\u003e\n\u003cp\u003eThe codes to process the data and the ReX outputs are available at https://github.com/kchardon/reliability-cerebellar-volumetry.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026rsquo; Contributions Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceptualization: Charles Laidi, Edouard Duchesnay; Methodology: Charles Laidi, Davide Boido, Katia Chardon; Formal analysis and investigation: Katia Chardon; Writing - original draft preparation: Katia Chardon; Writing - review and editing: Charles Laidi, Davide Boido, Katia Chardon, Edouard Duchesnay, Ting Xu, Marie Chupin.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eHoppenbrouwers SS, Schutter DJLG, Fitzgerald PB, Chen R, Daskalakis ZJ. The role of the cerebellum in the pathophysiology and treatment of neuropsychiatric disorders: A review. Brain Research Reviews. 2008 Nov;59(1):185\u0026ndash;200. Available from: http://dx.doi.org/10.1016/j.brainresrev.2008.07.005\u003c/li\u003e\n\u003cli\u003ePhillips JR, Hewedi DH, Eissa AM, Moustafa AA. The Cerebellum and Psychiatric Disorders. Front Public Health. 2015 May 5;3. Available from: http://dx.doi.org/10.3389/fpubh.2015.00066\u003c/li\u003e\n\u003cli\u003eSteele CJ, Chakravarty MM. Gray-matter structural variability in the human cerebellum: Lobule-specific differences across sex and hemisphere. NeuroImage. 2018 Apr;170:164\u0026ndash;73. Available from: http://dx.doi.org/10.1016/j.neuroimage.2017.04.066\u003c/li\u003e\n\u003cli\u003eDiedrichsen J, Balsters JH, Flavell J, Cussans E, Ramnani N. A probabilistic MR atlas of the human cerebellum. NeuroImage. 2009 May 15;46(1):39\u0026ndash;46. Available from: http://dx.doi.org/10.1016/j.neuroimage.2009.01.045\u003c/li\u003e\n\u003cli\u003eNopoulos PC, Ceilley JW, Gailis EA, Andreasen NC. An MRI study of cerebellar vermis morphology in patients with schizophrenia: evidence in support of the cognitive dysmetria concept. Biological Psychiatry. 1999 Sept;46(5):703\u0026ndash;11. Available from: http://dx.doi.org/10.1016/s0006-3223(99)00093-1\u003c/li\u003e\n\u003cli\u003eGur RE, Turetsky BI, Bilker WB, Gur RC. Reduced Gray Matter Volume in Schizophrenia. Arch Gen Psychiatry. 1999 Oct 1;56(10):905. Available from: http://dx.doi.org/10.1001/archpsyc.56.10.905\u003c/li\u003e\n\u003cli\u003eLaidi C, d\u0026rsquo;Albis M ‐A., Wessa M, Linke J, Phillips ML, Delavest M, et al. Cerebellar volume in schizophrenia and bipolar I disorder with and without psychotic features. Acta Psychiatr Scand. 2014 Nov 28;131(3):223\u0026ndash;33. Available from: http://dx.doi.org/10.1111/acps.12363\u003c/li\u003e\n\u003cli\u003eMoberget T, Doan NT, Aln\u0026aelig;s D, Kaufmann T, C\u0026oacute;rdova-Palomera A, et al. Cerebellar volume and cerebellocerebral structural covariance in schizophrenia: a multisite mega-analysis of 983 patients and 1349 healthy controls. Mol Psychiatry. 2017 May 16;23(6):1512\u0026ndash;20. Available from: http://dx.doi.org/10.1038/mp.2017.106\u003c/li\u003e\n\u003cli\u003eDiedrichsen J. A spatially unbiased atlas template of the human cerebellum. NeuroImage. 2006 Oct;33(1):127\u0026ndash;38. Available from: http://dx.doi.org/10.1016/j.neuroimage.2006.05.056\u003c/li\u003e\n\u003cli\u003eDiedrichsen J, Maderwald S, K\u0026uuml;per M, Th\u0026uuml;rling M, Rabe K, Gizewski ER, et al. Imaging the deep cerebellar nuclei: A probabilistic atlas and normalization procedure. NeuroImage. 2011 Feb;54(3):1786\u0026ndash;94. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.10.035\u003c/li\u003e\n\u003cli\u003eRomero JE, Coup\u0026eacute; P, Giraud R, Ta VT, Fonov V, Park MTM, et al. CERES: A new cerebellum lobule segmentation method. NeuroImage. 2017 Feb;147:916\u0026ndash;24. Available from: http://dx.doi.org/10.1016/j.neuroimage.2016.11.003\u003c/li\u003e\n\u003cli\u003eHan S, Carass A, He Y, Prince JL. Automatic cerebellum anatomical parcellation using U-Net with locally constrained optimization. NeuroImage. 2020 Sept;218:116819. Available from: http://dx.doi.org/10.1016/j.neuroimage.2020.116819\u003c/li\u003e\n\u003cli\u003eMorell-Ortega S, Ruiz-Perez M, Gadea M, Vivo-Hernando R, Rubio G, Aparici F, et al. DeepCERES: A deep learning method for cerebellar lobule segmentation using ultra-high resolution multimodal MRI. NeuroImage. 2025 Mar;308:121063. Available from: http://dx.doi.org/10.1016/j.neuroimage.2025.121063\u003c/li\u003e\n\u003cli\u003eCarass A, Cuzzocreo JL, Han S, Hernandez-Castillo CR, Rasser PE, Ganz M, et al. Comparing fully automated state-of-the-art cerebellum parcellation from magnetic resonance images. NeuroImage. 2018 Dec;183:150\u0026ndash;72. Available from: http://dx.doi.org/10.1016/j.neuroimage.2018.08.003\u003c/li\u003e\n\u003cli\u003eS\u0026ouml;r\u0026ouml;s P, W\u0026ouml;lk L, Bantel C, Br\u0026auml;uer A, Klawonn F, Witt K. Replicability, Repeatability, and Long-term Reproducibility of Cerebellar Morphometry. Cerebellum. 2021 Jan 9;20(3):439\u0026ndash;53. Available from: http://dx.doi.org/10.1007/s12311-020-01227-2\u003c/li\u003e\n\u003cli\u003eZuo XN, Anderson JS, Bellec P, Birn RM, Biswal BB, Blautzik J, et al. An open science resource for establishing reliability and reproducibility in functional connectomics. Sci Data. 2014 Dec 9;1(1). Available from: http://dx.doi.org/10.1038/sdata.2014.49\u003c/li\u003e\n\u003cli\u003eManj\u0026oacute;n JV, Coup\u0026eacute; P. volBrain: An Online MRI Brain Volumetry System. Front Neuroinform. 2016 July 27;10. Available from: http://dx.doi.org/10.3389/fninf.2016.00030\u003c/li\u003e\n\u003cli\u003eAshburner J. A fast diffeomorphic image registration algorithm. NeuroImage. 2007 Oct;38(1):95\u0026ndash;113. Available from: http://dx.doi.org/10.1016/j.neuroimage.2007.07.007\u003c/li\u003e\n\u003cli\u003eSmith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TEJ, Johansen-Berg H, et al. Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage. 2004 Jan;23:S208\u0026ndash;19. Available from: http://dx.doi.org/10.1016/j.neuroimage.2004.07.051\u003c/li\u003e\n\u003cli\u003eCoup\u0026eacute; P, Manj\u0026oacute;n JV, Fonov V, Pruessner J, Robles M, Collins DL. Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation. NeuroImage. 2011 Jan;54(2):940\u0026ndash;54. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.09.018\u003c/li\u003e\n\u003cli\u003eRousseau F, Habas PA, Studholme C. A Supervised Patch-Based Approach for Human Brain Labeling. IEEE Trans Med Imaging. 2011 Oct;30(10):1852\u0026ndash;62. Available from: http://dx.doi.org/10.1109/tmi.2011.2156806\u003c/li\u003e\n\u003cli\u003ePark MTM, Pipitone J, Baer LH, Winterburn JL, Shah Y, Chavez S, et al. Derivation of high-resolution MRI atlases of the human cerebellum at 3T and segmentation using multiple automatically generated templates. NeuroImage. 2014 July;95:217\u0026ndash;31. Available from: http://dx.doi.org/10.1016/j.neuroimage.2014.03.037\u003c/li\u003e\n\u003cli\u003eManj\u0026oacute;n JV, Coup\u0026eacute; P, Mart\u0026iacute;‐Bonmat\u0026iacute; L, Collins DL, Robles M. Adaptive non‐local means denoising of MR images with spatially varying noise levels. Magnetic Resonance Imaging. 2009 Dec 20;31(1):192\u0026ndash;203. Available from: http://dx.doi.org/10.1002/jmri.22003\u003c/li\u003e\n\u003cli\u003eManj\u0026oacute;n JV, Tohka J, Robles M. Improved estimates of partial volume coefficients from noisy brain MRI using spatial context. NeuroImage. 2010 Nov 1;53(2):480\u0026ndash;90. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.06.046\u003c/li\u003e\n\u003cli\u003eTustison NJ, Avants BB, Cook PA, Yuanjie Zheng, Egan A, Yushkevich PA, et al. N4ITK: Improved N3 Bias Correction. IEEE Trans Med Imaging. 2010 June;29(6):1310\u0026ndash;20. Available from: http://dx.doi.org/10.1109/tmi.2010.2046908\u003c/li\u003e\n\u003cli\u003eFonov V, Evans AC, Botteron K, Almli CR, McKinstry RC, Collins DL. Unbiased average age-appropriate atlases for pediatric studies. NeuroImage. 2011 Jan;54(1):313\u0026ndash;27. Available from: http://dx.doi.org/10.1016/j.neuroimage.2010.07.033\u003c/li\u003e\n\u003cli\u003eFonov V, Evans A, McKinstry R, Almli C, Collins D. Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage. 2009 July;47:S102. Available from: http://dx.doi.org/10.1016/s1053-8119(09)70884-5\u003c/li\u003e\n\u003cli\u003eTa VT, Giraud R, Collins DL, Coup\u0026eacute; P. Optimized PatchMatch for Near Real Time and Accurate Label Fusion. Lecture Notes in Computer Science. Springer International Publishing; 2014. p. 105\u0026ndash;12. Available from: http://dx.doi.org/10.1007/978-3-319-10443-0_14\u003c/li\u003e\n\u003cli\u003eGiraud R, Ta VT, Papadakis N, Manj\u0026oacute;n JV, Collins DL, Coup\u0026eacute; P. An Optimized PatchMatch for multi-scale and multi-feature label fusion. NeuroImage. 2016 Jan;124:770\u0026ndash;82. Available from: http://dx.doi.org/10.1016/j.neuroimage.2015.07.076\u003c/li\u003e\n\u003cli\u003eIglesias JE, Cheng-Yi Liu, Thompson PM, Zhuowen Tu. Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods. IEEE Trans Med Imaging. 2011 Sept;30(9):1617\u0026ndash;34. Available from: http://dx.doi.org/10.1109/tmi.2011.2138152\u003c/li\u003e\n\u003cli\u003eCarass A, Cuzzocreo JL, Han S, Hernandez-Castillo CR, Rasser PE, Ganz M, et al. Comparing fully automated state-of-the-art cerebellum parcellation from magnetic resonance images. NeuroImage. 2018 Dec;183:150\u0026ndash;72. Available from: http://dx.doi.org/10.1016/j.neuroimage.2018.08.003\u003c/li\u003e\n\u003cli\u003eVan Essen DC, Ugurbil K, Auerbach E, Barch D, Behrens TEJ, Bucholz R, et al. The Human Connectome Project: A data acquisition perspective. NeuroImage. 2012 Oct;62(4):2222\u0026ndash;31. Available from: http://dx.doi.org/10.1016/j.neuroimage.2012.02.018\u003c/li\u003e\n\u003cli\u003eManj\u0026oacute;n JV, Tohka J, Garc\u0026iacute;a‐Mart\u0026iacute; G, Carbonell‐Caballero J, Lull JJ, Mart\u0026iacute;‐Bonmat\u0026iacute; L, et al. Robust MRI brain tissue parameter estimation by multistage outlier rejection. Magnetic Resonance in Med. 2008 Mar 27;59(4):866\u0026ndash;73. Available from: http://dx.doi.org/10.1002/mrm.21521\u003c/li\u003e\n\u003cli\u003eManj\u0026oacute;n JV, Romero JE, Vivo-Hernando R, Rubio-Navarro G, De la Iglesia-Vaya M, Aparici-Robles F, et al. Deep ICE: A Deep learning approach for MRI Intracranial Cavity Extraction. arXiv; 2020. Available from: https://arxiv.org/abs/2001.05720\u003c/li\u003e\n\u003cli\u003eHe K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. arXiv; 2015. Available from: https://arxiv.org/abs/1512.03385\u003c/li\u003e\n\u003cli\u003eManj\u0026oacute;n JV, Romero JE, Coupe P. Deep learning based MRI contrast synthesis using full volume prediction using full volume prediction. Biomed Phys Eng Express. 2021 Dec 3;8(1):015013. Available from: http://dx.doi.org/10.1088/2057-1976/ac3c64\u003c/li\u003e\n\u003cli\u003eFaber J, K\u0026uuml;gler D, Bahrami E, Heinz LS, Timmann D, Ernst TM, et al. CerebNet: A fast and reliable deep-learning pipeline for detailed cerebellum sub-segmentation. NeuroImage. 2022 Dec;264:119703. Available from: http://dx.doi.org/10.1016/j.neuroimage.2022.119703\u003c/li\u003e\n\u003cli\u003eHenschel L, Conjeti S, Estrada S, Diers K, Fischl B, Reuter M. FastSurfer - A fast and accurate deep learning based neuroimaging pipeline. NeuroImage. 2020 Oct;219:117012. Available from: http://dx.doi.org/10.1016/j.neuroimage.2020.117012\u003c/li\u003e\n\u003cli\u003eXu T, Kiar G, Cho JW, Bridgeford EW, Nikolaidis A, Vogelstein JT, et al. ReX: an integrative tool for quantifying and optimizing measurement reliability for the study of individual differences. Nat Methods. 2023 June 1;20(7):1025\u0026ndash;8. Available from: http://dx.doi.org/10.1038/s41592-023-01901-3\u003c/li\u003e\n\u003cli\u003eBartko JJ. The Intraclass Correlation Coefficient as a Measure of Reliability. Psychol Rep. 1966 Aug;19(1):3\u0026ndash;11. Available from: http://dx.doi.org/10.2466/pr0.1966.19.1.3\u003c/li\u003e\n\u003cli\u003eMcGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996 Mar;1(1):30\u0026ndash;46. Available from: http://dx.doi.org/10.1037/1082-989x.1.1.30\u003c/li\u003e\n\u003cli\u003eShrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86(2):420\u0026ndash;8. Available from: http://dx.doi.org/10.1037/0033-2909.86.2.420\u003c/li\u003e\n\u003cli\u003eLiljequist D, Elfving B, Skavberg Roaldsen K. Intraclass correlation \u0026ndash; A discussion and demonstration of basic features. Chiacchio F, editor. PLoS ONE. 2019 July 22;14(7):e0219854. Available from: http://dx.doi.org/10.1371/journal.pone.0219854\u003c/li\u003e\n\u003cli\u003eKoo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine. 2016 June;15(2):155\u0026ndash;63. Available from: http://dx.doi.org/10.1016/j.jcm.2016.02.012\u003c/li\u003e\n\u003cli\u003eYushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., \u0026amp; Gerig, G. (2006). User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability. \u003cem\u003eNeuroImage\u003c/em\u003e, \u003cem\u003e31\u003c/em\u003e(3), 1116\u0026ndash;1128. https://doi.org/10.1016/j.neuroimage.2006.01.015\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"the-cerebellum","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"cere","sideBox":"Learn more about [The Cerebellum](http://link.springer.com/journal/12311)","snPcode":"12311","submissionUrl":"https://submission.nature.com/new-submission/12311/3","title":"The Cerebellum","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Cerebellum, Test-Retest Reliability, Magnetic Resonance Imaging, Segmentation","lastPublishedDoi":"10.21203/rs.3.rs-8724066/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8724066/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground.\u003c/strong\u003e Characterizing cerebellar morphology is fundamental for accurately mapping its structure and function across individuals, yet remains challenging due to its densely foliated architecture. Although multiple automated segmentation pipelines exist, the measurement reproducibility of these tools has not been benchmarked.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods.\u003c/strong\u003e We conducted a systematic assessment of robustness for cerebellar morphology estimates using four commonly used pipelines: one classic parcellation method (CERES), two deep-learning methods (ACAPULCO, DeepCERES), and one voxel-based morphometry toolbox (SUIT). Leveraging the HNU test-retest dataset, which provides MRI scans for ten timepoints per individual over a month, we evaluated the test-retest reliability for each of four pipelines using ReX, an integrative tool for quantifying and optimizing measurement reliability and individual differences. We quantified intra- and inter-individual variability, as well as the Intraclass Correlation Coefficient (ICC), for each pipeline at both global and region-of-interest levels.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults.\u003c/strong\u003e Overall, all pipelines yielded highly reliable segmentation volumes (ICC \u0026gt; 0.8). Across pipelines, DeepCERES demonstrated the strongest performance, exhibiting high inter-individual consistency and low intra-individual variability. Importantly, our analysis highlighted substantial heterogeneity in reliability across lobules for each method. Lobule X consistently showed reduced reliability whereas lobules I-V were reliably estimated across all pipelines.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion.\u003c/strong\u003e Our work evaluated the robustness of cerebellar segmentation pipelines. While DeepCERES offers a robust global performance, substantial lobule-specific variability underscores the need for reliability-aware pipeline selection to optimize morphology estimation in research.\u003c/p\u003e","manuscriptTitle":"Towards Reliable Measurement of Cerebellar Morphology : A comparative assessment of segmentation pipelines","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-04 05:17:23","doi":"10.21203/rs.3.rs-8724066/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-02-25T11:15:16+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-24T12:48:53+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-18T04:36:46+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"77876062877168954037970218679491844383","date":"2026-02-04T23:04:22+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"46643526830957950052128689538355962653","date":"2026-01-30T07:52:28+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-01-30T06:43:56+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-29T10:35:50+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-01-29T10:34:16+00:00","index":"","fulltext":""},{"type":"submitted","content":"The Cerebellum","date":"2026-01-28T17:19:41+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"the-cerebellum","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"cere","sideBox":"Learn more about [The Cerebellum](http://link.springer.com/journal/12311)","snPcode":"12311","submissionUrl":"https://submission.nature.com/new-submission/12311/3","title":"The Cerebellum","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"8f8095d3-5290-4f03-b815-fe9a48188668","owner":[],"postedDate":"February 4th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-02-25T11:27:37+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-04 05:17:23","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8724066","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8724066","identity":"rs-8724066","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}