Are neuroanatomical phenotypes for psychiatric disorders robust? An assessment of the reproducibility of grey matter differences in mental illness

Background Despite thousands of magnetic resonance imaging (MRI) studies reporting grey matter alterations in psychiatric disorders, the field has failed to converge on robust neuroanatomical phenotypes for any specific diagnosis. Here, we examine whether current practices will ever converge on such a phenotype, which is essential for tracking illness risk, progression, and treatment.

We evaluated the consistency of brain-wide maps of grey matter volume and cortical thickness alterations obtained for each of 59 study sites of five psychiatric disorders (schizophrenia, schizoaffective disorder, autism spectrum disorder, major depressive disorder, and bipolar disorder), totalling 2437 patients and 2065 controls. We calculated cross-site consistency using spatial correlations between pairs of site-specific difference maps and benchmarked the findings against 7 study sites of Alzheimer’s disease (654 patients, 937 controls). Findings Disorder-specific volume and thickness alterations showed low consistency, with a median pair-wise cross-site correlation of 𝑟 ≤ 0.16 for psychiatric disorders compared to 𝑟 = 0.54 in Alzheimer’s disease. Consistency estimates were not strongly associated with site-specific variations in 19 different demographic, clinical, and scanner characteristics of the study participants and were robust to data processing and analysis. Bootstrapping analyses indicated that consistent results (𝑟 > 0.5) could be obtained for schizophrenia if study-specific sample sizes exceed approximately 200 (for cases and controls), but consistent findings for other disorders may require much larger samples. Interpretation Our findings indicate that current widespread practices, involving case-control comparisons of convenience samples numbering between 30 and 100 patients, will not converge on robust neuroanatomical phenotypes for psychiatric disorders. Increasing sample size will facilitate this goal in schizophrenia, but much larger samples, or refined ascertainment strategies aiming to recruit phenotypically homogeneous patient subgroups, may be required for other disorders. Competing Interest Statement The authors have declared no competing interest. Funding Statement This study was funded by National Health and Medical Research Council, Australian Research Council, Singapore National Medical Research Council, Yong Loo Lin School of Medicine Research Core Funding. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Human Ethics Team of Monash University gave ethical approval for this work. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Footnotes Funding: National Health and Medical Research Council and Australian Research Council. Singapore National Medical Research Council, Yong Loo Lin School of Medicine Research Core Funding. Data Availability All data produced in the present study are available upon reasonable request to the authors