MRICombo: a deep-learning-based framework for universal volumetric segmentation grading-staging and malignancy detection across heterogeneous MRI.

OA: gold
AI-generated summary by qwen3.7-flash, 2026-09-22

MRICombo, a unified deep learning framework trained on 7,380 MRI sequences, achieves state-of-the-art performance in volumetric segmentation, grading, staging, and malignancy detection across heterogeneous imaging protocols with robust cross-protocol generalizability.

One-sentence paraphrase of the abstract; not a substitute for reading it. No clinical advice. How this works

AI-generated deep summary by qwen3.7-flash, 2026-09-21 · read from full text

MRICombo is a unified deep-learning framework designed to perform simultaneous volumetric segmentation, grading, staging, and malignancy detection across heterogeneous multi-sequence MRI scans. The authors developed this architecture using a diverse dataset of 7380 scans covering nine MRI sequences and 25 anatomical structures or tumor types, including brain, liver, breast, bladder, and nasopharyngeal cancers. Results indicate that the model achieves state-of-the-art performance in internal evaluations and demonstrates strong generalization and robustness even when specific imaging sequences are missing. The paper does not explicitly discuss endometriosis or adenomyosis; it was included in the corpus via a keyword match in the upstream search index.

Read from the paper's body, not the abstract. Not a substitute for reading the paper. No clinical advice. How this works

Abstract

Comprehensive magnetic resonance imaging (MRI) analysis in oncology involves multiple interrelated tasks including volumetric segmentation, grading, staging, and malignancy detection. However, most existing deep learning models are task-specific or sequence-specific, lacking the generalizability required for heterogeneous sequences. Here we present MRICombo, a unified multi-expert deep learning framework for universal anatomical delineation and tumor characterization across 9 heterogeneous imaging sequences. Developed using 7,380 MRI sequences from 2,354 individuals, MRICombo achieves state-of-the-art performance with mean Dice similarity coefficients of 0.836 for segmenting 14 critical anatomical structures and 0.625 for labeling 11 major tumor types. It also attains a mean area under the receiver operating characteristic curve (AUROC) of 0.920 for glioma grading, bladder and nasopharyngeal cancer staging, and breast and liver tumor malignancy detection. External validation on four independent datasets (1082 sequences from 734 individuals) and transfer learning evaluation (512 individuals) confirm robust cross-protocol generalizability. Furthermore, MRICombo supports flexible inference with missing sequences and offers decision interpretability through sequence clustering and expert contribution analysis. As a unified clinical solution, MRICombo significantly reduces deployment costs and has the potential to streamline diagnostic workflows, supporting more personalized oncology care.
Full text 92,170 characters · extracted from pmc-nxml · 5 sections · click to expand

Methods

This study utilized data from 12 publicly available MRI datasets, collectively termed MRIComboData, comprising 2354 participants with 7380 total scans for model training and internal validation. All datasets were obtained from established repositories including Zenodo, The Cancer Imaging Archive (TCIA) 41 , Medical Image Computing and Computer Assisted Intervention Society (MICCAI) challenges, GitHub, and Kaggle. Ethical approval and informed consent for the original data collection were obtained by the respective contributing institutions, as documented in their primary publications. This retrospective analysis of de-identified public data was exempt from additional institutional review board approval. The training cohort encompassed diverse anatomical regions and clinical tasks (Fig.  1 a-d, Supplementary Table  1 ). Segmentation tasks spanned five anatomical domains: (1) brain (whole tumor, tumor core, enhanced tumor); (2) head-neck (nasopharyngeal carcinoma, primary gross tumor volumes, involved metastatic lymph nodes); (3) breast (primary tumor); (4) abdomen (liver, spleen, right kidney, left kidney, gallbladder, esophagus, stomach, aorta, inferior vena cava, pancreas, right adrenal gland, left adrenal gland, duodenum); and (5) pelvis (colorectal cancer, bladder cancer, prostate, prostate cancer). These encompassed 25 distinct segmentation targets across 11 major tumor types and 14 critical organ structures. Classification tasks focused on tumor characterization in brain, breast, liver, bladder, and nasopharyngeal regions, as detailed in Supplementary Tables  10 – 14 . Each participant’s MRI data included multiple sequences (range: 1–4 sequences per participant), with sequence types varying by anatomical region and clinical indication (Supplementary Table  1 ). To evaluate model generalizability across independent datasets, we assembled four external validation cohorts totaling 734 participants (Supplementary Table  1 ). These comprised three segmentation datasets and one classification dataset from geographically and technically diverse sources. The external cohorts included: (1) Private-Liver ( n = 506), a proprietary dataset of hepatobiliary phase (HBP) MRI scans for liver segmentation, collected under institutional review board approval (Ethics Committee of Nanfang Hospital, Southern Medical University, approval number NFEC-2025-477); (2) BrTS2025-SSA ( n = 60), a multi-sequence brain tumor segmentation dataset (T1, T1c, T2, FLAIR) from sub-Saharan Africa; (3) NACT ( n = 64), a breast tumor segmentation dataset featuring DCE MRI sequences; and (4) LLD-MMRI2025 ( n = 104), a liver tumor classification dataset with AP and VP imaging. Critically, none of these datasets or their institutional sources were represented in the training cohort, ensuring rigorous evaluation of cross-population generalization. To assess transfer learning capability on anatomical structures absent from training data, we employed the Medical Segmentation Decathlon Hippocampus dataset ( n = 260) as an unseen test set. This dataset comprises T1 MRI scans with expert-annotated hippocampal segmentations. The hippocampus represents a neuroanatomical structure fundamentally distinct from any target in the training cohort, providing a stringent test of the model’s ability to generalize learned feature representations to novel anatomical contexts with fine-tuning. Additionally, we included a private uterus multi-class dataset ( n = 252) as an unseen evaluation benchmark for diagnostic classification. Each subject contains four MRI sequences, including T1, T1c, T2, and DWI, and is annotated into three diagnostic categories: Nabothian cyst, fibroid, and adenomyosis. This dataset provides a complementary evaluation setting for assessing transfer learning on heterogeneous multi-sequence MRI data with a multi-class diagnostic objective. All segmentation masks and classification labels across training, validation, and test cohorts were derived from expert radiologist annotations. For public datasets, annotations were previously validated through inter-rater reliability assessments or multi-center consensus protocols, as reported in primary publications 18 . The Private-Liver dataset annotations were performed by two board-certified radiologists with >3 years of abdominal imaging experience, with discrepancies resolved by consensus review. All annotations were adopted without modification for model development and evaluation. Training deep learning models on diverse and heterogeneous medical imaging datasets presents three primary challenges. First, the framework must simultaneously support end-to-end segmentation and classification within a unified architecture. Second, each task encompasses multiple clinically relevant sub-tasks: segmentation involves delineating various organs and tumors across different anatomical regions, while classification requires distinguishing tumor types and grading categories across multiple sites. Third, the incorporated datasets exhibit substantial heterogeneity in imaging protocols, including variations in sequence types (single-sequence versus multi-sequence studies) and acquisition parameters. To address these challenges, we implemented a standardized preprocessing pipeline across all datasets. All volumetric images were aligned to a common anatomical coordinate system and resampled to a uniform voxel spacing of 1.2 × 1.5 × 1.5 mm using linear interpolation. Intensity normalization was then performed using z-score normalization, ensuring zero mean and unit standard deviation across each entire volume. For segmentation tasks, we adopted a patch-based approach, extracting overlapping patches of 96 × 96 × 96 voxels from the preprocessed volumes. This strategy balances computational efficiency with preservation of spatial context for precise boundary delineation. For classification tasks, entire images were resampled to a consistent spatial size, maintaining global anatomical information while ensuring uniform input dimensions across all samples. This dual-path preprocessing framework effectively handles data heterogeneity while enabling simultaneous optimization for both segmentation and classification objectives. Our framework employs a residual 3D U-Net architecture, combining the volumetric encoder-decoder design of 3D U-Net 42 with residual blocks 39 . As illustrated in Supplementary Fig.  1 , MRICombo comprises four stages: (i) standardized input processing that handles missing and masked MRI sequences; (ii) sequence-specific feature extraction; (iii) a Prompt-Reconstructed Mixture-of-Experts (PR-MoE) module, which integrates a dual-prompt gating router, hierarchical Mixture-of-Experts (MoE) layers, and a Masked Feature Reconstruction (MFR) branch; and (iv) multi-task output heads for segmentation and classification. This framework supports nine MRI sequence types, including T1, T1c, T2, FLAIR, DWI, ADC, DCE, AP, and VP. For each task, the input is organized into a fixed-length eight-slot sequence-code vector, where DCE is represented by two slots corresponding to pre-contrast and post-contrast DCE images. Specifically, the segmentation pathway uses T1, T1c, T2, FLAIR, pre-contrast DCE, post-contrast DCE, ADC, and DWI, whereas the classification pathway uses T1, T1c, T2, FLAIR, pre-contrast DCE, post-contrast DCE, AP, and VP. Each input slot is accompanied by a binary sequence code indicating availability, eliminating the need for data imputation while maintaining model robustness across different clinical scenarios. In addition, 3D block-wise masking (ratio = 0.25) is applied to the available sequences during training (Supplementary Fig.  1 A, “Masked MRI”) to supervise the MFR branch (described below). The sequence code is processed through a learnable embedding layer with 256 channels to generate sequence-specific feature representations that guide the subsequent stages of the processing pipeline. Following the standardized input processing module, features are extracted through task-specific pathways (Supplementary Fig.  1 D): separate feature extraction networks are maintained for segmentation and classification, allowing each pathway to develop specialized representations optimized for its respective objective. The multi-sequence feature extraction module comprises eight separate blocks, each containing a single convolutional layer. For each input volume \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${X}_{s}\in {{\mathbb{R}}}^{H\times W\times D}$$\end{document} X s ∈ R H × W × D , the corresponding block encodes X s into a 32-channel feature representation. The resulting features from all sequences are then modulated by their respective sequence codes and aggregated by summation, producing a 32-channel sequence feature fusion map (Supplementary Fig.  1 B). To explicitly handle missing sequences and enforce robust cross-sequence representation learning, the MFR branch (Supplementary Fig.  1 B, G) employs a lightweight masked autoencoder-style decoder 43 to reconstruct the masked regions in the fused feature space, supervised by an L1 loss. This auxiliary objective forces the network to capture cross-sequence and cross-organ feature commonalities, yielding robust representations even when sequences are missing. The fused feature map, together with learnable dataset and task prompts, is then fed into the dual-prompt gating router (Supplementary Fig.  1 E) to select the top-k experts (with k=2 in our implementation) within the PR-MoE module. Both segmentation and classification pathways employ hierarchical PR-MoE structures that enable dynamic adaptation to diverse input patterns. Each PR-MoE layer consists of four expert networks and a dual-prompt gating router (Supplementary Fig.  1 E). The router takes three inputs: a dataset prompt encoding the source dataset, a task prompt specifying segmentation or classification, and the incoming feature map f . Notably, datasets covering the same anatomical region share a common dataset prompt identifier (e.g., the brain, breast, and bladder each map to a single identifier), encouraging the model to learn anatomy-specific representations across tasks. The dataset and task prompts are first projected into 32- and 16-dimensional embeddings via independent fully connected layers, concatenated, and passed through a two-layer multilayer perceptron (MLP) with ReLU activation 44 to produce a unified 32-dimensional prompt encoding. In parallel, f is compressed via global average pooling. The pooled features and the prompt encoding are then concatenated and passed through a second MLP, followed by a SoftMax, to generate expert weights. The expert outputs are aggregated by weighted summation along the expert channels. For the segmentation pathway, we implement an encoder-decoder structure with PR-MoE components at multiple resolution levels (Fig.  S1 F). At each level, four experts are deployed, each composed of DownBasicBlocks consisting of two consecutive 3D convolutions with instance normalization 45 and ReLU activation, followed by 3D max-pooling for down-sampling. The decoder mirrors this design using UpBasicBlocks with trilinear up-sampling and skip connections from the corresponding encoder levels. The final PR-MoE layer of the decoder performs expert selection and estimates the probability map \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{P}\in {{\mathbb{R}}}^{H\times W\times D}$$\end{document} P ^ ∈ R H × W × D . For the classification pathway, we utilize the encoder with PR-MoE layers but without a decoder. Before the classification head (Fig.  S1 C), features from the final encoder layer are processed through global average pooling followed by a dropout layer (rate = 0.2). The input and output sizes of the fully connected layer are 512 and 2, respectively. To enhance computational efficiency while maintaining model expressiveness, we employ a top-2 sparse gating mechanism, activating only the two most relevant experts per PR-MoE layer. We evaluate our framework against a comprehensive set of task-specific and general-purpose baselines. Task-specific segmentation benchmarks include nnUNet 35 , SwinUNETR 36 , and TransUNet 37 , while a tailored 3D ResNet baseline (ResNet-specific) 39 is used for single-task classification. For general-purpose segmentation, we compare against SAM-Med3D 20 and UniMRISegNet 38 . Regarding general classification, we extend the aforementioned ResNet-specific backbone to investigate two fusion strategies: Early Fusion (averaging raw input images) and Feature Fusion (averaging high-level unimodal representations). Finally, the Swin Transformer 40 is included as an additional, flexible general-purpose baseline for classification. To ensure fair comparison, all baselines were trained and evaluated using the same data splits, preprocessing, optimizer, random seed, and decision criterion as MRICombo. During training, we extracted patches of voxels from the 3D volume centered around a seed point. A series of randomly chosen augmentations, such as cropping, rotation ( ± 15 ∘ ), scaling (0.9–1.1), and adjustments to brightness ( ± 0.2), contrast ( ± 0.2), and gamma (0.8–1.2), were applied throughout the training. We used the AdamW optimizer 46 implemented in PyTorch for model optimization. The initial learning rate was set to 3 × 10 −5 and was applied uniformly to all trainable parameters. The learning rate was updated once at the beginning of each epoch using a polynomial decay schedule, defined as η e  =  η 0  × (1− e / E ) 0.9 , where η 0 denotes the initial learning rate, e denotes the current epoch, and E denotes the total number of training epochs. Weight decay was implemented as the decoupled weight-decay term of AdamW and was kept constant throughout training, without a separate decay schedule. The weight-decay coefficient was set to 3 × 10 −5 . During inference, we employ a sliding window approach with an overlap of 50% between adjacent patches, followed by a weighted averaging of the predictions based on the distance from the patch center to mitigate boundary artifacts. Our framework is trained by minimizing a multi-task objective function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\mathcal{L}}}}$$\end{document} L , which combines a segmentation loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{seg}}}}}$$\end{document} L seg , a classification loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{cls}}}}}$$\end{document} L cls , and a load-balancing regularization term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{balance}}}}}$$\end{document} L balance for the MoE module. For the segmentation task, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{seg}}}}}$$\end{document} L seg combines a Dice loss and a Binary Cross-Entropy (BCE) loss to address class imbalance. The segmentation loss is defined as: 1 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{seg}}}}}=\left(1-\frac{2{\sum }_{i=1}^{V}{p}_{i}{\hat{p}}_{i}+\epsilon }{{\sum }_{i=1}^{V}{p}_{i}+\mathop{\sum }_{i=1}^{V}{\hat{p}}_{i}+\epsilon }\right)-\frac{1}{V}\mathop{\sum }_{i=1}^{V}\left[{p}_{i}\log {\hat{p}}_{i}+(1-{p}_{i})\log (1-{\hat{p}}_{i})\right],$$\end{document} L seg = 1 − 2 ∑ i = 1 V p i p ^ i + ϵ ∑ i = 1 V p i + ∑ i = 1 V p ^ i + ϵ − 1 V ∑ i = 1 V p i log p ^ i + ( 1 − p i ) log ( 1 − p ^ i ) , where, V denotes the total number of voxels in the input 3D volume; i  ∈ {1, …,  V } is the spatial voxel index; p i  ∈ {0, 1} represents the ground-truth binary label for the i -th voxel; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hat{p}}_{i}\in [0,1]$$\end{document} p ^ i ∈ [ 0 , 1 ] denotes the model’s predicted probability that the i -th voxel belongs to the target class; ϵ is a small smoothing constant (e.g., 1 × 10 −5 ) added to ensure numerical stability and prevent division by zero. For the classification task, we employ the cross-entropy loss: 2 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{cls}}}}}=-\mathop{\sum }_{j=1}^{K}{y}_{j}\log {\hat{y}}_{j},$$\end{document} L cls = − ∑ j = 1 K y j log y ^ j , where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{{{{\bf{y}}}}}\in {{\mathbb{R}}}^{K}$$\end{document} y ^ ∈ R K represents the predicted probability distribution over K classes, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\bf{y}}}}\in {{\mathbb{R}}}^{K}$$\end{document} y ∈ R K denotes the one-hot encoded ground truth label. Neither loss term explicitly incorporates expert routing information. To encourage uniform expert utilization and prevent expert collapse, we introduce a load balancing loss for MoE module. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\bf{g}}}}\in {{\mathbb{R}}}^{E}$$\end{document} g ∈ R E denote the gating weights for E experts. The load balancing loss uses KL-divergence to measure the deviation from uniform distribution, as follows: 3 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\mathcal{L}}}}}_{{{{\rm{balance}}}}}={\sum }_{j=1}^{E}{w}_{j}\log ({w}_{j}\cdot E),$$\end{document} L balance = ∑ j = 1 E w j log ( w j ⋅ E ) , where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${w}_{j}={\mathbb{E}}[{g}_{j}]$$\end{document} w j = E [ g j ] is the average gating probability for expert j across all tokens in a batch. Combining the segmentation and classification losses with the MoE load-balancing regularization term, the overall training objective is defined as: 4 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\mathcal{L}}}}={{{{\mathcal{L}}}}}_{{{{\rm{seg}}}}}+\alpha {{{{\mathcal{L}}}}}_{{{{\rm{cls}}}}}+\beta {{{{\mathcal{L}}}}}_{{{{\rm{balance}}}}},$$\end{document} L = L seg + α L cls + β L balance , where α  = 0.5 is the classification weight coefficient that balances the segmentation and classification tasks, and β  = 0.05 is the load balancing coefficient that controls the strength of expert utilization regularization. These hyperparameters were empirically determined to achieve optimal performance across both tasks. This joint optimization strategy enables the model to simultaneously learn accurate boundaries of segmented regions, their corresponding class information, and optimal expert routing, thereby enhancing the overall performance in multi-task scenarios while maintaining computational efficiency through balanced expert utilization. For segmentation evaluation, we followed the recommendations in Metrics Reloaded 47 and employed two complementary metrics: the Dice Similarity Coefficient (DSC) ranging from 0 to 100% and the Average Surface Distance (ASD). These metrics collectively capture both volumetric accuracy and boundary precision. The DSC quantifies the spatial overlap between the predicted segmentation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{P}$$\end{document} P ^ and the ground truth P : 5 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\,{\mbox{DSC}}\,(P,\hat{P})=\frac{2| P\cap \hat{P}| }{| P|+| \hat{P}| }.$$\end{document} DSC ( P , P ^ ) = 2 ∣ P ∩ P ^ ∣ ∣ P ∣ + ∣ P ^ ∣ . The ASD measures the average discrepancy between the boundaries of the prediction and the ground truth: 6 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\,{\mbox{ASD}}\,(P,\hat{P})=\frac{1}{2}\left[\frac{1}{| \partial P| }{\sum}_{p\in \partial P}{{\min }_{q\in \partial \hat{P}}}\parallel p-q\parallel+\frac{1}{| \partial \hat{P}| }{\sum}_{q\in \partial \hat{P}}{{\min }_{p\in \partial P}}\parallel q-p\parallel \right].$$\end{document} ASD ( P , P ^ ) = 1 2 1 ∣ ∂ P ∣ ∑ p ∈ ∂ P min q ∈ ∂ P ^ ∥ p − q ∥ + 1 ∣ ∂ P ^ ∣ ∑ q ∈ ∂ P ^ min p ∈ ∂ P ∥ q − p ∥ . To measure MRICombo’s classification performance, the area under the receiver operating characteristic (AUROC), the area under the precision-recall curve (AUPRC), F1-score, and the Matthews correlation coefficient (MCC) were used to provide comprehensive evaluations. Comparisons between MRICombo and each baseline were performed using two-sided paired bootstrap tests ( B  = 1, 000 resamples) for both classification metrics (MCC, F1, AUROC, AUPRC) and segmentation metrics (Dice, ASD), with p  < 0.05 considered statistically significant. All analyses were conducted in Python 3.9.7 using NumPy 1.23, SciPy 1.10.1, and scikit-learn 1.1.3. Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Results

MRICombo (Fig.  1 f) is designed to serve as a unified framework for universal 3D MRI segmentation, tumor grading, and malignancy classification under varying imaging sequences. A key challenge in building such a framework lies in its ability to adapt to variability in imaging protocols, anatomical contexts, and disease presentations. To address this challenge, we constructed a diverse MRI dataset (MRIComboData) comprising 7380 scans from 2354 patients, covering nine MRI sequences and 25 anatomical structures and tumor types (Fig.  1 a–d). The nine supported sequences are T1, T1-weighted contrast-enhanced (T1c), T2, FLAIR, Diffusion-Weighted Imaging (DWI), apparent diffusion coefficient (ADC), Dynamic Contrast-Enhanced (DCE; both pre- and post-contrast), Arterial Phase (AP), and Venous Phase (VP). Figure  1 a presents a comprehensive taxonomy of segmentation targets and their corresponding MRI sequences used in this study. The anatomical structures and pathological regions are systematically organized into two main categories: Organs (14 types) and Tumors (11 types). The segmentation tasks primarily employ T1, T1c, T2, FLAIR, DCE, ADC, and DWI sequences, used individually or in combination according to the specific clinical task and target anatomy. Figure  1 b displays the classification tasks and their corresponding imaging protocols, covering five clinically significant scenarios: (1) brain tumor grading (high-grade glioma/low-grade glioma, HGG/LGG), (2) bladder cancer staging (non-muscle-invasive/muscle-invasive bladder cancer, NMIBC/MIBC), (3) liver tumor characterization (benign/malignant), (4) breast tumor characterization (benign/malignant), and (5) nasopharyngeal carcinoma T-staging (T1-T2 vs. T3-T4). The classification tasks draw on T1, T1c, T2, FLAIR, DCE, AP, and VP sequences, employed either individually or in combination according to the specific diagnostic requirements of each clinical scenario. Figure  1 c presents the distribution of MRI sequences utilized in segmentation tasks. T2 imaging emerges as the most frequently used sequence with 1523 instances, followed by T1c and T1 (622 instances each). Additional sequences including ADC, DCE, FLAIR, and DWI collectively bring the total to 4,038 sequences across all modalities. Figure  1 d illustrates the sequence distribution for classification tasks, where T2 imaging remains the predominant choice at 837 instances, followed by T1c and T1 (562 instances each), DCE (464 instances), and dynamic phase imaging encompassing both AP and VP (316 instances each), FLAIR (285 instances), resulting in a total of 3342 sequences analyzed. The scale and diversity of MRIComboData empower MRICombo to learn rich medical imaging representations capable of capturing a wide range of anatomical structures and pathological findings across different sequence modalities. The variability in imaging protocols and their corresponding segmentation and classification targets underscores the need for a unified framework that can handle both the distinctive characteristics of individual modalities and their combined features, while maintaining robust performance across diverse clinical tasks. The representations learned from MRIComboData can be effectively fine-tuned to facilitate transfer learning to unseen organ clusters and disease differentiations (Fig.  1 h), demonstrating strong generalization capability. Fig. 1 Overview of MRICombo and MRIComboData. a – d Composition and key characteristics of the MRIComboData. a Sunburst chart displaying anatomical regions (organs and tumors) segmented by MRICombo and their corresponding MRI sequences. b Sunburst chart illustrating tumor classification tasks, including benign/malignant discrimination and tumor grading, with associated MRI sequences. c Bar plot quantifying the number of sequences utilized for segmentation tasks. d Bar plot quantifying the number of sequences employed for classification tasks. e The generation of multi-position and multi-sequence MRI data. f Architecture of MRICombo (see Supplementary Fig.  1 for details), featuring multiple expert modules that process diverse anatomical positions and variable MRI sequence combinations. g Outputs of MRICombo across six clinical applications: (i) segmentation of various organs and tumors; (ii) breast tumor malignancy detection; (iii) liver tumor malignancy detection; (iv) brain tumor grading (HGG/LGG); (v) bladder cancer invasive staging (MIBC/NMIBC); and (vi) nasopharyngeal carcinoma (NPC) T-staging. h Schema of transfer learning based on MRICombo’s pretrained parameters for application to unseen organ. Schematic e and g were created in BioRender. Zhang, Z. (2026)  https://BioRender.com/3cta8xm . LAG left adrenal gland, RAG right adrenal gland, IVC inferior vena cava, WT whole brain tumor, TC brain tumor core, ET enhanced tumor, Bladder-C bladder cancer, GTVp primary gross tumor volumes, GTVn involved metastatic lymph nodes, Colorectal-C colorectal cancer, Breast-T breast tumor, Liver-T liver tumor, AP arterial phase, VP venous phase, HGG high-grade glioma, LGG low-grade glioma, NMIBC non-muscle-invasive bladder cancer, MIBC muscle-invasive bladder cancer. Source data are provided as a Source Data file. a – d Composition and key characteristics of the MRIComboData. a Sunburst chart displaying anatomical regions (organs and tumors) segmented by MRICombo and their corresponding MRI sequences. b Sunburst chart illustrating tumor classification tasks, including benign/malignant discrimination and tumor grading, with associated MRI sequences. c Bar plot quantifying the number of sequences utilized for segmentation tasks. d Bar plot quantifying the number of sequences employed for classification tasks. e The generation of multi-position and multi-sequence MRI data. f Architecture of MRICombo (see Supplementary Fig.  1 for details), featuring multiple expert modules that process diverse anatomical positions and variable MRI sequence combinations. g Outputs of MRICombo across six clinical applications: (i) segmentation of various organs and tumors; (ii) breast tumor malignancy detection; (iii) liver tumor malignancy detection; (iv) brain tumor grading (HGG/LGG); (v) bladder cancer invasive staging (MIBC/NMIBC); and (vi) nasopharyngeal carcinoma (NPC) T-staging. h Schema of transfer learning based on MRICombo’s pretrained parameters for application to unseen organ. Schematic e and g were created in BioRender. Zhang, Z. (2026)  https://BioRender.com/3cta8xm . LAG left adrenal gland, RAG right adrenal gland, IVC inferior vena cava, WT whole brain tumor, TC brain tumor core, ET enhanced tumor, Bladder-C bladder cancer, GTVp primary gross tumor volumes, GTVn involved metastatic lymph nodes, Colorectal-C colorectal cancer, Breast-T breast tumor, Liver-T liver tumor, AP arterial phase, VP venous phase, HGG high-grade glioma, LGG low-grade glioma, NMIBC non-muscle-invasive bladder cancer, MIBC muscle-invasive bladder cancer. Source data are provided as a Source Data file. A comprehensive comparison was conducted to evaluate our proposed MRICombo framework against five state-of-the-art segmentation approaches: nnUNet 35 , SwinUNETR 36 , TransUNet 37 , SAM-Med3D 20 , and UniMRISegNet 38 on an internal test set. The segmentation performance was quantified using the Dice Similarity Coefficient (DSC) and the average surface distance (ASD). Our results confirm the statistically significant superiority of the proposed MRICombo framework. Overall, MRICombo consistently surpassed all competing methods, achieving average DSC scores of 83.61% for organ segmentation and 62.50% for tumor segmentation. On the organ segmentation task, MRICombo outperformed the next-best model, UniMRISegNet, by 2.54% in DSC (Fig.  2 a, p  < 0.001). For the more challenging tumor segmentation, our framework also secured a superior performance, leading UniMRISegNet by 2.52% in DSC (Fig.  2 b, p  < 0.001). Furthermore, MRICombo demonstrated improved capability in surface detection, attaining ASD of 0.65 mm for organs and 4.64 mm for tumors. This represents a significant reduction in ASD of 1.82 mm and 2.17 mm compared to the best existing method, as shown in Fig.  2 c, p  = 0.002 and Fig.  2 d, p  = 0.015, respectively. These results collectively indicate that MRICombo provides a more powerful segmentation encoder than all benchmarked models. Fig. 2 Quantitative and qualitative evaluation of segmentation performance on the internal test set. a , b Bar plots showing Dice Similarity Coefficient (DSC) scores for ( a ) 14 organ segmentation tasks and ( b ) 11 tumor segmentation tasks. c , d Bar plots displaying Average Surface Distance (ASD) scores for ( c ) organ and ( d ) tumor segmentation tasks. For ( a )–( d) , bars show the mean DSC or ASD across all test cases from the corresponding segmentation tasks, with n = 311 independent test cases for organ segmentation in a and c , and n = 479 independent test cases for tumor segmentation in b and d . The same n was used for each method. e Comparative analysis of our framework’s average DSC performance across all organ and tumor segmentation tasks. Each data point is one independent test case. The number of test cases per box, left to right, is: liver ( n = 32); spleen ( n = 20); right kidney ( n = 20); left kidney ( n = 20); gallbladder ( n = 19); esophagus ( n = 20); stomach ( n = 20); aorta ( n = 20); inferior vena cava ( n = 20); pancreas ( n = 20); right adrenal gland ( n = 20); left adrenal gland ( n = 20); duodenum ( n = 20); prostate ( n = 40); brain enhanced tumor ( n = 48); brain tumor core ( n = 57); brain whole tumor ( n = 57); gross tumor ( n = 31); metastatic tumor ( n = 24); nasopharyngeal tumor ( n = 57); breast tumor ( n = 33); liver tumor ( n = 12); colorectal tumor ( n = 31); bladder tumor ( n = 45); prostate tumor ( n = 84). f Representative visualization of segmentation results, showcasing abdominal organs and primary gross tumor volumes (GTVp) across different methods. e Presents the results as a box plot, where the lower bound, center line, and upper bound of the box correspond to the first quartile (Q1), median, and third quartile (Q3), respectively, defining the interquartile range (IQR) that contains the central 50% of the data. The whiskers extend to the furthest data points within 1.5 × IQR, and any points beyond are classified as outliers. Statistical test: two-sided paired bootstrap test (1000 bootstrap resamples); no adjustment for multiple comparisons. Source data are provided as a Source Data file. a , b Bar plots showing Dice Similarity Coefficient (DSC) scores for ( a ) 14 organ segmentation tasks and ( b ) 11 tumor segmentation tasks. c , d Bar plots displaying Average Surface Distance (ASD) scores for ( c ) organ and ( d ) tumor segmentation tasks. For ( a )–( d) , bars show the mean DSC or ASD across all test cases from the corresponding segmentation tasks, with n = 311 independent test cases for organ segmentation in a and c , and n = 479 independent test cases for tumor segmentation in b and d . The same n was used for each method. e Comparative analysis of our framework’s average DSC performance across all organ and tumor segmentation tasks. Each data point is one independent test case. The number of test cases per box, left to right, is: liver ( n = 32); spleen ( n = 20); right kidney ( n = 20); left kidney ( n = 20); gallbladder ( n = 19); esophagus ( n = 20); stomach ( n = 20); aorta ( n = 20); inferior vena cava ( n = 20); pancreas ( n = 20); right adrenal gland ( n = 20); left adrenal gland ( n = 20); duodenum ( n = 20); prostate ( n = 40); brain enhanced tumor ( n = 48); brain tumor core ( n = 57); brain whole tumor ( n = 57); gross tumor ( n = 31); metastatic tumor ( n = 24); nasopharyngeal tumor ( n = 57); breast tumor ( n = 33); liver tumor ( n = 12); colorectal tumor ( n = 31); bladder tumor ( n = 45); prostate tumor ( n = 84). f Representative visualization of segmentation results, showcasing abdominal organs and primary gross tumor volumes (GTVp) across different methods. e Presents the results as a box plot, where the lower bound, center line, and upper bound of the box correspond to the first quartile (Q1), median, and third quartile (Q3), respectively, defining the interquartile range (IQR) that contains the central 50% of the data. The whiskers extend to the furthest data points within 1.5 × IQR, and any points beyond are classified as outliers. Statistical test: two-sided paired bootstrap test (1000 bootstrap resamples); no adjustment for multiple comparisons. Source data are provided as a Source Data file. We further performed a comprehensive anatomical structure performance analysis across 25 distinct categories to validate the clinical utility of MRICombo for specific organs and tumors (Fig.  2 e). The framework exhibited robust performance in organ segmentation, consistently achieving DSC values above 95% for the liver (96.61%), left kidney (95.66%), right kidney (95.52%), and spleen (95.05%). Solid performance was also observed in other critical organs, including the aorta (91.56%), stomach (89.15%), prostate (88.31%), and pancreas (86.31%). Regarding tumor segmentation, MRICombo attained high DSC values for brain tumors, reaching 88.55% for whole brain tumors and 80.01% for tumor cores. Crucially, to address diverse clinical oncology needs, we evaluated the framework’s capability on various challenging primary and metastatic cancers. MRICombo achieved competitive DSC scores for liver tumors (64.62%), nasopharyngeal tumors (62.67%), metastatic tumors (61.17%), bladder tumors (60.25%), breast tumors (59.92%), and colorectal tumors (57.63%). Finally, in the most anatomically challenging and poorly contrasted regions, such as the prostate tumor and gross tumor volumes, the framework maintained baseline functional performance with DSCs of 36.81% and 44.21%, respectively, highlighting areas for future targeted optimization. In addition to quantitative evaluation, qualitative assessment was performed to validate segmentation quality across diverse anatomical locations. Fig.  2 f presents representative examples from two challenging scenarios: an abdominal multi-organ case from the AMOS dataset and a nasopharyngeal carcinoma (NPC) case from the NPC dataset. Visual inspection revealed that MRICombo consistently produced segmentations with high fidelity to ground truth annotations across all evaluated regions, whereas competing methods exhibited systematic errors including boundary over-segmentation and volumetric under-segmentation. Notably, alternative approaches failed to accurately delineate the tumor margins in NPC cases, likely attributable to insufficient modeling of complex spatial relationships and tissue heterogeneity in the tumor microenvironment. SAM-Med3D, despite leveraging foundation model capabilities, exhibited inconsistent performance on organ boundaries with ambiguous intensity transitions. UniMRISegNet showed improved results compared to single-task methods but still struggled with precise tumor margin delineation. In contrast, MRICombo maintained robust segmentation accuracy across both abdominal organs and head-neck tumors, demonstrating superior generalization. These qualitative findings corroborate the quantitative metrics, confirming MRICombo’s advantages for anatomically complex and clinically challenging segmentation tasks. We systematically evaluated the classification performance of MRICombo against four competing approaches—3D ResNet-specific 39 , Early Fusion, Feature Fusion, and Swin Transformer 40 —across five distinct tumor classification tasks: brain tumor subtyping (high-grade glioma vs. low-grade glioma), breast tumor characterization (benign vs. malignant), liver tumor differentiation (benign vs. malignant), bladder cancer staging (muscle-invasive vs. non-muscle-invasive), and nasopharyngeal carcinoma T-staging (T1-T2 vs. T3-T4). MRICombo demonstrated consistently superior performance across all tumor types, achieving the highest area under the receiver operating characteristic curve (AUROC) in every classification task (Fig.  3 a–e). Specifically, MRICombo attained AUROC values of 0.992 (95% CI: 0.970–1.000) for brain tumor, 0.902 (95% CI: 0.806–0.978) for breast tumor, 0.917 (95% CI: 0.834-0.980) for liver tumor, 0.926 (95% CI: 0.832-0.990) for bladder cancer, and 0.862 (95% CI: 0.753-0.948) for nasopharyngeal carcinoma, representing improvements of 0.9%, 9.0%, 19.3%, 1.5%, and 0.4% over the respective second-best methods. Complementary precision-recall analysis corroborated MRICombo’s superior diagnostic capability, yielding area under the precision-recall curve (AUPRC) values of 0.997 (95% CI: 0.989-1.000) for brain tumor, 0.950 (95% CI: 0.894-0.990) for breast tumor, 0.949 (95% CI: 0.885-0.991) for liver tumor, 0.900 (95% CI: 0.779-0.985) for bladder cancer, and 0.950 (95% CI: 0.894-0.985) for nasopharyngeal carcinoma (Fig.  3 f-j). We further benchmarked MRICombo against six radiologists (junior, middle, senior) across the same tasks (Supplementary Table  5 ). MRICombo outperformed all human readers in every diagnosis and metric, achieving mean scores of 0.992 (brain tumor), 0.917 (breast tumor), 0.889 (bladder cancer), 0.925 (liver tumor), and 0.888 (nasopharyngeal carcinoma), versus the best radiologist scores of 0.928, 0.715, 0.630, 0.809, and 0.828, respectively. Fig. 3 Performance evaluation of MRICombo for malignancy detection, invasive staging, glioma grading, and T-staging on the internal test set. MRICombo achieved competitive AUROC ( a – e ) and AUPRC ( f – j ) performance compared to four state-of-the-art methods, outperforming them in all five tumor types. It also exhibited superior performance in ( k ) MCC and ( l ) F1-score. In ( k ) and ( l ), the box center line denotes the median, the box bounds denote the first (Q1) and third (Q3) quartiles, and the whiskers extend to the most extreme data points within 1.5 × IQR of the box bounds (minimum and maximum within this range); points beyond the whiskers are outliers. Each box summarizes results across the five tumor types ( n = 5; individual points overlaid). m Decision curve analysis further confirmed the enhanced clinical utility of MRICombo across a wide threshold range. All AUROC and AUPRC curves represent 95% confidence intervals derived from 1000 bootstrap replicates. MCC Matthews correlation coefficient, NPC nasopharyngeal carcinoma, AUROC area under the receiver operating characteristic curve, AUPRC area under the precision-recall curve. Source data are provided as a Source Data file. MRICombo achieved competitive AUROC ( a – e ) and AUPRC ( f – j ) performance compared to four state-of-the-art methods, outperforming them in all five tumor types. It also exhibited superior performance in ( k ) MCC and ( l ) F1-score. In ( k ) and ( l ), the box center line denotes the median, the box bounds denote the first (Q1) and third (Q3) quartiles, and the whiskers extend to the most extreme data points within 1.5 × IQR of the box bounds (minimum and maximum within this range); points beyond the whiskers are outliers. Each box summarizes results across the five tumor types ( n = 5; individual points overlaid). m Decision curve analysis further confirmed the enhanced clinical utility of MRICombo across a wide threshold range. All AUROC and AUPRC curves represent 95% confidence intervals derived from 1000 bootstrap replicates. MCC Matthews correlation coefficient, NPC nasopharyngeal carcinoma, AUROC area under the receiver operating characteristic curve, AUPRC area under the precision-recall curve. Source data are provided as a Source Data file. Comprehensive statistical validation using Matthews correlation coefficient (MCC) and F1-score further confirmed MRICombo’s performance advantage (Fig.  3 k, l). Across the five datasets, MRICombo achieved the highest cross-dataset mean MCC of 0.759 (95% CI: 0.642-0.874) and F1-score of 0.898 (95% CI: 0.856-0.943), significantly outperforming every baseline on MCC ( p < 0.05). Among the baseline models, Feature Fusion consistently ranked as the strongest baseline across all four metrics, whereas ResNet-specific, Early Fusion, and Swin Transformer achieved broadly comparable performance. These findings demonstrate MRICombo’s robust generalization across diverse tumor types and anatomical sites. Notably, MRICombo achieved competitive performance compared to models specifically designed and optimized for individual tumor types, while employing a single unified architecture across all five tasks. Decision curve analysis revealed the clinical utility of MRICombo across threshold probabilities ranging from 0.2 to 0.9 (Fig.  3 m), demonstrating superior net benefit compared to treat-all or treat-none strategies throughout this range. Incomplete sequence conditions closely mirror real clinical scenarios where certain MRI sequences may be unavailable due to scan time constraints, patient intolerance, or equipment limitations 33 , 34 . We systematically evaluated MRICombo’s performance under all possible incomplete sequence combinations using the multi-sequence internal test set (Supplementary Table  1 ). For datasets with n available sequences, we tested all 2 n  − 2 possible incomplete combinations, excluding only the complete sequence (all present) and null sequence (all absent) scenarios. Here, n denotes the number of sequences available within a given dataset, rather than the union of all sequence types across the full study. For example, brain tumor datasets with four sequences (T1, T2, T1c, FLAIR) yielded 2 4  − 2 = 14 incomplete combinations, including single-sequence scenarios (e.g., T1 only), dual-sequence scenarios (e.g., T1+T2), and triple-sequence scenarios (e.g., T1+T2+FLAIR). This exhaustive evaluation strategy ensures comprehensive assessment of MRICombo robustness across all clinically relevant sequence availability patterns (See Supplementary Table  1 for incomplete combinations). MRICombo demonstrated robust segmentation capabilities across all incomplete sequence combinations (Fig.  4 a, b). Averaged across all incomplete sequence combinations, organ segmentation achieved a mean DSC of 86.99% compared to 88.31% with complete sequences, representing only a 1.49% relative performance decrease. Tumor segmentation showed a mean DSC of 63.00% with incomplete sequences versus 64.96% with complete sequences, indicating a 3.02% relative reduction. Overall, the segmentation performance degradation in DSC remained below 5% for both organ and tumor segmentation tasks, even in the most challenging single-sequence scenarios. Surface distance measurements corroborated these findings. MRICombo achieved an ASD of 0.26 mm across incomplete sequence combinations compared to 0.20 mm with complete sequences for organ segmentation (Fig.  4 b). Tumor segmentation exhibited an ASD of 6.69 mm for incomplete sequences versus 4.24 mm for complete sequences. The ASD increase was minimal for organ segmentation (0.06 mm) and comparatively larger for tumor segmentation (2.45 mm), consistent with the inherently greater difficulty of delineating tumor boundaries with reduced sequence information. Fig. 4 Robustness of MRICombo to missing input sequences on the internal multi-sequence test set. Performance was assessed when one or more MRI sequences were unavailable. a DSC comparison between incomplete-sequence and complete-sequence inputs for organ segmentation and tumor segmentation. b ASD comparison between incomplete-sequence and complete-sequence inputs for organ segmentation and tumor segmentation. For a and b , n = 40 independent test cases for organ segmentation and n = 336 independent test cases for tumor segmentation. c AUROC comparison between incomplete-sequence and complete-sequence inputs for four tumor classification tasks. d AUPRC comparison between incomplete-sequence and complete-sequence inputs for four tumor classification tasks. For c and d , n = 798 for brain tumor, n = 94 for breast tumor, n = 128 for liver tumor, and n = 330 for NPC. Statistical test: two-sided paired bootstrap test (1000 bootstrap resamples); no adjustment for multiple comparisons. NPC nasopharyngeal carcinoma. Source data are provided as a Source Data file. Performance was assessed when one or more MRI sequences were unavailable. a DSC comparison between incomplete-sequence and complete-sequence inputs for organ segmentation and tumor segmentation. b ASD comparison between incomplete-sequence and complete-sequence inputs for organ segmentation and tumor segmentation. For a and b , n = 40 independent test cases for organ segmentation and n = 336 independent test cases for tumor segmentation. c AUROC comparison between incomplete-sequence and complete-sequence inputs for four tumor classification tasks. d AUPRC comparison between incomplete-sequence and complete-sequence inputs for four tumor classification tasks. For c and d , n = 798 for brain tumor, n = 94 for breast tumor, n = 128 for liver tumor, and n = 330 for NPC. Statistical test: two-sided paired bootstrap test (1000 bootstrap resamples); no adjustment for multiple comparisons. NPC nasopharyngeal carcinoma. Source data are provided as a Source Data file. The classification performance of MRICombo remained competitive across incomplete sequence combinations, although statistically significant differences between incomplete and complete sequences were observed across tumor types (Fig.  4 c, d). Averaged across all incomplete sequence combinations, AUROC measurements for complete versus incomplete sequences were as follows: brain tumor (0.992 vs. 0.914), breast tumor (0.902 vs. 0.833), nasopharyngeal carcinoma (0.862 vs. 0.796), and liver tumor (0.917 vs. 0.832) (Fig.  4 c). Corresponding AUPRC values showed similar trends: brain tumor (0.997 vs. 0.963), breast tumor (0.950 vs. 0.871), nasopharyngeal carcinoma (0.950 vs. 0.904), and liver tumor (0.949 vs. 0.880) (Fig.  4 d). The average AUROC and AUPRC reductions across all tumor types were 8.11% and 5.96%, respectively, when averaged over all incomplete sequence combinations. Breast tumor and nasopharyngeal carcinoma classifications demonstrated the smallest performance gaps (7.65% and 7.66% AUROC reduction), while liver tumor classification exhibited the largest decrease (9.27% AUROC reduction). Brain tumor classification showed an intermediate reduction of 7.86%. Performance retention analysis revealed differential sequence dependencies across anatomical regions. When averaged across incomplete sequence combinations, brain tumor classification retained 92.1% of complete sequence AUROC performance, breast tumor retained 92.4%, nasopharyngeal carcinoma preserved 92.3%, and liver tumor sustained 90.7% of original performance levels. These results demonstrate MRICombo’s resilience to incomplete input data across diverse sequence availability patterns, a critical advantage for real-world clinical deployment. Figure  5 a presents the t-SNE visualization of the sequence feature extraction module. It can be observed that the sequence feature extraction experts map different anatomical regions with the same sequence type into a unified feature space. Notably, distinct clustering effects are evident for all sequence types except T1, which did not achieve satisfactory clustering performance. Figure  5 b further reveals the inherent heterogeneity of multi-sequence datasets (BraTS, NPC, ProstateX, csPCa, ISPY), showing significant feature space discrepancies between different sequences from the same organ. Figure  5 c illustrates how our fusion module effectively integrates information from heterogeneous sequences into a common feature space. This integration enables the model to focus more effectively on semantic distinctions within the images. After processing by the fusion module, inter-sequence discrepancies are substantially reduced, and features primarily cluster according to their organ origins rather than acquisition protocols. Fig. 5 Visualization of feature representations and expert activation patterns. a t-SNE visualization of sequence-specific feature representations across all segmentation training data. b t-SNE visualization of multi-sequence features from different anatomical regions across five datasets. c t-SNE visualization of the adaptive multi-sequence feature fusion module on five datasets. d Expert weight distribution across encoder layers (Encoder 0 --Encoder 3 ) for 15 tasks (5 classification and 10 segmentation). e Expert weight distribution across encoder layers (Encoder 0 --Encoder 3 ) for 10 segmentation tasks. All analyses employ the Top-2 expert activation strategy, with color coding indicating normalized weights of the four experts. Each subplot corresponds to an individual task, where the horizontal axis represents network layers and the vertical axis shows expert weight proportions. E0--E3 denote Encoder 0 --Encoder 3 . a t-SNE visualization of sequence-specific feature representations across all segmentation training data. b t-SNE visualization of multi-sequence features from different anatomical regions across five datasets. c t-SNE visualization of the adaptive multi-sequence feature fusion module on five datasets. d Expert weight distribution across encoder layers (Encoder 0 --Encoder 3 ) for 15 tasks (5 classification and 10 segmentation). e Expert weight distribution across encoder layers (Encoder 0 --Encoder 3 ) for 10 segmentation tasks. All analyses employ the Top-2 expert activation strategy, with color coding indicating normalized weights of the four experts. Each subplot corresponds to an individual task, where the horizontal axis represents network layers and the vertical axis shows expert weight proportions. E0--E3 denote Encoder 0 --Encoder 3 . As shown in Fig.  5 d, e, the expert activation patterns of our framework reveal a consistent routing strategy characterized by shared components and specialized adaptations. These adaptation behaviors are orchestrated by the dual-prompt mechanism of our framework, where the dataset prompt and task prompt collectively guide the expert selection process. A detailed comparison between classification and segmentation tasks demonstrates a shared-yet-specialized routing behavior across encoder layers. In the NPC dataset, for example, both tasks activate Expert 2 in Encoder 0 –Encoder 1 , while their respective task prompts drive specialization: classification engages Expert 3 whereas segmentation selects Expert 4. This pattern persists through deeper layers, with Encoder 2 sharing Expert 1 but differentiating through complementary experts (Expert 3 for classification versus Expert 2 for segmentation), and Encoder 3 sharing Expert 2 while specializing with Expert 4 (classification) versus Expert 3 (segmentation). This structured scheme, where dataset prompts establish common feature extraction pathways while task prompts enable specialized processing, is consistently observed across brain tumor and bladder cancer datasets. This layered routing strategy demonstrates that our framework does not simply assign disjoint sets of experts to different tasks. Instead, at each hierarchical level, through the coordinated action of dataset and task prompts, it preserves a common expert for task-agnostic feature extraction while dynamically recruiting specialized experts to fulfill task-specific requirements. Such a design elegantly balances specialization with efficient cross-task representation sharing. Beyond task-level commonality, anatomically adjacent organ regions also exhibit strong consistency in expert selection, reflecting biologically-grounded coherence. As shown in Fig.  5 e, abdominal datasets including AMOS and ATLAS display highly aligned expert activation patterns. This trend is biologically intuitive since organs within the abdominal cavity such as the liver, kidneys and spleen share tissue properties, developmental origins, and imaging characteristics, which encourages the MoE router to adopt common feature extraction strategies. A similar tendency is observed in pelvic regions including bladder cancer, ProstateX and csPCa, as well as head and neck regions including NPC, HNTS and BraTS. These findings indicate that the MoE framework not only learns from task-level supervision but also implicitly captures intrinsic anatomical and imaging relationships, thereby autonomously organizing its representations in a biologically meaningful manner. Our framework demonstrates robust generalization capabilities across both segmentation and classification tasks. As shown in Fig.  6 a, the segmentation performance evaluation across three organ types reveals minimal performance degradation when transferring from internal to external datasets. Specifically, liver segmentation achieves a DSC of 96.61% on internal data and 91.42% on external data, maintaining a 94.6% performance retention rate. Brain tumor segmentation shows excellent stability with scores of 88.55% vs 86.95% (98.2% retention), while breast tumor segmentation demonstrates positive transfer with an improvement from 59.92% to 66.05%. These findings confirm that the framework maintains stable segmentation accuracy across different anatomical structures and imaging protocols in external validation datasets. Fig. 6 Generalization performance comparison on internal and external test sets. a Segmentation generalizability performance across three target types, liver, brain tumor, and breast tumor, measured by DSC. For a , internal test cases were liver ( n = 32), brain tumor ( n = 57), and breast tumor ( n = 33), while external test cases were liver ( n = 506), brain tumor ( n = 60), and breast tumor ( n = 64). b Classification generalizability performance for liver tumor benign/malignant classification measured by AUROC and AUPRC. For b , n = 64 and n = 104 independent test cases were used for the internal and external test sets, respectively. DSC Dice similarity coefficient, AUROC area under the receiver operating characteristic curve, AUPRC area under the precision-recall curve. Source data are provided as a Source Data file. a Segmentation generalizability performance across three target types, liver, brain tumor, and breast tumor, measured by DSC. For a , internal test cases were liver ( n = 32), brain tumor ( n = 57), and breast tumor ( n = 33), while external test cases were liver ( n = 506), brain tumor ( n = 60), and breast tumor ( n = 64). b Classification generalizability performance for liver tumor benign/malignant classification measured by AUROC and AUPRC. For b , n = 64 and n = 104 independent test cases were used for the internal and external test sets, respectively. DSC Dice similarity coefficient, AUROC area under the receiver operating characteristic curve, AUPRC area under the precision-recall curve. Source data are provided as a Source Data file. For the liver tumor benign/malignant classification task, our framework exhibits similar performance as illustrated in Fig.  6 b. The AUROC metric shows a decrease from 0.917 on internal data to 0.841 on external data (8.3% degradation), while AUPRC decreases from 0.949 to 0.849 (10.5% degradation). These results indicate that despite potential differences in imaging parameters, patient demographics, and data acquisition protocols in external datasets, our framework maintains high discriminative capability for tumor characterization. The consistent performance across both tasks demonstrates the effectiveness of our multi-task learning framework in learning generalizable feature representations. The cross-domain stability in segmentation tasks confirms that the learned features possess good transferability across different clinical settings, while the robust classification performance validates the framework’s practical utility for real-world deployment. These findings support the clinical applicability and reliability of our proposed unified framework across diverse medical imaging scenarios. Table  1 compares the generalization performance of different models on unseen hippocampus segmentation data under both from-scratch and transfer learning settings. Among the task-specific models trained from scratch, nnUNet establishes a strong fully-supervised benchmark with an average DSC of 88.70%, while SwinUNETR 36 and TransUNet 37 achieve 86.03% and 86.73%, respectively. In the transfer learning setting, the general-purpose model SAM-Med3D 20 achieves only 58.85% in average DSC, significantly lagging behind the specialized models. However, both UniMRISegNet 38 and our proposed MRICombo exhibit robust transfer learning capabilities. Specifically, MRICombo attains the highest performance among all evaluated general-purpose models with an average DSC of 87.13%, representing a 1.00% improvement over UniMRISegNet. Remarkably, despite being adapted via transfer learning rather than being trained from scratch on this specific dataset, our MRICombo outperformed fully trained task-specific architectures like SwinUNETR and TransUNet, and yields highly competitive performance approaching the highly optimized nnUNet baseline. Table 1 Performance Comparison on Unseen Hippocampus Segmentation Data (DSC, %) Setting Model Anterior hippocampus Posterior hippocampus Average Scratch nnUNet 89.13 88.26 88.70 SwinUNETR 87.10 84.96 86.03 TransUNet 88.10 85.36 86.73 Transfer SAM-Med3D 60.99 56.70 58.85 UniMRISegNet 87.42 84.84 86.13 MRICombo (ours) 88.32 85.94 87.13 Performance Comparison on Unseen Hippocampus Segmentation Data (DSC, %) We conducted a comprehensive ablation study to systematically evaluate the individual and synergistic contributions of the adaptive fusion, MoE, and MFR modules within our proposed framework. All experiments were performed using a universal single-task architecture as the baseline to ensure fair comparison. As shown in Table  2 , our proposed modules demonstrate substantial performance improvements over this baseline. Table 2 Ablation study of MRICombo showing negative transfer in naive multi-task (unified) learning Method Classification (AUROC) Average Segmentation (DSC, %) Average Brain Liver Breast Bladder Nasopharynx Organs Tumors Classification Baseline (unified) 0.971 0.665 0.810 0.881 0.838 0.833 – – – Segmentation Baseline (unified) – – – – – – 79.90 59.58 69.74 Multi-task (unified) 0.962 0.521 0.785 0.883 0.832 0.797 77.15 56.28 66.72 Multi-task (unified) + Adaptive Fusion 0.983 0.755 0.815 0.892 0.841 0.857 80.06 60.02 70.04 Multi-task (unified) + MoE 0.986 0.835 0.865 0.908 0.839 0.887 81.80 61.92 71.86 MRICombo w/o MFR 0.994 0.898 0.900 0.923 0.848 0.913 83.60 61.74 72.67 MRICombo 0.992 0.917 0.902 0.926 0.862 0.920 83.61 62.50 73.06 Classification performance is measured by AUROC, while segmentation performance is evaluated using DSC (%). Ablation study of MRICombo showing negative transfer in naive multi-task (unified) learning Classification performance is measured by AUROC, while segmentation performance is evaluated using DSC (%). The universal single-task baselines established competitive performance benchmarks: the classification baseline achieved an AUROC of 0.833, while the segmentation baseline attained an average DSC of 69.74%. Joint training with a shared backbone network, however, led to performance degradation across both tasks. The average classification AUROC decreased to 0.797 (a 3.6% absolute reduction), while the average segmentation DSC dropped to 66.72%, with tumor segmentation declining to 56.28% DSC (a 3.0% absolute reduction from the single-task baseline). This deterioration stems from gradient conflicts between these heterogeneous tasks. Segmentation, particularly for precise tumor boundary delineation, demands localized, fine-grained spatial features, whereas classification relies on global, context-aware semantic representations. When constrained to share a common encoder, gradients from competing objectives propagate in divergent directions, forcing a suboptimal compromise. The introduction of adaptive fusion alone showed moderate recovery in classification performance (AUROC: 0.857). However, tumor segmentation remained suboptimal (60.02% DSC) compared with the single-task baseline, indicating that adaptive fusion alone cannot fully resolve the fundamental feature representation conflicts between tasks. The incorporation of the MoE module effectively addressed this limitation. The Multi-task + MoE configuration not only recovered performance losses but substantially exceeded all baselines, achieving an average AUROC of 0.887 and an average DSC of 71.86%. Combining adaptive fusion and MoE processing (denoted as MRICombo w/o MFR) generated highly synergistic effects, elevating the average classification AUROC to 0.913 and the average segmentation DSC to 72.67%. Finally, the integration of the Masked Feature Reconstruction (MFR) module completes the proposed MRICombo framework, achieving the highest overall performance with an average classification AUROC of 0.920 and an average segmentation DSC of 73.06%. Crucially, the addition of the MFR module noticeably improves the highly challenging tumor segmentation task, raising its DSC from 61.74% (w/o MFR) to 62.50%. This verifies our hypothesis that reconstructing masked deep features forces the network to learn robust, cross-sequence commonalities, thereby mitigating the information loss from heterogeneous modalities and establishing MFR as a definitive component for comprehensive multi-task learning. In summary, the proposed modules deliver substantial improvements over the universal baseline. While adaptive fusion and MoE provide critical task specialization and feature integration, the MFR mechanism acts as a powerful catalyst for extracting universal representations. Their complete integration yields a robust universal model capable of excelling at both classification and segmentation tasks across diverse anatomical regions. As shown in Table  3 , we compare the computational efficiency of different models on the segmentation task, categorizing them into task-dependent models (top section) and unified models (bottom section). The computational resource requirements of task-dependent models scale linearly with the number of tasks N , whereas unified models maintain fixed resource usage independent of N . Table 3 Comparison of computational complexity across different models in segmentation task Model GPU Memory Usage (G) Params (M) Inference Time (s) nnUNet 7.2 ×  N 30.8 ×  N 0.78 SwinUNETR 13.5 ×  N 62.1 ×  N 0.95 TransUNet 11.0 ×  N 41.5 ×  N 0.82 SAM-Med3D 8.2 100.5 3.32 UniMRISegNet 9.1 14.2 0.58 MRICombo (ours) 15.9 118.2 0.53 Comparison of computational complexity across different models in segmentation task A specific comparison with established task-specific architectures like nnUNet, SwinUNETR, and TransUNet underscores the advantages of our proposed MRICombo in multi-task deployment scenarios. While a single efficient nnUNet model requires 7.2 G of GPU memory and 30.8 M parameters, deploying multiple such task-specific models results in linearly growing resource demands—specifically, 7.2 ×  N G memory and 30.8 ×  N M parameters. Heavier transformer-based models like SwinUNETR impose even steeper scaling costs (13.5 ×  N G memory and 62.1 ×  N M parameters). In contrast, MRICombo maintains a constant computational footprint of 15.9 G GPU memory and 118.2 M parameters, regardless of the number of tasks, achieving substantial resource savings as the number of tasks scales (e.g., GPU memory savings start at N ≥3 even when compared to the lightweight nnUNet) (in this study, N  = 15). Among unified models, MRICombo requires moderately more GPU memory (15.9 G) and parameters (118.2 M) than SAM-Med3D (8.2 G, 100.5 M) and UniMRISegNet (9.1 G, 14.2 M), but achieves faster inference (0.53 s vs. 3.32 s and 0.58 s) while supporting both segmentation and classification tasks simultaneously. Notably, MRICombo’s inference time of 0.53 s is substantially faster than all evaluated task-specific models, including nnUNet (0.78 s), TransUNet (0.82 s), and SwinUNETR (0.95 s), while concurrently providing comprehensive multi-task capability. This superior inference speed, combined with fixed resource costs, greatly reduces the need to maintain and execute multiple separate models, thereby significantly simplifying the deployment architecture and reducing system complexity. Moreover, such a low-footprint model is essential for enabling direct integration and real-time execution on MRI scanners, where hardware resources are often constrained and computational headroom is limited.

Discussion

We introduce MRICombo, a unified multi-task deep learning framework for heterogeneous clinical MRI analysis. It performs universal anatomical segmentation, tumor localization, grading, staging, and malignancy detection across multiple organ systems within a single architecture. Unlike conventional task-specific or universal models, MRICombo employs a dual-prompt guided mixture-of-expert (MoE) architecture where dataset prompts and task prompts dynamically modulate expert routing based on task types and clinical objectives. This design stems from the insight that diverse MRI sequences and disease manifestations share underlying anatomical and imaging patterns, which enables the learning of highly transferable representations across diseases and body regions. Our results show that MRICombo achieves state-of-the-art performance across diverse tasks (Figs.  2 – 4 and 6 ), while exhibiting notable robustness to incomplete imaging sequences, a critical requirement for potential real-world clinical deployment. A distinguishing feature of MRICombo is its resilience to incomplete sequence availability. Clinical MRI protocols are inherently heterogeneous due to scan time constraints, patient intolerance, equipment variations, and institutional preferences 32 – 34 . MRICombo maintained high diagnostic accuracy with minimal performance decline, showing an average AUROC reduction ~8% and an average DSC reduction below 5% under missing sequence conditions. This robustness is clinically significant, as it enables the framework to accommodate existing clinical workflows without requiring protocol standardization, a major barrier to artificial intelligence adoption in routine practice. Beyond missing sequences, the framework is also highly resilient to the inherent variability of acquired imaging. Specifically, variations in slice thickness and sequence protocols can alter anatomical representations and intensity distributions, potentially impacting feature extraction. While our framework maintains stable performance across diverse settings (Supplementary Tables  6 and 7 ), we acknowledge that residual domain shifts may still exist and could influence performance in extreme scenarios. Architecturally, MRICombo offers fundamental insights into multi-task learning principles in medical imaging. Our analysis reveals a sophisticated shared-yet-specialized routing mechanism that operates hierarchically across network layers. Rather than employing disjoint expert sets for different tasks, the framework maintains a core of shared experts for fundamental feature extraction while dynamically engaging specialized experts to address task-specific requirements. This balanced approach enables efficient knowledge sharing across tasks while preserving the unique characteristics needed for different clinical objectives, resulting in a more robust and scalable architecture for diverse clinical tasks. Beyond task-level optimization, MRICombo demonstrates biologically-grounded coherence in its learned representations. The framework spontaneously organizes its expert selection patterns according to anatomical relationships, with adjacent organ regions consistently activating similar expert combinations. This emergent property indicates that the framework successfully captures the intrinsic biological continuity of human anatomy, developing feature representation strategies that respect real-world anatomical organization. Such biologically meaningful organization not only enhances framework interpretability but also suggests that the framework learns representations aligned with fundamental anatomical principles. Furthermore, MRICombo effectively processes multi-sequence data through a dedicated architecture. Specialized sequence modules extract unified features within each sequence type, while an adaptive fusion module synthesizes these heterogeneous inputs into a semantically coherent representation. This process emphasizes diagnostically relevant information, enabling the framework to operate robustly across diverse imaging protocols and institutional practices. From a translational perspective, MRICombo offers practical advantages for clinical integration. Computational analysis demonstrated that for applications exceeding three tasks, encompassing 15 tasks in this study, the unified framework substantially reduces storage and computational requirements compared to deploying multiple task-specific models, while maintaining comparable inference speeds. This efficiency is particularly relevant for deployment scenarios including embedded scanner implementations for real-time analysis, cloud-based teleradiology platforms, and resource-limited healthcare settings. By consolidating multiple diagnostic functions, the framework may reduce repetitive workflow burdens on radiologists, enabling greater focus on complex clinical reasoning and patient management. Several limitations warrant consideration. First, the dataset’s sequence imbalance, with a predominance of T2 imaging, may hinder performance on tasks requiring underrepresented sequences like DWI for prostate or multi-phase liver assessment. Second, although our stratified analysis demonstrates that MRICombo attains both the highest Dice and the lowest ASD for small prostate lesions (Supplementary Table  8 ), the segmentation of small lesions remains inherently challenging due to low contrast and limited volume, an issue common across current methodologies. Finally, our retrospective comparison against individual experts does not capture the consensus-driven environment of a multidisciplinary tumor board. Furthermore, the framework currently relies solely on imaging data; future prospective studies should aim to validate the model in real-world clinical workflows and augment it with clinical metadata to enhance its diagnostic and prognostic utility. In summary, this work demonstrates the feasibility of a unified framework capable of performing multiple segmentation and classification tasks across heterogeneous MRI sequences, reducing the reliance on task-specific models. The proposed framework achieves performance comparable to task-specific models on the binary classification tasks evaluated, suggesting that unified approaches may offer a promising alternative to traditional “one-task-one-model” designs. Future work remains necessary to validate its applicability to more complex, multi-dimensional oncologic decision-making in real-world clinical settings.

Introduction

Magnetic Resonance Imaging (MRI) has been an indispensable clinical diagnostic tool owing to its superior soft tissue contrast and absence of ionizing radiation 1 . Multi-sequence MRI is widely employed in disease diagnosis, as different MRI sequences provide complementary information regarding the structural and functional details of organs 2 . For instance, T1-weighted (T1) sequences are effective in visualizing anatomical structures with excellent spatial resolution, while T2-weighted (T2) and Fluid Attenuated Inversion Recovery (FLAIR) sequences exhibit high sensitivity to pathological changes such as edematous tissues. The integration of these diverse MRI sequences yields comprehensive radiological data that significantly enhances diagnostic accuracy and clinical decision-making 3 . In clinical practice, multi-sequence MRI enables radiologists to routinely perform complex tasks. For example, they can distinguish malignant tumors from benign ones based on distinct imaging patterns and integrate information from multiple sequences to accurately recognize the boundaries of target regions. However, these manual procedures are time-consuming and exhibit limited reproducibility due to inter-observer variability and subjective interpretation. Specifically, malignant tumor differentiation and tumor grading traditionally require invasive histopathological examination as the “gold standard," which increases both the complexity and duration of diagnostic workflows and may delay the initiation of optimal treatment. In contrast, deep learning-based methods have demonstrated technological advantages in multi-modality image analysis, offering not only automated efficiency but also high accuracy and consistency. For instance, the Medical Segmentation Decathlon benchmark has demonstrated that deep learning models can achieve near-expert performance in multi-organ/tumor segmentation. The tasks in this benchmark include the segmentation of the heart, prostate, liver, and pancreas, among others 4 . In tumor classification, another deep learning-based method achieves 91.0% accuracy in benign and malignant differentiation surpassing a traditional method, which yields an accuracy of only 86.4%. Consistent breakthroughs have been reported in similar tasks such as brain tumor grading 5 , 6 and breast cancer detection 7 . Despite these advances, current deep learning-based segmentation or classification models exhibit fundamental limitations that restrict clinical translation. Most segmentation models design dedicated modules for specific organs (e.g., liver 8 , kidney 9 , pancreas 10 ) or specific tumor types (e.g., glioma 11 , hepatocellular carcinoma 12 , colorectal cancer 13 ). While these models perform well on specific tasks, they lack cross-organ and cross-disease generalization capabilities. In classification tasks, existing research primarily focuses on benign-malignant differentiation for specific anatomical sites (e.g., breast masses 14 , liver lesions 15 ) or grading/typing tasks for specific tumors, with relatively narrow model applicability. In clinical applications, the deployment of such models typically follows a “one-task-one-model" development workflow, which increases demands on computational resources, storage, and maintenance. It poses significant challenges for adapting to the diverse and evolving requirements of clinical diagnosis. For example, breast tumor diagnosis requires concurrent tumor segmentation 16 , classification/grading 17 , and prognostic assessment, all of which single-task models cannot accommodate. More importantly, existing deep learning methods predominantly focus on single-task objectives (segmentation or classification), whereas in clinical diagnostic practice, radiologists typically need to simultaneously perform multiple interrelated analytical tasks. Therefore, there is an urgent need for a unified multi-task framework that integrates multiple interrelated analytical tasks such as segmentation and classification. To address these challenges, medical image analysis is evolving toward generalization and multi-task approaches. In segmentation, two-dimensional universal segmentation models such as MedSAM 18 and SAM-Med2D 19 have demonstrated strong performance, but universal segmentation of three-dimensional volumetric images remains more complex due to high annotation costs and computational resource intensity. Although three-dimensional universal segmentation models like SAM-Med3D 20 have been developed, they often require multiple interactions to achieve optimal segmentation. However, this is particularly true for complex anatomical structures, and these models lack the efficiency necessary for large-scale automated analysis. In classification, self-supervised learning (SSL) provides a pathway to universal models by leveraging large unlabeled datasets to learn powerful representations, thus diminishing the need for manual annotation. However, prevailing SSL-based strategies 21 – 24 remain constrained by a two-stage paradigm of pre-training followed by task-specific fine-tuning. This paradigm not only results in a discernible performance gap between stages but also scales inefficiently. Since each downstream task necessitates a separately fine-tuned model, the deployment footprint increases in a manner directly proportional to the number of tasks, thereby failing to deliver a unified “one-model-multi-task" solution. Existing multi-task work 9 , 15 , 25 – 29 , while capable of simultaneously handling segmentation and classification, is limited to single anatomical site task combinations, unable to meet the complex diagnostic scenario requirements of multi-organ and multi-disease coexistence in clinical practice. Recent advances further refine specialized pipelines for individual organs and sequence settings 30 , 31 , yet a unified, multi-organ, multi-sequence framework remains lacking. More problematically, missing sequence issues are prevalent in clinical practice. Brain tumor diagnosis typically requires complete T1, T2, T1c, and FLAIR sequences, yet sequence data is frequently missing due to heterogeneity in multi-institutional datasets, including variations in acquisition parameters, scanner protocol, time constraints, and image artifacts such as patient motion 32 – 34 . Under these realistic constraints, maximizing the diagnostic information value of available sequences to achieve performance comparable to complete sequences has become a critical challenge in imaging medicine. Here, we present MRICombo, a unified deep-learning-based framework that enables simultaneous anatomical and tumor segmentation, grading, staging, and malignancy detection from arbitrary combinations of available MRI sequences. A fundamental insight driving our approach is that diverse MRI sequences and disease types share common imaging features, enabling the learning of generalized representations across tasks. To exploit this property, we introduce a Prompt-Reconstructed Mixture-of-Experts (PR-MoE) architecture that integrates sequence-specific feature extraction, dual-prompt gating (with dataset and task prompts), and a Masked Feature Reconstruction (MFR) branch to enable synergistic multi-task learning. When evaluated on 12 datasets used for internal development and evaluation, MRICombo achieved state-of-the-art performance across multiple metrics. It also demonstrated strong generalization on four independent external datasets, comprising three segmentation tasks and one classification task, and effective transfer learning on two additional unseen tasks involving anatomical segmentation and tumor classification. Moreover, the MRICombo retained robust accuracy in missing-sequence scenarios. Collectively, these results validate the MRICombo’s strong task handling, generalization, and transfer learning capabilities, illustrating the potential of unified architectures in oncologic imaging applications.

Supplementary Material

Source Data Source Data

Text is read by the "Ask this paper" AI Q&A widget below. Extraction quality varies by source — PMC NXML preserves structure cleanly, OA-HTML may include some navigation residue, and OA-PDF can have broken hyphenation. The publisher copy (via DOI) is the canonical version.

My notes (saved in your browser only)

Ask this paper AI returns verbatim quotes from the full text · source: pmc-nxml

Answers must be backed by verbatim quotes from this paper's full text. Hallucinated quotes are dropped automatically; if no verbatim passage answers the question, we say so. How this works

Citation neighborhood (no data yet)

We don't have any in-corpus citations linked to this paper yet. This is a recent paper (2026) — citers typically take a year or two to land, and the OpenAlex reference graph may still be filling in.

Source provenance

europepmc
last seen: 2026-09-20T09:27:46.357103+00:00