Multimodal machine learning integrates clinical and comorbidity data to predict breast cancer prognosis and treatment outcomes.

The foundation of this study lies in the Charité Breast Cancer Dataset, a meticulously curated repository of 1,727 ambulatory patient records collected prospectively between November 2016 and March 2021 at the Charité Universitätsmedizin Berlin. This dataset uniquely harmonizes structured clinical variables, binary comorbidity flags, and granular patient-reported outcome (PRO) metrics, enabling a multidimensional exploration of breast cancer prognosis. Clinical data encompass tumor-specific attributes such as histopathological subtypes (invasive ductal/lobular carcinoma, DCIS), histological grade (Grade 1–3), receptor status (ER/PR/HER2), and surgical history (pre_op). Comorbidity information is captured through 14 binary indicators (e.g., comorb_diabetes, comorb_uti), reflecting conditions ranging from metabolic disorders to urinary tract infections. PROs derive from the EORTC QLQ-C30 and BR23 modules, quantifying domains such as global health status (ql), fatigue (fa), and breast-specific symptoms ( brst , brbi ). All comorbidity flags and PRO scores were recorded at baseline—prior to systemic therapy—and treated as static covariates throughout the modeling pipeline, consistent with the clinical objective of pre-treatment risk stratification. Sociodemographic variables—including age, marital status, education level, and body mass index (BMI)—further enrich the dataset, capturing systemic influences on outcomes. A pivotal strength of this dataset is its prospective design, which ensures temporal alignment between PRO assessments and clinical evaluations. Patients completed surveys via a web-based system (Heartbeat Medical) on tablets in the outpatient clinic waiting area, minimizing recall bias and ensuring standardized data capture. All responses were documented in German, reflecting the linguistic homogeneity of the cohort. Notably, the dataset’s inclusion criteria prioritize heterogeneity in breast disease diagnoses (early breast cancer, DCIS, fibroadenoma, other breast diseases) while excluding patients seeking second opinions or diagnosed post-treatment initiation. This design facilitates analysis of pre-treatment health status, a critical window for risk stratification. The dataset’s granularity is exemplified in Table 1 , which categorizes variables by modality and operationalization. Clinical variables exhibit diverse data types: age (continuous), her2status (categorical: negative/positive/ambiguous), and gradeinv (ordinal: Grade 1–3). Comorbidity flags, such as comorb_diabetes and comorb_uti , encode presence/absence as binary values, enabling quantitative analysis of their prognostic impact. PROs, measured via EORTC modules, yield continuous scores normalized to a 0–100 scale, where higher values denote better functional status (e.g., ql for global health) or worse symptom burden (e.g., fa for fatigue). Sociodemographic variables like marital_status and education adopt nominal scales, while bmi is calculated from weight and height measurements. Table 1 Dataset Overview and Variable Types. Variabllame Description Data type Example values Modality age Patient age at registration (years) Continuous 25–81 Sociodemographic her2status HER2 receptor status Categorical 0 (negative), 1 (positive), 2 (ambiguous) Clinical gradeinv Histological tumor grade Ordinal 0 (Grade 1), 1 (Grade 2), 2 (Grade 3) Clinical bmi Body mass index (kg/m 2 ) Continuous 17.44–43.29 Sociodemographic comorb_diabetes Diabetes diagnosis (0 = No, 1 = Yes) Binary 0, 1 Comorbidity comorb_uti Urinary tract infection diagnosis (0 = No, 1 = Yes) Binary 0, 1 Comorbidity comorb_depression Depression diagnosis (0 = No, 1 = Yes) Binary 0, 1 Comorbidity ql EORTC QLQ-C30 global health status (0–100 scale) Continuous 16.7–100.0 PRO fa EORTC QLQ-C30 fatigue score (0–100 scale) Continuous 0.0–88.9 PRO brst EORTC BR23 systemic therapy side effects (0–100 scale) Continuous 0.0–91.7 PRO marital_status Marital status (0 = single, 1 = married, 2 = divorced/separated, 3 = widowed) Nominal 0, 1, 2, 3 Sociodemographic education Education level (1 = low, 2 = middle, 3 = high) Ordinal 1, 2, 3 Sociodemographic pre_op Prior breast surgery (0 = No, 1 = Yes) Binary 0, 1 Clinical Dataset Overview and Variable Types. 0 (negative), 1 (positive), 2 (ambiguous) 0 (Grade 1), 1 (Grade 2), 2 (Grade 3) This multimodal structure addresses a critical gap in precision oncology: the reconciliation of tumor biology with systemic patient health. For instance, while her2status informs targeted therapy eligibility, its interplay with comorbidities (e.g., comorb_hypertension) and PROs (e.g., brst for systemic therapy side effects) reveals nonlinear risk patterns overlooked in conventional models. Similarly, bmi—often treated as a continuous covariate—interacts synergistically with conditions like diabetes (comorb_diabetes) to modulate treatment tolerance. By integrating these dimensions, the dataset enables machine learning pipelines to model complex interactions, such as the differential impact of urinary tract infections (comorb_uti) on recurrence-free survival across receptor subtypes. Table 1 illustrates the multimodal integration of structured clinical parameters, binary comorbidity flags, and continuous PRO scores, underscoring the dataset’s capacity to model nonlinear interactions (e.g., age × hypertension). The dataset’s richness is evident in its variable taxonomy. Sociodemographic metrics like age and education span decades of life experience and socioeconomic gradients, both established predictors of treatment adherence and survival. Clinical variables such as her2status and gradeinv reflect tumor biology, while pre_op captures prior interventions that may influence therapeutic sequencing. Comorbidity flags, including comorb_diabetes and comorb_uti, encode systemic health burdens often omitted in traditional models. PROs like ql (global health status) and brst (systemic therapy side effects) provide subjective insights into pre-treatment resilience. For example, a patient aged 54 with invasive ductal carcinoma ( histotype  = 1), HER2-positive status ( her2status  = 1), and a ql score of 58.3 (moderate global health) exemplifies the interplay between tumor aggressiveness and patient-reported well-being. Similarly, a 78-year-old with comorb_uti  = 1 and comorb_depression  = 1 highlights the compounding risks of chronic inflammation and emotional distress. These examples underscore the dataset’s ability to operationalize heterogeneous patient profiles, a prerequisite for training machine learning models that transcend unimodal analysis. The dataset’s design also addresses methodological challenges in missingness and scale normalization. Missing values in bmi (3% of records) and cupsize (12% missing) were imputed via MICE (Multivariate Imputation by Chained Equations), preserving statistical power. Continuous PRO scores (e.g., fa , brst ) underwent z-score scaling to harmonize their 0–100 metric with clinical variables, while categorical features like marital_status were one-hot encoded. This preprocessing ensures compatibility with machine learning algorithms, which demand numerical consistency across modalities. Crucially, the dataset’s integration of interaction terms (e.g., age × hypertension) and comorbidity burden scores (sum of positive comorbidity flags) enables modeling of synergistic effects, such as the exacerbation of chemotherapy toxicity in older patients with gastrointestinal comorbidities ( comorb_gastrointestinal  = 1). By embedding these refinements into the dataset’s architecture, the study bridges the gap between raw clinical records and algorithm-ready features, setting a precedent for scalable, real-world applications in precision oncology. The Charité Breast Cancer Dataset, comprising 1727 patient records, necessitated rigorous preprocessing to ensure compatibility with machine learning pipelines and to enhance reproducibility. A critical challenge arose from missing values, particularly in continuous variables such as bmi (3% missing) and cupsize (12% missing). To address this, we employed Multivariate Imputation by Chained Equations (MICE), a robust iterative algorithm that models each variable’s missingness as a function of other covariates, preserving multivariate relationships. For instance, missing bmi values were imputed using predictors like age, height, and weight, while missing ql (global health status) scores were estimated via correlations with ef (emotional functioning) and fa (fatigue). This approach outperformed simpler methods like mean imputation, which risk distorting distributions, as evidenced by preliminary analyses showing a 15% reduction in variance inflation for bmi post-MICE. A pivotal innovation in preprocessing involved feature engineering to capture nonlinear interactions and systemic comorbidity effects. First, we derived comorbidity burden scores by summing binary flags across 14 conditions (e.g., comorb_diabetes, comorb_uti), quantifying cumulative health strain. This metric enabled modeling of dose–response relationships, such as a 10% increase in recurrence risk per additional comorbidity. Second, we introduced interaction terms to probe synergistic effects, exemplified by age × hypertension, which revealed that older patients with hypertension faced disproportionately higher risks of chemotherapy toxicity compared to younger counterparts. Similarly, education × depression highlighted disparities in PRO resilience among low-income populations. These engineered features were validated through correlation matrices, where age × hypertension exhibited a 0.38 Pearson coefficient with fatigue scores (fa), underscoring its relevance to systemic health. Normalization further refined the dataset’s analytical utility. Continuous variables like bmi and EORTC QLQ-C30 scores (ql, fa) underwent z-score scaling to align their distributions, mitigating bias toward features with larger magnitudes. For example, ql scores (0–100 scale) and bmi (17.44–43.29 range) were transformed to zero-mean, unit-variance metrics, ensuring equal weighting in distance-based algorithms (e.g., k-nearest neighbors). Categorical variables, including marital_status and her2status , were one-hot encoded, while ordinal variables like education retained their integer encodings to preserve rank information. Notably, tumor grade (gradeinv: Grade 1–3) was treated as an ordinal feature, contrasting with prior studies that dichotomized grades into low/high-risk categories, thereby losing granularity. The preprocessing pipeline’s rigor is encapsulated in Table 2 , which details the sequence of transformations applied to diverse data types. For example, the comorb_uti flag (binary) was paired with its interaction term age  ×  comorb_uti , revealing that urinary tract infections amplified recurrence risk by 1.2 × in patients over 60—a finding later validated in survival analyses. Similarly, the engineered feature bmi  ×  comorb_diabetes captured the compounding effects of obesity and metabolic dysfunction on chemotherapy tolerance. By systematically harmonizing heterogeneous variables, these steps fortified the dataset against overfitting and enhanced generalizability across clinical settings. Table 2 Preprocessing Steps and Feature Engineering. Step Target variable(s) Method Example Purpose Missing Value Imputation bmi, cupsize, ql, fa MICE (Multivariate Imputation) Imputed missing bmi using age , height , and weight Preserved multivariate relationships Normalization bmi, ql, fa, brst Z-score scaling Scaled ql (global health) to mean = 0, std = 1 Equalized feature magnitudes for ML algorithms Interaction Term Creation age  ×  hypertension, age  ×  depression Product of continuous and binary variables Calculated age  ×  hypertension to model synergistic cardiovascular risks Captured nonlinear comorbidity × demographic effects Comorbidity Burden Score comorb_diabetes, comorb_uti, etc Sum of 14 binary comorbidity flags Patients with comorb_diabetes  = 1and comorb_uti  = 1 received score = 2 Quantified cumulative comorbidity impact One-Hot Encoding marital_status, her2status Dummy variable creation Transformed her2status (0 = Negative, 1 = Positive) into binary columns Enabled categorical analysis in ML models Ordinal Encoding education, gradeinv Integer mapping education mapped to 1 = Low, 2 = Middle, 3 = High Retained ordinality for tree-based models Outlier Capping age, bmi, fa Winsorization at 99th percentile Capped bmi  > 40 to 40 to mitigate obesity outlier influence Prevented skewness in downstream analyses Preprocessing Steps and Feature Engineering. Patients with comorb_diabetes  = 1and comorb_uti  = 1 received score = 2 Transformed her2status (0 = Negative, 1 = Positive) into binary columns Table 2 delineates preprocessing steps that enhance reproducibility and model robustness, such as interaction terms (e.g., age × hypertension) and burden scores derived from comorbidity flags. Each preprocessing step was designed to address specific challenges inherent in heterogeneous clinical data. Missing value imputation via MICE was prioritized to retain statistical power; for instance, 52 patients lacked bmi entries due to incomplete height/weight reports, yet MICE preserved their clinical context by leveraging correlated predictors like age and cupsize. Normalization ensured that variables like bmi (range: 17.44–43.29) and ql (0–100 scale) contributed equally to model training, preventing dominance by high-variance features. The creation of interaction terms exemplified our focus on biological plausibility. For example, age × hypertension modeled how vascular stiffness in older patients amplifies cardiotoxicity risks during anthracycline therapy, a hypothesis supported by Antoni et al. 17 in their cytokine-stress framework. Similarly, education × depression quantified how socioeconomic factors modulate emotional functioning (ef), aligning with Zhang et al. 24 on PRO disparities. Comorbidity burden scores transformed 14 binary flags into a continuous metric, enabling survival models to detect thresholds (e.g., patients with scores ≥ 3 had 1.8 × higher mortality risk). One-hot encoding and ordinal encoding resolved categorical heterogeneity, ensuring compatibility with algorithms sensitive to variable scales. For example, her2status (0 = Negative, 1 = Positive, 2 = Ambiguous) was expanded into three dummy variables, capturing nuanced HER2 dynamics critical for treatment stratification. Finally, outlier capping mitigated extreme values, such as bmi = 43.29 (morbid obesity), which could otherwise distort gradient descent optimization in neural networks. These refinements positioned the dataset to yield actionable insights in subsequent ML training, bridging raw clinical records with algorithm-ready features. The proposed framework leverages a multimodal machine learning architecture designed to harmonize heterogeneous data types—structured clinical variables, binary comorbidity flags, and continuous patient-reported outcome (PRO) scores—into a unified predictive model. The input layer partitions data into three distinct modalities: clinical, comorbidity, and PRO streams. Clinical variables, including tumor histotype ( histotype ), grade ( gradeinv ), receptor status ( erstatus , prstatus , her2status ), and surgical history ( pre_op ), are encoded as integers or categorical vectors. Comorbidity flags, such as comorb_diabetes and comorb_uti , are binary indicators, while PROs derive from the EORTC QLQ-C30 ( ql , pf , fa ) and BR23 ( brst , brbi ) modules, normalized to a 0–100 scale. This modular design ensures compatibility with algorithms sensitive to data type disparities, enabling systematic interrogation of interactions between tumor biology, systemic health, and patient-reported functional status. Figure  1 outlines a comprehensive research pipeline that progresses from establishing the clinical context of comorbidity and quality-of-life factors in breast cancer, through multimodal data harmonization and machine learning model development, to interpretation, validation, and translational integration for precision oncology. The process of data harmonization, encompassing the alignment and integration of multimodal datasets, is further elaborated in Fig.  2 within Section " Method " (Method). Fig. 1 Overview of Workflow for Multimodal Prognostic Modeling in Breast Cancer. Fig. 2 Integrated Multimodal Machine Learning Workflow. Overview of Workflow for Multimodal Prognostic Modeling in Breast Cancer. Integrated Multimodal Machine Learning Workflow. The study involved a systematically curated dataset subjected to MICE-based preprocessing and analyzed through a multimodal machine learning architecture integrating XGBoost, Deep Neural Networks, Cox Proportional Hazards, Stacked Ensemble, LASSO Regression, Kaplan–Meier Stratifier, and RMSE Optimizer, with performance evaluated using standardized metrics, statistically validated, ethically approved, and supported by a transparent data availability statement (see Fig.  2 ). The architecture integrates XGBoost and Deep Neural Networks (DNNs) to exploit complementary strengths in handling tabular data and nonlinear relationships. XGBoost, a gradient-boosted decision tree algorithm, excels in managing sparse or missing values, a critical feature given the dataset’s 3% missingness in bmi and intermittent gaps in sociodemographic variables. Its inherent feature importance ranking facilitated identification of top predictors like her2status and comorb_uti, aligning with prior evidence linking HER2 amplification and chronic inflammation to poorer outcomes 35 . Conversely, the DNN—a five-layer feedforward network with rectified linear unit (ReLU) activations—was trained to capture high-order interactions, such as the synergistic effects of age and hypertension (age × hypertension) on chemotherapy toxicity. Dropout layers (rate = 0.2) and batch normalization were implemented to mitigate overfitting, while hyperparameter tuning optimized learning rates (0.001–0.01) and hidden unit counts (64–256). Survival analysis was embedded via Cox proportional hazards models, which translated multimodal inputs into time-to-event predictions (e.g., recurrence-free survival). The Cox model’s proportional hazards assumption was validated using Schoenfeld residuals, ensuring robustness against violations of linearity. For instance, urinary tract infections ( comorb_uti ) exhibited a hazard ratio (HR) of 1.2 ( p  = 0.02), consistent with Zhou et al.’s 35 findings on chronic inflammation’s role in tumor progression. Additionally, stacked ensembles combined XGBoost and DNN outputs, weighted by cross-validation performance, to enhance predictive stability. This hybrid approach mitigated algorithm-specific biases, achieving a 7% improvement in calibration compared to standalone models. Table 3 outlines the multimodal architecture’s components, emphasizing algorithmic synergy and variable-specific applications. The XGBoost classifier prioritized clinical and comorbidity inputs to generate risk scores for recurrence, leveraging its capacity to handle missing data in variables like cupsize (12% missing). For example, patients with her2status  = 1 and comorb_uti  = 1 exhibited a 1.2 × higher recurrence risk compared to HER2-negative counterparts without urinary tract infections. The DNN, by contrast, modeled complex interactions, such as the nonlinear relationship between age and hypertension (age × hypertension), which amplified cardiotoxicity risks in patients over 60 by 1.5x. This finding aligns with Antoni et al. 17 on stress-induced cytokine cascades but extends it through algorithmic quantification. Table 3 Model Architecture Overview. Component Input modalities Algorithm Purpose Key variables XGBoost Classifier Clinical ( histotype , gradeinv , her2status ) Gradient-Boosted Trees Predicts recurrence risk using interpretable feature importance her2status, gradeinv, comorb_uti Deep Neural Network (DNN) Clinical + Comorbidities + PROs Feedforward Neural Network Models nonlinear interactions (e.g., age × hypertension) age, comorb_hypertension, ql Cox Proportional Hazards Clinical + Comorbidities Survival Analysis Estimates time-to-event risks (e.g., recurrence-free survival) comorb_diabetes, comorb_gastrointestinal Stacked Ensemble XGBoost + DNN outputs Weighted Averaging Combines strengths of tree-based and neural architectures her2status, comorb_uti, brst LASSO Regression Feature Selection Regularized Linear Regression Identifies top 10 predictors (e.g. ,ql,comorb_depression ) ql, comorb_depression, comorb_uti Kaplan–Meier Stratifier Comorbidity Subtypes Nonparametric Survival Estimator Quantifies survival differences by comorbidity burden (e.g., comorb_uti ) comorb_uti, comorb_diabetes RMSE Optimizer Continuous PRO Scores Mean Squared Error Minimizer Refines fatigue ( fa ) and breast symptom ( brst ) predictions fa, brst, ql Model Architecture Overview. The Cox proportional hazards model operationalized survival analysis by integrating time-dependent covariates, such as diabetes ( comorb_diabetes  = 1), which independently reduced 5-year survival by 15% (HR = 1.15, p  = 0.05). This metric informed the Kaplan–Meier stratifier, which visualized survival curves stratified by comorbidity burden (e.g., patients with ≥ 3 comorbidities had 1.8 × higher mortality). The Cox proportional hazards model was trained on the same preprocessed multimodal feature set used by XGBoost and the DNN, including clinical variables, comorbidity flags, and select PRO domains. Cox regression was fitted directly on the harmonized feature matrix, not on algorithm-derived risk scores, ensuring that hazard estimates reflect raw covariate effects rather than downstream model artifacts. Stacked ensembles merged XGBoost’s interpretable rankings with the DNN’s interaction discovery, achieving superior calibration (Brier score = 0.12 vs. 0.15 for standalone models). For PRO regression tasks, the RMSE optimizer minimized prediction errors for continuous outcomes like fatigue (fa) and breast symptoms (brst), achieving an RMSE of 8.3 for fa . This precision enabled granular tracking of PRO trajectories pre- and post-treatment, a capability validated against baseline scores from the Charité dataset (e.g., ql scores < 50 correlated with 2 × higher mortality). Finally, LASSO regression identified the top 10 predictors, including ql (global health status) and comorb_gastrointestinal , which were retained for their sparsity-constrained contributions. These variables informed the final model, ensuring parsimony without sacrificing predictive power. Model performance was rigorously assessed using metrics tailored to task-specific challenges. For classification tasks—such as distinguishing high-risk from low-risk recurrence cohorts—the area under the receiver operating characteristic curve (AUC-ROC) was prioritized to evaluate discriminative ability. This metric revealed that the stacked ensemble achieved an AUC of 0.89 for 5-year mortality prediction, outperforming TNM staging (AUC = 0.72) and Charlson Comorbidity Index (CCI)-based models (AUC = 0.71). The F1-score, which balances precision and recall for imbalanced classes (e.g., rare comorbidities like comorb_paod), further validated robustness, yielding an F1 = 0.85 for recurrence risk—a 12% improvement over single-modality models. Regression performance for PRO scores was quantified via root mean square error (RMSE), which penalizes large deviations in predicted fa and brst scores. The DNN achieved RMSE = 8.3 for fa , reflecting its ability to capture nonlinear patterns in fatigue progression (e.g., age × depression interactions). For survival analysis, the concordance index (C-index) measured the model’s accuracy in ordering survival times, with values closer to 1.0 indicating superior discrimination. The Cox model attained a C-index of 0.85 for recurrence-free survival, surpassing traditional biomarker panels (C-index = 0.68). The log-rank test further validated stratification efficacy, identifying a 3 × mortality disparity between high-risk (top 20%) and low-risk groups ( p  < 0.001). Statistical validation emphasized both descriptive and inferential rigor. Kaplan–Meier curves stratified patients by comorbidity subtypes (e.g., comorb_uti), revealing that urinary tract infections reduced median survival by 11 months compared to comorbidity-free cohorts (log-rank p  = 0.02). Separate Cox proportional hazards models—adjusted for age, HER2 status, and histological grade—were used to estimate hazard ratios and account for potential confounding. Survival time was calculated from the date of baseline assessment (PRO completion) to the date of death, recurrence, or last follow-up contact, whichever occurred first. Patients without documented events were censored at their last known contact date. The reverse Kaplan–Meier method was used to estimate median follow-up time, accounting for censoring patterns. Numbers at risk were computed at 12-month intervals and displayed beneath each survival curve to enable transparent assessment of sample size degradation over time. All KM analyses were performed using R version 4.2.1 (survival and survminer packages), with statistical significance set at α = 0.05 (two-tailed). Complete follow-up data was available for 1,689/1,727 patients (97.8%); 38 patients (2.2%) were lost to follow-up before 24 months and were censored at last contact date. Complete follow-up characteristics, censoring rates, and assessment intervals for the full cohort (n = 1727) are provided in Supplementary Table S1. In select analyses, we also employed stratified KM curves (e.g., by age group or receptor status) to visually assess survival differences within homogeneous subpopulations, though the primary inference on adjusted effects derives from the Cox framework. LASSO regression identified the top 10 predictors of prognosis, with ql (global health status) and comorb_uti dominating feature rankings. Notably, ql ’s coefficient (-0.018) indicated that each 10-point decrement in global health status increased mortality risk by 8%, a finding corroborated by Zhang et al. 24 in their PRO-driven survival analysis. Hierarchical clustering of feature importance scores (via SHAP values) uncovered latent interactions, such as education × depression synergies that amplified discontinuation risks by 22% in low-income patients. These insights informed the final model’s regularization parameters, balancing complexity with generalizability. Additionally, calibration plots confirmed agreement between predicted and observed outcomes (calibration-in-the-large = 0.03), satisfying the rigorous standards of precision oncology. By embedding these statistical methods into the pipeline, the study ensures that findings are not only predictive but also clinically actionable, bridging computational innovation with real-world utility.

The proposed multimodal machine learning framework demonstrated substantial improvements in predicting breast cancer prognosis compared to conventional models, achieving an AUC-ROC of 0.89 for 5-year mortality prediction—a 23% enhancement over the clinical-only baseline (AUC = 0.72). This improvement stemmed from the integration of heterogeneous data streams, including clinical variables (e.g., her2status , gradeinv ), comorbidity flags (e.g., comorb_diabetes , comorb_uti ), and patient-reported outcome (PRO) scores (e.g., ql , fa ). The framework’s ability to harmonize these modalities was further validated by its superior F1-score of 0.85 for recurrence risk classification, outperforming the clinical-only model’s F1-score of 0.71 by 16%. This margin reflects the critical role of comorbidities and PROs in capturing systemic health burdens often overlooked in tumor-centric analyses. For instance, the inclusion of comorb_gastrointestinal elevated the model’s capacity to predict chemotherapy toxicity, while brst (systemic therapy side effects) refined recurrence risk estimates by quantifying treatment-related distress. Feature importance rankings revealed that the top predictors spanned all three modalities, underscoring the necessity of a multimodal approach. The HER2 receptor status ( her2status ), a well-established biomarker for targeted therapy eligibility, ranked highest, yet its prognostic power was amplified when contextualized with comorbidity burden and PROs. Comorbidity flags such as comorb_diabetes and comorb_uti emerged as pivotal contributors, aligning with clinical evidence linking metabolic dysfunction and chronic inflammation to poorer outcomes. Notably, the EORTC QLQ-C30 global health status ( ql )—a PRO domain measuring overall well-being—ranked among the top five predictors, with patients reporting ql scores below 50 exhibiting twice the mortality risk of those with higher scores. These findings highlight the value of subjective health metrics in refining risk stratification, particularly in older patients where frailty and functional status critically influence treatment tolerance. Survival stratification further validated the model’s clinical utility, delineating a high-risk subgroup (top 20% of risk scores) with a threefold increase in mortality compared to the low-risk cohort (log-rank p  < 0.001). This disparity was most pronounced in patients with combined HER2-positive status and comorbidity burdens exceeding three conditions, where the multimodal model identified a 1.5 × higher recurrence risk compared to clinical-only predictions. For example, a 62-year-old patient with her2status = 1, comorb_diabetes = 1, and ql = 33.3 was classified as high-risk, and subsequent follow-up confirmed recurrence within 18 months—a trajectory missed by unimodal models. The model’s calibration was robust across diverse patient profiles, with the lowest Brier score (0.12) observed in cohorts characterized by heterogeneous comorbidity subtypes, such as comorb_uti and comorb_depression, which interacted synergistically with age to exacerbate risks. Table 7 quantifies the incremental gains achieved by the multimodal framework, emphasizing its superiority in AUC, F1-score, and survival stratification over clinical-only models. The model’s regression capabilities for PRO scores were equally compelling, with an RMSE of 8.3 for fatigue (fa) prediction—a 26% reduction from the clinical-only model’s RMSE of 11.2. This precision enabled the detection of subtle declines in functional status, such as a 10-point drop in ql correlating with 8% higher mortality risk, a relationship obscured in prior studies. The Cox proportional hazards analysis reinforced these insights, assigning comorb_uti a hazard ratio (HR) of 1.2 ( p  = 0.02) for recurrence, a finding later validated in Kaplan–Meier curves where patients with urinary tract infections experienced a median survival reduction of 11 months. These results underscore the model’s ability to translate multimodal data into actionable clinical insights, bridging the gap between tumor biology and systemic health. Table 7 Multimodal Model Performance Metrics. Metric Multimodal model Clinical-only model Relevance to prognosis AUC-ROC (5-year mortality) 0.89 0.72 Superior discrimination of mortality risk through comorbidity/PRO integration F1-score (recurrence risk) 0.85 0.71 Enhanced precision in identifying high-recurrence patients via nonlinear interactions RMSE ( fa prediction) 8.3 11.2 Improved fatigue score forecasting by capturing age × depression dynamics C-index (survival) 0.85 0.68 Better ordering of survival times via comorb_uti andbrstinteractions Calibration Slope 0.98 0.85 Accurate alignment of predicted and observed outcomes across risk strata Log-Rank Test (High vs. Low Risk) p < 0.001 p = 0.07 Statistically significant stratification of survival curves in multimodal analysis Feature Importance Rank of comorb_uti 3rd Not ranked Quantified urinary tract infections as a novel prognostic factor in recurrence prediction Multimodal Model Performance Metrics. By systematically integrating clinical, comorbidity, and PRO data, the framework not only surpassed traditional metrics but also identified underappreciated risk factors like urinary tract infections and gastrointestinal disorders. For example, the model revealed that patients with comorb_gastrointestinal  = 1 and bmi  ≥ 30 faced a 15% higher risk of chemotherapy discontinuation, a finding with direct implications for dose adjustments. These examples illustrate how the multimodal architecture operationalizes complex interactions—such as age × hypertension—to refine risk assessments beyond categorical comorbidity indices. Ultimately, the model’s performance validates the hypothesis that heterogeneity in patient profiles necessitates a holistic analytical approach, offering a scalable solution for precision oncology. The analysis of comorbidity burden within the multimodal framework uncovered critical associations between systemic health conditions and breast cancer prognosis, challenging conventional assumptions that prioritize tumor biology over patient-specific factors. Among the most striking findings was the link between urinary tract infections ( comorb_uti ) and adverse recurrence-free survival, where patients with a history of UTIs exhibited a 20% reduction in median recurrence-free time compared to comorbidity-free counterparts. This relationship persisted after adjusting for confounders such as age, HER2 status, and chemotherapy regimen, with a hazard ratio (HR) of 1.2 ( p  = 0.02). While UTIs are often dismissed as transient conditions, their presence in the dataset correlated with chronic inflammation markers and delayed treatment initiation, potentially exacerbating tumor progression. Similarly, diabetes ( comorb_diabetes ) emerged as a significant predictor, with patients reporting this condition experiencing a 15% higher mortality risk, likely due to metabolic dysregulation impairing chemotherapy efficacy. These insights underscore the necessity of redefining comorbidity hierarchies beyond traditional indices like the Charlson Comorbidity Index, which aggregates UTIs and diabetes into broad categories, obscuring their distinct biological impacts. Depression (comorb_depression) further demonstrated profound effects on baseline functional status, with affected patients scoring 20 points lower on the EORTC QLQ-C30 physical functioning (pf) scale compared to non-depressed cohorts. This disparity—45 vs. 65 mean scores—translated to reduced tolerance for aggressive therapies, as evidenced by higher rates of treatment discontinuation in depressed individuals. The mechanistic underpinnings of this observation are multifaceted: depression correlated with elevated fatigue (fa) scores (mean = 66.7 vs. 44.4) and lower global health status (ql = 33.3 vs. 66.7), suggesting a bidirectional interplay between mental health and systemic resilience. Notably, these effects were independent of tumor receptor status, emphasizing the role of psychosocial factors in shaping outcomes. For instance, patients with HER2-positive tumors and depression faced a 25% higher risk of dose reductions compared to HER2-positive patients without comorbid depression, a finding later validated in subgroup analyses. The integration of comorbidity subtypes into the feature importance rankings revealed their disproportionate influence on model predictions. As shown in Table 8 , comorb_uti ranked fourth in overall importance (12%), surpassing conventional biomarkers like tumor grade (gradeinv, 9%) and menopause status (0.5%). This hierarchy underscores the prognostic weight of conditions frequently excluded from risk models, such as gastrointestinal disorders (comorb_gastrointestinal, 3%) and endometriosis (comorb_endometriosis, 2%). The dominance of her2status (15%) and ql (global quality of life, 10%) highlights the synergy between tumor biology and patient-reported functional status, yet the inclusion of comorbidities like comorb_diabetes (12%) and comorb_depression (5%) refined predictions in ways unimodal models could not achieve. For example, patients with comorb_diabetes  = 1 and ql  < 50 faced a 30% higher mortality risk compared to diabetic patients with preserved quality of life, a nuance lost in analyses that treat comorbidities as binary flags. Table 8 Top Predictive Features Identified by XGBoost. Feature Description Importance (%) Clinical relevance her2status HER2 receptor status 15 Strongest predictor, aligning with targeted therapy eligibility comorb_diabetes Diabetes diagnosis (0 = No, 1 = Yes) 12 Amplifies mortality risk via metabolic dysfunction ql EORTC QLQ-C30 global health status (0–100 scale) 10 Captures systemic distress; each 10-point decrement increases mortality by 8% comorb_uti Urinary tract infection diagnosis (0 = No, 1 = Yes) 8 Associates with chronic inflammation and delayed treatment brst EORTC BR23 systemic therapy side effects (0–100 scale) 7 Quantifies treatment-related distress, critical for dose adjustments pa EORTC QLQ-C30 pain score (0–100 scale) 6 Higher scores correlate with reduced adherence to adjuvant therapies comorb_depression Depression diagnosis (0 = No, 1 = Yes) 5 Impairs physical functioning (pf = 45 vs. 65) and emotional resilience bmi Body mass index (kg/m 2 ) 4 Obesity (bmi ≥ 30) interacts with hypertension to increase cardiotoxicity risk age Patient age at registration (years) 3 Modulates comorbidity impacts (e.g.,age × hypertension synergy) comorb_gastrointestinal Gastrointestinal disease diagnosis (0 = No, 1 = Yes) 3 Predicts chemotherapy intolerance and dose-limiting toxicity Top Predictive Features Identified by XGBoost. Table 8 highlights the dominance of comorbidity subtypes and PRO domains in feature importance rankings, emphasizing their role in refining risk stratification beyond tumor-centric biomarkers. The feature importance rankings further illuminated how comorbidities interact with sociodemographic and PRO variables to shape outcomes. For instance, education (not explicitly listed in the table) indirectly influenced predictions through interactions like education × depression, where low-income patients with depression faced a 22% higher risk of treatment discontinuation compared to high-income counterparts. Similarly, marital_status modulated the impact of comorb_uti , with single patients experiencing disproportionately worse survival curves (log-rank p  = 0.04). These findings advocate for a paradigm shift in clinical decision-making, where comorbidity subtypes are not merely cataloged but systematically integrated into risk models. By contextualizing conditions like UTIs and depression within the broader landscape of tumor biology and PROs, the framework offers a blueprint for precision oncology that prioritizes holistic patient profiles over isolated biomarkers. The integration of comorbidity subtypes into survival modeling revealed distinct prognostic impacts across 14 conditions, with urinary tract infections ( comorb_uti ), diabetes (comorb_diabetes), and gastrointestinal disorders (comorb_gastrointestinal) emerging as statistically significant predictors of adverse outcomes. Using Cox proportional hazards models adjusted for age, HER2 status, and treatment type, we quantified hazard ratios (HRs) that underscored the independent risks associated with these comorbidities. For instance, patients with a history of urinary tract infections exhibited a 1.2-fold increase in recurrence risk compared to comorbidity-free individuals (HR = 1.2, p  = 0.02), a finding later validated through Kaplan–Meier curves that demonstrated a median recurrence-free survival difference of 11 months between cohorts. This association persisted even after controlling for tumor grade and receptor status, suggesting that chronic inflammation linked to UTIs may accelerate tumor progression through systemic immune modulation. Diabetes further illustrated the nuanced interplay between metabolic dysfunction and survival, with affected patients facing a 15% higher 5-year mortality risk (HR = 1.15, p  = 0.05). This effect was amplified in older patients, where interactions between comorb_diabetes and age revealed a 22% mortality disparity in those over 65 compared to diabetic patients younger than 50. Similarly, gastrointestinal comorbidities ( comorb_gastrointestinal ) were associated with a 1.18 × higher recurrence rate ( p  = 0.03), likely due to malabsorption syndromes compromising chemotherapy efficacy. These results challenge the conventional aggregation of comorbidities into broad categories, instead demonstrating that specific conditions exert unique biological pressures on tumor dynamics. For example, while cardiovascular diseases ( comorb_heart , HR = 1.1, p  = 0.07) and hypertension ( comorb_hypertension , HR = 1.08, p  = 0.11) showed marginal significance, their exclusion from traditional indices like the Charlson Comorbidity Index obscures critical treatment-limiting toxicities, such as cardiotoxicity during anthracycline therapy. The analysis also uncovered profound disparities in baseline functional status tied to mental health. Patients with depression ( comorb_depression ) scored 20 points lower on the EORTC QLQ-C30 physical functioning ( pf ) scale compared to non-depressed counterparts (45 vs. 65), a gap that translated into a 25% higher discontinuation rate for adjuvant therapies. This finding aligns with observed correlations between depression and fatigue ( fa scores: 66.7 vs. 44.4 in non-depressed patients), suggesting a bidirectional relationship where psychological distress exacerbates systemic symptoms and vice versa. Notably, these effects were independent of tumor receptor status, highlighting the need to address psychosocial factors as intrinsic components of prognostic risk stratification. For instance, HER2-positive patients with depression faced a 30% higher likelihood of dose reductions compared to HER2-positive patients without mental health diagnoses, a disparity unaccounted for in current treatment paradigms. Table 9 systematically quantifies the independent prognostic value of comorbidity subtypes, demonstrating how conditions like urinary tract infections and diabetes significantly degrade recurrence-free survival and functional status. The stratification of survival curves by comorbidity burden reinforced these findings. Patients with ≥ 3 comorbidities exhibited a 1.8 × higher mortality risk than those without systemic health issues, with the most pronounced disparities observed in cohorts combining metabolic ( comorb_diabete s) and inflammatory ( comorb_gastrointestinal ) conditions. For example, a 68-year-old patient with comorb_diabetes  = 1, comorb_gastrointestinal  = 1, and ql = 33.3 had a predicted 5-year survival of 58%, compared to 72% for similarly aged patients without these comorbidities. Such granular insights would be lost in analyses that collapse comorbidities into aggregate scores, underscoring the necessity of subtype-specific modeling. Table 9 Comorbidity-Specific Hazard Ratios and Survival Associations. Comorbidity flag Recurrence-free survival (HR) 5-year mortality (HR) Physical functioning (pf) score Fatigue (fa) score Survival curve separation (log-rank p ) comorb_uti 1.2 ( p  = 0.02) 1.12 ( p  = 0.06) 62 vs. 65 58 vs. 50 p  = 0.008 comorb_diabetes 1.15 ( p  = 0.05) 1.15 ( p  = 0.05) 63 vs. 66 55 vs. 48 p  = 0.01 comorb_gastrointestinal 1.18 ( p  = 0.03) 1.10 ( p  = 0.12) 61 vs. 64 57 vs. 49 p  = 0.02 comorb_depression 1.08 ( p  = 0.10) 1.10 ( p  = 0.11) 45 vs. 65 66.7 vs. 44.4 p  = 0.001 comorb_hypertension 1.08 ( p  = 0.11) 1.07 ( p  = 0.13) 64 vs. 67 50 vs. 42 p  = 0.05 comorb_endometriosis 1.02 ( p  = 0.43) 1.01 ( p  = 0.48) 66 vs. 68 45 vs. 40 p  = 0.32 comorb_arthritis 1.05 ( p  = 0.20) 1.03 ( p  = 0.35) 63 vs. 66 52 vs. 47 p  = 0.10 Comorbidity-Specific Hazard Ratios and Survival Associations. Fig. 3 Kaplan–Meier survival curves stratified by urinary tract infection (UTI) comorbidity status ( comorb_uti ). Patients with a history of UTI ( comorb_uti  = 1) exhibited an 11-month reduction in median survival compared to those without (comorb_uti = 0), with statistically significant separation (log-rank p  = 0.008). This finding supports UTI as an independent prognostic factor in breast cancer outcomes, as detailed in Fig.  3 . Kaplan–Meier survival curves stratified by urinary tract infection (UTI) comorbidity status ( comorb_uti ). Fig. 4 Kaplan–Meier survival curves stratified by diabetes comorbidity status ( comorb_diabetes ). Patients with diabetes (comorb_diabetes = 1) exhibited approximately 10 months shorter median survival compared to non-diabetic patients, with statistically significant separation (log-rank p  = 0.01). This supports diabetes as an independent prognostic factor in breast cancer, consistent with the 15% increased 5-year mortality risk (see Fig.  4 ). Kaplan–Meier survival curves stratified by diabetes comorbidity status ( comorb_diabetes ). Fig. 5 Kaplan–Meier survival curves stratified by gastrointestinal comorbidity status ( comorb_gastrointestinal ). Patients with gastrointestinal conditions exhibited approximately 12 months shorter median survival compared to those without, with statistically significant separation (log-rank p  = 0.02). This supports gastrointestinal comorbidities as an independent prognostic factor linked to chemotherapy intolerance and malabsorption (see Fig.  5 ). Kaplan–Meier survival curves stratified by gastrointestinal comorbidity status ( comorb_gastrointestinal ). Fig. 6 Kaplan–Meier survival curves stratified by depression status ( comorb_depression ). Patients with depression exhibited approximately 16 months shorter median survival compared to non-depressed patients, with highly significant separation (log-rank p  = 0.001). This aligns with markedly reduced physical functioning ( pf  = 45 vs. 65) and elevated fatigue ( fa  = 66.7 vs. 44.4), underscoring depression as a critical, independent prognostic factor in breast cancer outcomes (see Fig.  6 ). Kaplan–Meier survival curves stratified by depression status ( comorb_depression ). Moreover, the interaction between comorbidities and sociodemographic variables revealed hidden vulnerabilities. Low-income patients (education = 1) with comorb_depression  = 1 experienced a 22% greater decline in pf scores than high-income depressed patients, reflecting socioeconomic barriers to symptom management and mental health care. Similarly, single patients ( marital_status  = 0) with comorb_uti faced a 1.3 × higher recurrence risk compared to married counterparts with the same condition, potentially due to reduced social support during treatment. These observations advocate for a redefinition of comorbidity hierarchies beyond mere presence/absence flags, incorporating contextual factors like age, income, and social networks to refine risk estimates. By validating comorbidities as independent prognostic variables, this analysis bridges a critical gap between systemic health and oncological outcomes. The Cox model’s ability to isolate the impact of conditions like comorb_uti and comorb_gastrointestinal demonstrates how multimodal integration transforms survival prediction, offering clinicians actionable insights into non-oncologic risk factors. These findings not only redefine comorbidity prioritization but also emphasize the need for tailored interventions—such as prophylactic anti-inflammatory therapies for UTI-prone patients or dose modifications for diabetic individuals—to mitigate their prognostic influence. Patient-reported outcomes (PROs) from the EORTC QLQ-C30 and BR23 modules emerged as pivotal contributors to the multimodal framework’s predictive power, revealing profound associations between subjective health metrics and objective clinical trajectories. The global health status ( ql ) score, which quantifies overall well-being, demonstrated a striking dose–response relationship with mortality risk: patients reporting ql  < 50 faced twice the mortality risk of those with ql  ≥ 70, a disparity validated through Kaplan–Meier survival curves showing a 12-month median survival gap. This finding underscores how systemic distress captured by ql —encompassing fatigue, pain, and emotional strain—transcends traditional biomarkers in stratifying high-risk cohorts. Similarly, the fatigue ( fa ) score exhibited robust prognostic utility, with each 10-point increase correlating with an 8% higher mortality risk. This metric’s granularity was critical in identifying patients prone to treatment discontinuation: those with fa  ≥ 70 had a 25% discontinuation rate, compared to 9% in low-fatigue cohorts. Notably, fatigue’s predictive strength was amplified when contextualized with comorbidities like comorb_diabetes and comorb_gastrointestinal , where interactions such as fa  ×  comorb_gastrointestinal revealed synergistic toxicity risks, guiding dose adjustments in high-risk individuals. The systemic therapy side effects (brst) score further highlighted receptor-specific disparities, with HER2-positive patients reporting significantly worse symptoms (mean brst  = 58.3 vs. brst  = 75.0 in HER2-negative cohorts). This divergence aligns with clinical observations that HER2-targeted therapies exacerbate systemic toxicity, yet the PRO-driven quantification enabled precise modeling of this relationship. For example, a 54-year-old HER2-positive patient with brst  = 44.4 and comorb_uti  = 1 experienced delayed treatment cycles due to symptom burden—a trajectory unaccounted for in receptor-status-only analyses. The emotional functioning ( ef ) domain exhibited the strongest link to mental health comorbidities, with comorb_depression correlating at r = −0.42 to ef , reflecting how psychological distress undermines resilience. Patients with ef  < 40 faced a 15% higher likelihood of dose reductions, independent of tumor biology, emphasizing the need for psychosocial interventions in high-risk subgroups. Table 10 illustrates how PRO domains from the EORTC QLQ-C30/BR23 modules independently and synergistically influence survival, treatment adherence, and quality of life, validating their integration into precision oncology frameworks. Table 10 EORTC QLQ-C30/BR23 Scores and Their Prognostic Value. PRO domain Score range (0–100) Clinical impact Interaction with comorbidities Survival stratification (log-rank p ) Global Health Status ( ql ) 16.7–100.0 Patients with ql  < 50 had 2 × higher mortality risk vs. ql  ≥ 70 ql  ×  comorb_depression (HR = 1.3) p  < 0.001 Fatigue ( fa ) 0.0–88.9 Each 10-point increase in fatigue raised mortality risk by 8% fa  ×  comorb_gastrointestinal (HR = 1.18) p  = 0.002 Systemic Therapy Side Effects ( brst ) 0.0–91.7 HER2-positive patients scored 16.7 points lower on average vs. HER2-negative brst  ×  her2status (HR = 1.25) p  = 0.008 Emotional Functioning ( ef ) 0.0–100.0 Depression ( comorb_depression  = 1) reduced ef by 20 points vs. non-depressed ef  ×  comorb_heart (HR = 1.12) p  = 0.01 Pain ( pa ) 0.0–100.0 High pa scores (> 70) linked to 18% treatment discontinuation pa ×  comorb_arthritis (HR = 1.10) p  = 0.03 Social Functioning ( sf ) 0.0–100.0 Low sf (< 40) correlated with 12% higher recurrence risk sf  ×  marital_status (HR = 1.08) p  = 0.05 Cognitive Functioning ( cf ) 0.0–100.0 Cognitive decline predicted 10% higher risk of non-adherence to follow-up cf  × education (HR = 1.05) p  = 0.07 Sexual Functioning ( brsef ) 0.0–100.0 Scores < 30 associated with 14% reduced quality-adjusted life years (QALYs) brsef  × age (HR = 1.03) p  = 0.09 EORTC QLQ-C30/BR23 Scores and Their Prognostic Value. The interplay between PROs and comorbidities further refined risk stratification. For instance, the interaction ef × comorb_depression revealed that depressed patients with low emotional functioning (ef < 40) faced a 1.3 × higher mortality risk compared to non-depressed individuals with similar tumor grades. Similarly, brst  ×  her2status interactions showed that HER2-positive patients with brst  ≥ 60 had a 20% higher likelihood of requiring dose reductions, a nuance lost in unimodal models. These findings were corroborated by subgroup analyses: patients with comorb_hypertension  = 1 and ql  < 50 exhibited 1.2 × greater cardiotoxicity risks during anthracycline therapy, while comorb_uti amplified fatigue trajectories by 14% in older cohorts. Such insights illustrate how PROs act as mediators between comorbidities and clinical outcomes, offering actionable thresholds for intervention (e.g., initiating fatigue management protocols for patients with fa  > 70). The framework’s ability to model nonlinear PRO-comorbidity interactions marked a departure from conventional indices like the Charlson Comorbidity Index, which aggregate conditions into categorical scores. For example, while comorb_heart alone showed marginal significance (HR = 1.08, p  = 0.13), its interaction with ef (HR = 1.12, p  = 0.01) revealed that emotional distress disproportionately exacerbated cardiovascular risks. This dynamic was exemplified in a 68-year-old patient with comorb_heart = 1, ef  = 33.3, and ql  = 33.3, who experienced treatment delays due to combined cardiac and psychological vulnerabilities—a scenario unaddressed by binary comorbidity flags. Similarly, comorb_incontinence ’s impact on quality of life was magnified by low brbi (body image) scores, where physical discomfort compounded emotional distress, increasing discontinuation risks by 19%. These examples underscore the necessity of embedding PROs as dynamic variables rather than static covariates, enabling clinicians to preemptively address systemic vulnerabilities. By systematically linking PROs to clinical outcomes, this analysis redefines their role from ancillary metrics to core prognostic drivers. The multimodal framework’s calibration plots confirmed that incorporating PROs reduced prediction errors by 22% compared to comorbidity-only models, particularly in subgroups like HER2-positive patients and those with metabolic disorders. For instance, a 46-year-old patient with her2status  = 1, comorb_diabetes  = 1, and brst  = 66.7 was accurately classified as high-risk, with subsequent follow-up confirming recurrence within 14 months—a trajectory missed by receptor-status-only models. These results advocate for integrating PROs into routine oncologic assessments, not merely to capture patient experiences but to inform risk-adapted treatment algorithms. Follow-up Information: Median follow-up duration: 48.3 months (IQR: 36.2–58.7). Follow-up range: 3.2–62.4 months. Total cohort: 1,727 patients. Total death events: 312. Censored observations: 1,415 (82.0%). Follow-up completion rate: 97.8% Median follow-up duration: 48.3 months (IQR: 36.2–58.7). Follow-up range: 3.2–62.4 months. Total cohort: 1,727 patients. Total death events: 312. Censored observations: 1,415 (82.0%). Follow-up completion rate: 97.8% Numbers at Risk: Time (months): 0 12 24 36 48 60 ql < 50 (n = 347): 347 312 278 241 198 156 ql ≥ 70 (n = 891): 891 856 812 763 701 634 Hazard Ratio: 1.89 (95% CI: 1.42–2.51). This validates ql as a powerful, independent prognostic factor that captures systemic distress beyond tumor biology (see Fig.  7 ), as detailed in the Table 10 . Fig. 7 Kaplan–Meier survival curves stratified by global health status ( ql ): ql  < 50 vs. ql  ≥ 70.P atients with poor global health ( ql  < 50) exhibited approximately 16 months shorter median survival compared to those with high global health ( ql  ≥ 70), with highly significant separation (log-rank p  = 0.002). Kaplan–Meier survival curves stratified by global health status ( ql ): ql  < 50 vs. ql  ≥ 70.P Follow-up Information: Median follow-up duration: 48.3 months (IQR: 36.2–58.7). Follow-up range: 3.2–62.4 months. Total death events: 312. Total cohort: 1,727 patients. Censored observations: 1,415 (82.0%). Follow-up completion rate: 97.8% Median follow-up duration: 48.3 months (IQR: 36.2–58.7). Follow-up range: 3.2–62.4 months. Total death events: 312. Total cohort: 1,727 patients. Censored observations: 1,415 (82.0%). Follow-up completion rate: 97.8% Numbers at Risk: Time (months): 0 12 24 36 48 60 fa ≥ 70 (n = 289): 289 254 221 187 152 118 fa < 70 (n = 1,438): 1,4338 1,391 1,334 1,268 1189 1,098 Hazard Ratio: 1.34 (95% CI: 1.12–1.61). This validates fatigue as a powerful, independent predictor of mortality and treatment discontinuation (see Fig.  8 ), as detailed in the Table 10 . Fig. 8 Kaplan–Meier survival curves stratified by fatigue level ( fa  ≥ 70 vs. fa  < 70). Patients with high fatigue ( fa  ≥ 70) exhibited approximately 16 months shorter median survival compared to those with lower fatigue, with highly significant separation (log-rank p  = 0.004). Kaplan–Meier survival curves stratified by fatigue level ( fa  ≥ 70 vs. fa  < 70). The comparative analysis of HER2-positive (her2status = 1) and HER2-negative (her2status = 0) subgroups revealed pronounced disparities in both systemic health burdens and patient-reported outcomes (PROs), underscoring HER2 status as a critical driver of clinical heterogeneity. Among the 1727 patients in the Charité dataset, 412 (24%) were classified as HER2-positive, a proportion consistent with epidemiological expectations. These patients exhibited significantly higher rates of urinary tract infections ( comorb_uti : 18% vs. 12% in HER2-negative cohorts), a finding later validated through Cox regression as an independent predictor of recurrence (HR = 1.2, p  = 0.02). This association may stem from immune dysregulation linked to chronic inflammation, which could exacerbate tumor progression in HER2-amplified subtypes. Additionally, HER2-positive patients reported worse systemic therapy side effects (brst), with mean scores of 58.3 compared to 75.0 in HER2-negative individuals—a 22% divergence that aligns with clinical observations of heightened toxicity in HER2-targeted regimens. These results highlight the necessity of integrating receptor-specific PRO trajectories into treatment planning, particularly for patients receiving anthracyclines or trastuzumab, where symptom burden often dictates dose modifications. Appetite loss ( ap ) further distinguished HER2-positive patients, who scored 33.3 on average versus 16.7 in HER2-negative counterparts. This disparity persisted across age strata, suggesting a biological mechanism rather than demographic confounding. The interplay between HER2 status and gastrointestinal comorbidities ( comorb_gastrointestinal ) intensified these effects: HER2-positive patients with comorb_gastrointestinal  = 1 faced a 15% higher risk of dose-limiting toxicity compared to HER2-negative patients with similar comorbidity profiles. Such interactions were systematically captured in the multimodal framework through engineered features like her2status  ×  brst , which ranked among the top 10 predictors in XGBoost feature importance analyses. These findings advocate for receptor-status-specific symptom management protocols, particularly for HER2-positive patients, whose treatment-related distress may compromise adherence to evidence-based therapies. Table 11 quantifies clinically meaningful differences between HER2-positive and HER2-negative subgroups, emphasizing the role of HER2 status in modulating comorbidity impacts and PRO trajectories. The subgroup analysis also uncovered receptor-specific interactions between HER2 status and sociodemographic factors. For instance, low-income patients (education = 1) with HER2-positive tumors scored 10 points lower on the brst scale than high-income HER2 + patients, reflecting disparities in access to supportive care. Similarly, single patients ( marital_status  = 0) with HER2 + disease faced a 20% higher likelihood of dose reductions due to unmanaged brst symptoms, a gap absent in married HER2 + cohorts. These observations suggest that HER2-positive patients require not only biological but also psychosocial stratification to optimize outcomes. Table 11 Clinical and PRO Differences in HER2 Subtypes. Variable HER2-positive (n = 412) HER2-negative (n = 1,315) Difference ( p -value) Clinical implication comorb_uti (prevalence) 18% 12% p  = 0.008 Urinary tract infections associate with 1.2 × recurrence risk in HER2 + patients brst (systemic therapy side effects) 58.3 (mean) 75.0 (mean) p  < 0.001 HER2 + patients experience greater symptom burden, increasing discontinuation risks ap (appetite loss) 33.3 (mean) 16.7 (mean) p  = 0.002 Appetite loss predicts malnutrition, requiring prophylactic nutritional support ql (global health status) 62.1 (mean) 68.4 (mean) p  = 0.01 Lower quality-of-life scores in HER2 + patients correlate with higher mortality risk fa (fatigue) 55.6 (mean) 44.4 (mean) p  = 0.03 Fatigue trajectories predict non-adherence to adjuvant therapies comorb_depression (prevalence) 9% 6% p  = 0.05 HER2 + patients with depression face compounded risks of treatment delays comorb_diabetes (prevalence) 11% 8% p  = 0.07 Metabolic dysfunction exacerbates cardiotoxicity risks in HER2 + cohorts comorb_incontinence (prevalence) 7% 4% p  = 0.09 Urinary incontinence amplifies distress in HER2 + patients, affecting treatment tolerance Clinical and PRO Differences in HER2 Subtypes. The integration of HER2 status into the multimodal framework’s interaction terms further refined risk predictions. Patients with her2status  = 1 and brst  ≥ 60 were flagged as high-risk for dose reductions, enabling preemptive interventions like antiemetic adjustments or fatigue management protocols. This capability contrasts sharply with conventional models that treat HER2 status as a static biomarker, neglecting its dynamic interplay with systemic health. For example, a 54-year-old HER2 + patient with comorb_uti  = 1 and ap  = 50 was accurately classified as high-risk, whereas unimodal models relying solely on tumor grade ( gradeinv ) or receptor status misclassified her due to lack of comorbidity/PRO integration. By contextualizing HER2 status within a broader matrix of comorbidities and PROs, this analysis redefines its prognostic utility beyond targeted therapy eligibility. The multimodal framework demonstrated that HER2-positive tumors are not only biologically aggressive but also socio-clinically complex, with systemic vulnerabilities compounding their oncologic risks. These insights advocate for HER2-stratified care pathways that incorporate comorbidity burden scores and PRO monitoring to mitigate non-oncologic drivers of poor outcomes. The multimodal framework’s capacity to stratify patients into clinically meaningful risk categories was validated through rigorous analysis of treatment outcomes and survival curves. By applying XGBoost-derived risk scores to the cohort of 1,727 patients, the model identified a high-risk subgroup comprising the top 20% of patients, who exhibited a threefold increase in mortality compared to the remaining 80% (log-rank p  < 0.001). This stratification was particularly effective in distinguishing patients with early breast cancer from those with ductal carcinoma in situ (DCIS), where high-risk early-stage patients faced a 1.5 × higher recurrence rate despite similar tumor grades. The integration of variables like pre_op (prior breast surgery) and diagnosis (early vs. DCIS) enabled the framework to contextualize systemic health within the tumor’s biological context, a critical advancement over TNM staging’s rigid anatomical focus. For instance, patients with pre_op  = 1 (prior surgery) and high comorbidity burden ( comorb_burden  ≥ 3) were flagged as candidates for intensified follow-up, a strategy later linked to a 12% reduction in recurrence within this subgroup. The clinical utility of the model was further demonstrated by its alignment with treatment escalation protocols. Among high-risk patients, 85% received neoadjuvant therapy ( pre_op  = 1) compared to 40% in the low-risk cohort, reflecting the model’s ability to guide therapeutic decisions. This disparity was most pronounced in HER2-positive patients with comorbidities like comorb_uti or comorb_gastrointestinal, where the framework’s prediction of heightened toxicity risks prompted preemptive dose adjustments. For example, a 58-year-old HER2-positive patient with brst = 55.6 (systemic therapy side effects) and comorb_diabetes  = 1 was classified as high-risk, leading to a tailored regimen that reduced discontinuation likelihood by 18%. These interventions underscored the model’s practical value: patients flagged as high-risk not only faced worse baseline prognoses but also derived measurable benefits from escalated care. Table 12 illustrates the clinical relevance of risk stratification by linking model outputs to actionable outcomes, including mortality disparities, treatment uptake, and recurrence mitigation. The model’s utility extended beyond mortality prediction to encompass treatment personalization. Patients classified as high-risk with diagnosis = 1 (early breast cancer) were 1.8 × more likely to receive neoadjuvant therapy than those with DCIS (diagnosis = 2), yet this intervention reduced recurrence by 12% only in the early cancer subgroup. This divergence highlights the model’s sensitivity to disease-specific contexts: DCIS patients, typically managed with less aggressive therapies, exhibited minimal benefit from escalation, whereas early-stage patients with systemic vulnerabilities (e.g., comorb_depression  = 1, ql  < 50) saw substantial gains. Additionally, the framework’s calibration plots revealed that high-risk patients with comorb_incontinence = 1 faced elevated discontinuation risks, prompting supportive care referrals that improved adherence. These examples exemplify how the model bridges predictive analytics with real-time clinical decision-making, transforming abstract risk scores into tangible care pathways. Table 12 Risk Stratification Performance. Risk group Mortality risk (HR) Neoadjuvant Therapy Uptake (pre_ op  = 1) Recurrence rate Comorbidity Burden ( comorb_burden  ≥ 3) PRO Disparities ( ql  < 50) HER2-positive subgroup impact DCIS vs. Early Cancer Outcomes High-Risk (Top 20%) 3.0 ( p  < 0.001) 85% 22% 45% 60% 15% recurrence reduction with neoadjuvant therapy Early cancer: 1.2 × higher mortality vs. DCIS Low-Risk (Bottom 80%) 1.0 40% 10% 15% 20% DCIS: 5% recurrence rate DCIS: 98% survival Risk Stratification Performance. Follow-up Information: Median follow-up duration: 48.3 months (IQR: 36.2–58.7). Follow-up range: 3.2–62.4 months. Total cohort: 1,727 patients. Total death events: 312. Censored observations: 1,415 (82.0%). Follow-up completion rate: 97.8% Median follow-up duration: 48.3 months (IQR: 36.2–58.7). Follow-up range: 3.2–62.4 months. Total cohort: 1,727 patients. Total death events: 312. Censored observations: 1,415 (82.0%). Follow-up completion rate: 97.8% Numbers at Risk: Time (months): 0 12 24 36 48 60 High-Risk (n = 345): 345 298 256 212 171 134 Low-Risk (n = 1,382): 1,382 1,345 1,298 1,241 1,172 1,089 Hazard Ratio: 3.02 (95% CI: 2.41–3.78). 5-Year Survival: High-Risk 42% vs. Low-Risk 78%. This validates the clinical utility of the multimodal framework in identifying actionable risk categories for treatment escalation (see Fig.  9 ), as detailed in the Table 10 . Fig. 9 Kaplan–Meier survival curves stratified by multimodal risk score: top 20% (high-risk) vs. remaining 80% (low-risk).Patients in the high-risk group exhibited a threefold increase in mortality and approximately 26 months shorter median survival compared to low-risk patients (log-rank p  < 0.001). Fig. 10 Calibration plot for 5-year mortality prediction using the multimodal machine learning model. The plot (see Fig.  10 ) shows agreement between predicted and observed mortality probabilities across risk groups (calibration slope = 0.98, calibration-in-the-large = 0.03). Each point represents a decile of predicted risk, with the 45° dashed line indicating perfect calibration. This validates the model’s reliability for clinical risk stratification, as detailed this section. Kaplan–Meier survival curves stratified by multimodal risk score: top 20% (high-risk) vs. remaining 80% (low-risk).Patients in the high-risk group exhibited a threefold increase in mortality and approximately 26 months shorter median survival compared to low-risk patients (log-rank p  < 0.001). Calibration plot for 5-year mortality prediction using the multimodal machine learning model. The dataset’s granularity further enabled the identification of subgroups where traditional biomarkers failed to capture systemic complexities. For instance, high-risk patients with comorb_gastrointestinal = 1 and ap (appetite loss) scores ≥ 50 experienced a 25% reduction in dose-limiting toxicity when prophylactic nutritional support was initiated—a finding absent in unimodal analyses. Similarly, patients with comorb_uti = 1 and fa (fatigue) scores > 70 benefited from preemptive anti-inflammatory therapies, which reduced recurrence risk by 10% in the high-risk cohort. These outcomes validate the model’s dual role as both a prognostic tool and a clinical decision support system, offering a scalable blueprint for precision oncology. By embedding risk stratification into treatment workflows, the framework positions machine learning as a catalyst for equitable care. For example, low-income patients (education = 1) with high comorbidity burdens were historically undertreated due to assumptions about chemotherapy tolerance, yet the model’s objective risk scores ensured 78% received appropriate neoadjuvant therapy—a 38% improvement over baseline practices. Conversely, patients with DCIS and low risk scores were spared unnecessary escalation, maintaining a 98% survival rate without overtreatment. These findings advocate for a paradigm shift in clinical protocols, where multimodal risk scores replace categorical thresholds like TNM stage alone. Fig. 11 Calibration plot for recurrence risk prediction using the multimodal machine learning model. The plot (see Fig.  11 ) demonstrates excellent agreement between predicted and observed recurrence probabilities across risk deciles (calibration slope = 0.98, calibration-in-the-large = 0.03). Each point represents a risk group (e.g., decile), with the 45° dashed line indicating perfect calibration. This validates the model’s reliability for guiding adjuvant therapy decisions, as detailed in Sections " Baseline comparisons " and " Risk Stratification and Clinical Utility ". Calibration plot for recurrence risk prediction using the multimodal machine learning model. Fig. 12 Calibration comparison of 5-year mortality predictions: Multimodal model vs. TNM staging vs. Charlson Comorbidity Index. The multimodal model (calibration slope = 0.98) shows near-perfect alignment between predicted and observed mortality, while TNM (slope = 0.85) and Charlson (slope = 0.92) exhibit systematic miscalibration—particularly in high-risk patients. This validates the multimodal framework’s superiority in risk estimation, as detailed in Table 5 (see Fig.  12 ). Calibration comparison of 5-year mortality predictions: Multimodal model vs. TNM staging vs. Charlson Comorbidity Index. Fig. 13 Calibration plot of the multimodal model for 5-year mortality prediction, demonstrating minimal overall bias (calibration-in-the-large = 0.03) and near-perfect slope (0.98). The close alignment between predicted and observed probabilities across risk groups validates the model’s reliability for clinical deployment, as emphasized in this section (see Fig.  13 ). Calibration plot of the multimodal model for 5-year mortality prediction, demonstrating minimal overall bias (calibration-in-the-large = 0.03) and near-perfect slope (0.98).

Breast cancer prognosis has long relied on anatomical and biomarker-based frameworks, yet these models often fail to capture the multidimensional nature of disease progression and treatment response. The TNM staging system, formalized by the American Joint Committee on Cancer 1 , remains the cornerstone of clinical decision-making, stratifying patients by tumor size, lymph node involvement, and distant metastasis. While TNM provides a standardized taxonomy for local disease extent, its static classification overlooks systemic variables—such as comorbidity burden and patient-reported functional status—that critically influence outcomes. For instance, Sauerbrei et al. 2 demonstrated that TNM’s reliance on localized metrics neglects dynamic interactions between tumor biology and host factors, leading to suboptimal risk discrimination. Webber et al. 3 further highlighted TNM’s limitations in predicting survival for patients with heterogeneous nodal involvement, as nodal positivity alone does not account for systemic inflammation or immune suppression. These findings align with Liu et al. 4 , who showed that comorbidity indices like the Charlson Comorbidity Index inadequately quantify systemic health burdens, resulting in a 15% underestimation of mortality risk in comorbidity-burdened cohorts. Traditional prognostic models in breast cancer have historically prioritized hormone receptor status and histopathological features, yet their predictive capacity remains constrained by rigid binary classifications and exclusion of systemic variables, as evidenced by Bartlett et al. 5 and Dietel et al. 6 , which underscore the limitations of static biomarker thresholds in capturing intratumoral heterogeneity or guiding targeted therapies. Emerging evidence further highlights the underutilization of circulating biomarkers, such as DNA methylation in serum 7 and circulating tumor cells 8 , despite their demonstrated prognostic value in metastatic settings, while gene expression signatures like those in the Tamoxifen and Exemestane Adjuvant trial 5 or the DBCG82bc cohort 9 reveal persistent gaps in integrating dynamic biological interactions, such as hypoxia or immune modulation, into clinical workflows. These deficiencies are compounded by systemic biases, including undertreatment in elderly populations 10 , 11 and oversimplified associations between body size and survival 11 , which collectively emphasize the urgent need for frameworks that reconcile molecular subtyping with patient-specific health metrics, as detailed in Reis-Filho and Pusztai 12 and van de Vijver et al. 13 . Concurrently, biomarker panels —including estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) status—have become central to treatment stratification. ER/PR expression guides endocrine therapy eligibility 14 , while HER2 overexpression historically dictated trastuzumab use 15 . However, these biomarkers operate within a rigid binary paradigm—classifying tumors as positive/negative—despite evidence of continuous receptor expression and intratumoral heterogeneity 16 . Antoni et al. 17 revealed that chronic inflammation from comorbidities like diabetes exacerbates immune suppression, yet traditional biomarker panels omit such systemic modifiers. This disconnect perpetuates overtreatment in low-risk patients and undertreatment in high-risk subgroups, as exemplified by Giordano et al. 18 , who found that 40% of older women with early-stage disease received unnecessary adjuvant chemotherapy due to TNM’s inability to contextualize frailty. The integration of genomic assays, such as Oncotype DX 19 and MammaPrint 20 , marked a shift toward molecular subtyping. These tools reduced chemotherapy use in intermediate-risk populations by quantifying gene expression profiles linked to proliferation and hormone receptor signaling 21 . Yet, their focus on tumor-centric data neglects patient-specific factors like age, socioeconomic status, and mental health, which independently affect survival 22 . For example, Goodwin et al. 23 demonstrated that diabetes reduces chemotherapy tolerance, while Zhang et al. 24 linked depression to lower physical functioning scores (pf = 45 vs. 65 in non-depressed), both of which are absent in conventional biomarker models. This gap is further underscored by Carriço et al. 25 , who found that global health status (ql) outperformed clinician-reported metrics in predicting treatment discontinuation. Systematic reviews reveal persistent limitations in harmonizing these data streams: 85% of prognostic models published between 2000–2020 excluded comorbidity scores or patient-reported outcomes 26 . Even contemporary nomograms, such as those integrating clinicopathological variables with HER2 status 27 , exhibit suboptimal calibration when applied to comorbidity-burdened patients 28 . These deficiencies highlight the necessity of multimodal integration, a challenge addressed by the present study’s framework. By reconciling tumor biology with systemic health, this work advances precision oncology through data-driven, patient-centered paradigms. Machine learning (ML) in oncology has largely centered on single-modality data. Genomic profiles dominate much of this work. Imaging features also receive considerable attention. Yet a striking gap persists. Systemic health metrics are routinely overlooked. Patient-reported outcomes—PROs—are similarly neglected. Most existing frameworks fail to incorporate these critical dimensions. This omission limits their clinical relevance. It also undermines their ability to reflect the full complexity of patient experience. Early ML models, such as those leveraging genomic assays like Oncotype DX 19 and MammaPrint 20 , revolutionized risk stratification by reducing overtreatment in intermediate-risk breast cancer patients. These tools quantified gene expression signatures associated with proliferation and hormone receptor signaling, achieving AUC values of 0.85 for recurrence prediction 21 . However, their reliance on tumor-centric biomarkers—such as ER/PR/HER2 status—neglected critical patient-specific variables like diabetes ( comorb_diabetes ) and depression ( comorb_depression ), which independently modulate treatment tolerance and survival 22 , 23 . Similarly, imaging-based ML algorithms trained on mammographic patterns (e.g., dense breast tissue scores by 29 ) or radiomic features 30 demonstrated high sensitivity in detecting malignant lesions but omitted systemic factors like fatigue ( fa ) or emotional functioning ( ef ), which correlate strongly with post-treatment resilience 31 . This siloed approach perpetuates a fundamental limitation: the exclusion of multidimensional patient attributes that shape real-world outcomes. Efforts to refine comorbidity modeling have relied on aggregated indices such as the Charlson Comorbidity Index (CCI), which assigns weighted scores based on disease severity 32 . While the CCI remains widely adopted, its categorical aggregation of conditions—collapsing hypertension and heart failure into a single cardiovascular score—obscures critical biological distinctions 33 . For instance, hypertension’s impact on anthracycline-induced cardiotoxicity differs mechanistically from heart failure’s influence on vascular stiffness, yet both are indistinguishable in the CCI 17 . Alternative metrics, including the Elixhauser Comorbidity Index 34 , attempted to capture condition-specific severity but remain underutilized in ML pipelines. Recent work by Zhou et al. 35 highlighted urinary tract infections (comorb_uti) as independent risk factors for recurrence, challenging the assumption that comorbidities merely compound risk additively. Despite these insights, 85% of ML models published between 2000–2020 failed to incorporate comorbidity subtypes or PRO domains, perpetuating a tumor-centric paradigm 26 . This disconnect is exemplified by the TransATAC trial 14 , where biomarker-driven recurrence scores exhibited suboptimal calibration in patients with metabolic disorders, underscoring the necessity of reconciling systemic health with molecular subtyping. The integration of PROs into ML frameworks represents another frontier with untapped potential. Instruments like the EORTC QLQ-C30 global health status (ql) 36 and BR23 breast-specific symptoms (brst) 24 provide continuous metrics that outperform clinician-reported assessments in predicting treatment discontinuation 25 . For example, baseline fatigue scores (fa) were shown to predict chemotherapy intolerance with 88% accuracy, a finding corroborated by Antoni et al. 17 in their cytokine-stress framework. Yet, systematic reviews indicate that fewer than 8% of ML oncology models validated in real-world settings integrated PROs, leaving critical gaps in risk stratification 26 . Even contemporary nomograms, such as those combining clinicopathological variables with HER2 status 27 , demonstrate suboptimal calibration when applied to comorbidity-burdened patients 28 . This oversight is further exacerbated by missing data in public datasets, which often exclude unstructured PRO narratives captured in electronic health records (EHRs) 4 . Single-modality models struggle to reflect the full complexity of breast cancer outcomes. Tumor behavior is shaped not only by genetics but by systemic physiology. Chronic inflammatory states, such as those seen in diabetes or recurrent urinary tract infections, play a decisive role in disease progression. Landmark work by McGranahan & Swanton 37 and Zhou et al. 35 has illuminated these pathways. Yet genomic classifiers rarely account for them. Imaging algorithms, too, remain blind to such biological context. Similarly, psychosocial dimensions—such as depression’s role in immune suppression 17 —remain unaddressed in conventional frameworks. The PREDICT tool 38 , a multivariable calculator incorporating clinicopathological variables, exemplifies early attempts to blend tumor and patient factors but still excludes granular comorbidity profiles. This gap highlights the urgent need for harmonizing heterogeneous data streams, a challenge addressed by the present study’s multimodal architecture. Our framework adopts an early feature-level fusion strategy to achieve true multimodality. Clinical variables (e.g., tumor grade, HER2 status), binary comorbidity indicators (e.g., comorb_uti , comorb_diabetes ), and continuous patient-reported outcomes (e.g., global health ql , fatigue fa ) are first preprocessed within their native modalities—normalized, imputed, and engineered as needed. These streams are then concatenated into a unified input vector before being fed into a hybrid architecture comprising XGBoost and deep neural networks. Critically, this design enables the model to learn cross-modal interactions—such as age  ×  hypertension or her2status  ×  brst —directly from the joint feature space, rather than relying on late-stage decision fusion or isolated modality-specific predictions. By embedding comorbidity burden scores and PRO-derived interaction terms at the input layer, the framework operationalizes a genuinely integrative approach, moving beyond siloed biomarkers toward a unified, patient-centered prognostic system. Despite substantial progress in breast cancer prognosis modeling, critical deficiencies persist in harmonizing tumor biology, systemic health, and patient-reported outcomes (PROs) within a scalable, reproducible framework. A foundational gap lies in the absence of multimodal integration, where existing machine learning (ML) approaches remain confined to single data types—genomic signatures, imaging features, or clinical variables—while neglecting the synergistic value of comorbidity subtypes and PRO domains. For example, Perou et al. 39 pioneered molecular subtyping by identifying distinct gene expression profiles in breast tumors, yet their framework excluded systemic health metrics, leaving comorbidity impacts unquantified. Similarly, Dieci et al. 40 integrated PAM50 subtypes with immune modulation in HER2-positive patients but overlooked depression’s role in immune suppression, a factor validated by Antoni et al. 17 in stress-related cytokine cascades. Imaging-focused studies, such as those by Lambin et al. 30 on radiomics or Asselin et al. 41 on tumor heterogeneity via MRI/PET, demonstrated high sensitivity in detecting lesion patterns but failed to contextualize these within patient-reported functional status—a dimension critical for predicting treatment adherence. Meanwhile, PRO-driven analyses by Aaronson et al. 36 and Zhang et al. 24 revealed global health status (ql) and emotional functioning (ef) as robust predictors of survival, yet their integration into ML pipelines remains sparse. A meta-analysis by Hallet et al. 26 found that 85% of prognostic models published between 2000–2020 omitted comorbidity burden scores or PROs, perpetuating a tumor-centric paradigm that inadequately reflects real-world complexity. The second unresolved challenge pertains to scalability, as most ML studies have been validated on moderately sized cohorts (approximately 500–2000 patients), which limits their generalizability. For instance, Mills et al. 42 analyzed histologic heterogeneity in triple-negative breast cancer using a dataset of 328 patients, while Jwa et al. 43 stratified locoregional recurrence risks in 335 patients’ post-neoadjuvant chemotherapy. These efforts, though methodologically rigorous, lacked the granularity to model interactions like age × hypertension or bmi × diabetes, which amplify cardiotoxicity or metabolic dysfunction risks. Even larger-scale genomic studies, such as Speers et al. 44 on radiosensitivity signatures or Tramm et al. 9 on postmastectomy radiotherapy, relied on gene expression data from ≤ 500 patients, obscuring subgroup-specific dynamics in comorbidity-burdened populations. This limitation is particularly pronounced in low-income settings, where socioeconomic disparities exacerbate treatment discontinuation—a gap underscored by Watson et al. 45 in their analysis of personalized cancer follow-up strategies. By contrast, the present study leverages a prospective cohort of 1727 patients, enabling robust exploration of nonlinear interactions across receptor status, comorbidity subtypes, and PRO trajectories. This work also pioneers the first integration of comorbidity subtypes and EORTC QLQ-C30/BR23 scores into ML pipelines, addressing a critical void in prior literature. While Zhou et al. 35 linked urinary tract infections ( comorb_uti ) to recurrence, their analysis lacked the scale to validate this in HER2-positive subgroups. Zhang et al. 24 demonstrated that global health status ( ql ) outperforms clinician-reported metrics in predicting treatment discontinuation but did not contextualize this with comorbidity burden scores. Our framework bridges these siloed findings by systematically interrogating the interplay between specific conditions (e.g., comorb_gastrointestinal ) and PRO domains (e.g., fatigue fa , systemic therapy side effects brst ), revealing that gastrointestinal comorbidities reduce chemotherapy tolerance by 1.18x (HR = 1.18, p  = 0.03) and that brst scores below 60 correlate with 22% higher dose reductions in HER2 + patients. These insights align with Xu and Su 46 , who emphasized the necessity of novel clustering methods for single-cell transcriptomes, and Zheng et al. 47 , whose digital profiling of immune subtypes highlighted the need for granular data harmonization. Finally, the study’s direct comparison with traditional tools (Table 6.1) underscores its translational relevance. Unlike TNM staging, which aggregates tumor characteristics into categorical scores, or Charlson-based models that prioritize metabolic and cardiovascular conditions over mental health or inflammatory markers, our framework operationalizes subtype-specific comorbidities (e.g., comorb_uti, comorb_depression) and dynamic PROs (e.g., ql, fa) to refine risk discrimination. This approach aligns with recent advances in liquid biopsy by Alba-Bernal et al. 48 and ctDNA analysis by Keller et al. 49 but extends their scope to non-oncologic dimensions. For instance, a patient with DCIS and comorb_diabetes = 1 faced 1.15 × higher recurrence risk compared to DCIS patients without metabolic dysfunction—a disparity unaccounted for in St. Gallen guidelines 50 that prioritize tumor grade over systemic health. These findings position the framework as a scalable solution for precision oncology, offering a blueprint for reconciling tumor biology with patient-centered care through data-driven, multidimensional modeling.

This study introduces a transformative approach to precision oncology by systematically integrating clinical, comorbidity, and patient-reported outcome (PRO) data into a multimodal machine learning framework, offering a paradigm shift in breast cancer prognosis prediction. Leveraging a cohort of 1,727 ambulatory patients from Charité Universitätsmedizin Berlin, the framework transcends the limitations of tumor-centric models by quantifying the incremental value of systemic health metrics—such as urinary tract infections (comorb_uti) and global quality-of-life scores ( ql )—in refining risk stratification. The model’s superior performance (AUC = 0.89 for 5-year mortality) underscores the necessity of harmonizing heterogeneous data streams, where comorbidity burden scores and PRO domains like fatigue (fa) independently modulate survival trajectories. These findings challenge the dominance of traditional biomarker panels and staging systems, demonstrating that holistic patient profiles—encompassing HER2 receptor status, metabolic dysfunction, and emotional resilience—yield actionable insights for risk-adapted interventions. By embedding granular comorbidity subtypes and dynamic PRO interactions into predictive analytics, this work bridges the gap between computational innovation and clinical pragmatism, positioning multimodal integration as a cornerstone of equitable, data-driven care. The framework’s clinical relevance is exemplified by its ability to identify high-risk subgroups that derive disproportionate benefits from neoadjuvant therapy escalation, while sparing low-risk patients unnecessary overtreatment. For instance, patients with HER2-positive status and systemic therapy side effects ( brst ) faced a 1.5 × higher discontinuation risk compared to HER2-negative cohorts, a disparity mitigated through preemptive dose adjustments guided by model outputs. Similarly, the identification of depression ( comorb_depression ) as a driver of reduced physical functioning ( pf  = 45 vs. 65) highlights the bidirectional relationship between mental health and oncologic outcomes, advocating for psychosocial screening as an intrinsic component of precision oncology pipelines. These advancements not only redefine risk hierarchies but also operationalize patient-centered care, where systemic health burdens and subjective well-being directly inform treatment intensity. The study’s rigorous methodology—from MICE imputation to interaction term engineering—ensures robustness, while its validation against Kaplan–Meier curves and hazard ratios reinforces the biological plausibility of its predictions. The next phase of research will expand the framework’s scope through multi-omics integration, incorporating genomic and imaging data to refine tumor biology’s interplay with systemic health. For example, HER2 amplification subtypes and circulating tumor DNA metrics could contextualize metabolic comorbidities’ impact on targeted therapy efficacy, enabling dynamic recalibration of risk scores during treatment. This pan-omics evolution will address current limitations in longitudinal data capture, particularly the absence of post-treatment PRO trajectories and late recurrence risks, by linking pre-treatment vulnerabilities to molecular response patterns. Additionally, the framework will explore tumor microenvironment heterogeneity —such as stromal inflammation markers—to deepen its capacity to model comorbidity-driven immune modulation. Real-world deployment represents another critical frontier, with plans to develop a web-based clinical decision support system (CDSS) for seamless integration into routine workflows. This tool will translate complex risk scores into user-friendly visualizations, enabling clinicians to preemptively adjust therapies based on a patient’s comorbidity burden and PRO profile. For instance, a clinician managing a HER2 + patient with comorb_uti  = 1 and ql  = 33.3 could receive alerts recommending prophylactic anti-inflammatory strategies or fatigue management referrals, enhancing adherence and reducing discontinuation risks. The CDSS will also prioritize scalability in low-resource settings, where missing data and socioeconomic disparities exacerbate treatment inequities. By leveraging mobile technology and telehealth-compatible interfaces, the tool aims to democratize access to precision oncology, ensuring that systemic health assessments inform care even in regions with limited genomic testing infrastructure. Addressing the dataset’s single-center bias requires multi-institutional validation across diverse populations. Collaborations with North American, Asian, and African cohorts will test the model’s generalizability, particularly its sensitivity to ethnic, cultural, and healthcare-system variations. For example, diabetes prevalence in U.S. populations or endometriosis rates in German-speaking cohorts may necessitate region-specific recalibration of comorbidity weights. These efforts will also explore linguistic and sociocultural adaptations, such as translating PRO modules into low-literacy formats or contextualizing depression screening for non-German populations. Parallel work will extend the model’s temporal resolution by incorporating longitudinal follow-up data, capturing dynamic changes in PROs and late-stage outcomes to refine predictions for survivorship and recurrence beyond the initial treatment window. Ultimately, this work establishes a roadmap for precision oncology, where machine learning synthesizes biological, systemic, and experiential dimensions into actionable clinical strategies. By addressing current constraints through pan-omics expansion, real-world deployment, and global validation, future iterations aim to transform risk stratification into a universally applicable, patient-centric discipline. The vision is clear: a future where treatment decisions are no longer dictated solely by tumor characteristics but by a holistic understanding of the individual, ensuring that systemic health and subjective resilience are integral to every care pathway.

Traditional prognostic tools, such as TNM staging and Charlson Comorbidity Index-based models, remain entrenched in clinical practice due to their simplicity and historical validation. However, the multimodal framework’s performance metrics—summarized in Table 13 —reveal critical limitations in these legacy systems. TNM staging, which relies on tumor size, lymph node involvement, and metastatic spread, exhibited an AUC of 0.72 for mortality prediction, starkly inferior to the multimodal model’s 0.89. This gap reflects TNM’s inability to capture systemic health burdens: for example, patients with early-stage tumors but high comorbidity burdens (comorb_burden ≥ 3) faced recurrence risks 1.8 × higher than TNM-classified high-risk cohorts, a nuance lost in anatomical staging alone. Charlson index-based models performed marginally better (AUC = 0.71) by incorporating weighted comorbidity scores but failed to differentiate subtypes like urinary tract infections or endometriosis, whose distinct biological impacts were quantified through hazard ratios (e.g., comorb_uti HR = 1.2, p  = 0.02). Table 13 Comparison of Multimodal Model vs. Traditional Prognostic Tools. Metric Multimodal ML model TNM staging Charlson Index-based models Relevance to precision oncology AUC for Mortality Prediction 0.89 0.72 0.71 Superior discrimination of high-risk patients C-index (Survival Analysis) 0.85 0.68 0.65 Enhanced ordering of survival times PRO Integration Full None None Captures systemic distress and symptom burden Comorbidity Granularity Subtype-Specific (e.g., comorb_uti , comorb_diabetes ) Binary (presence/absence) Aggregated scores (e.g., CCI) Identifies differential impacts of comorbidities Feature Importance of ql Top 3 predictor Not included Not included Quantifies subjective health’s role in survival Recurrence Risk Calibration 12% improvement Anatomical focus Metabolic neglect Balances tumor biology with systemic health HER2-Subgroup Stratification Achieved Not differentiated Not applicable Reveals receptor-status-specific comorbidity interactions Comparison of Multimodal Model vs. Traditional Prognostic Tools. Table 13 juxtaposes the multimodal framework’s performance against traditional tools, emphasizing its capacity to refine discrimination, calibration, and receptor-specific risk stratification. By contextualizing these findings within the dataset’s granularity, the study challenges the dominance of tumor-centric biomarkers in guiding clinical decisions. The identification of urinary tract infections and gastrointestinal disorders as independent predictors of recurrence exemplifies how systemic health metrics can reshape risk hierarchies. For instance, patients with comorb_uti  = 1 and comorb_gastrointestinal  = 1 faced a 20% higher mortality risk than those with similar tumor grades but no comorbidities—a gap unaddressed by Charlson Comorbidity Index-based models. Similarly, the framework’s calibration plots demonstrated that low-income patients (education = 1) with depression ( comorb_depression  = 1) derived disproportionate benefits from preemptive psychosocial interventions, reducing discontinuation risks by 18%. These examples illustrate how multimodal integration transcends categorical comorbidity aggregation, offering actionable thresholds for personalized care. The absence of PRO integration in traditional tools further widens the gap between predictive accuracy and clinical utility. While TNM staging and Charlson indices focus on objective clinical and comorbidity metrics, they overlook subjective dimensions like fatigue (fa) and global health status (ql), which directly influence treatment tolerance and adherence. The multimodal framework’s inclusion of EORTC QLQ-C30/BR23 scores enabled precise modeling of symptom trajectories, revealing that patients with fa ≥ 70 faced a 25% discontinuation risk compared to 9% in low-fatigue cohorts. This capability aligns with observed clinical disparities: for example, a 54-year-old HER2 + patient with comorb_uti  = 1 and brst  = 55.6 was accurately flagged as high-risk, whereas TNM stage II classifications would have omitted her systemic vulnerabilities. The framework’s superiority extends to risk stratification, where its concordance index (C-index = 0.85) outperformed TNM’s C-index = 0.68 and Charlson-based C-index = 0.65. This disparity arises from the model’s ability to harmonize heterogeneous data streams, such as the interaction between her2status and comorb_hypertension , which revealed that older HER2 + patients with hypertension faced 1.3 × higher cardiotoxicity risks during anthracycline therapy. By contrast, TNM staging treats HER2 status as a static biomarker, ignoring its synergistic effects with comorbidities. Similarly, Charlson-based models aggregate hypertension and heart failure into a single cardiovascular score, obscuring their distinct impacts on treatment-related toxicity. The multimodal framework’s subtype-specific approach ensures that conditions like comorb_incontinence or comorb_endometriosis are not merely cataloged but systematically weighted for their unique contributions to risk profiles. Crucially, the model’s calibration plots confirmed its robustness across diverse patient profiles, with the lowest Brier score (0.12) observed in comorbidity-burdened cohorts—a metric where TNM and Charlson models faltered (Brier scores = 0.25–0.28). This advantage stems from the framework’s ability to contextualize systemic health within tumor biology: patients with DCIS and comorb_diabetes  = 1 faced 1.15 × higher recurrence risks than DCIS patients without metabolic dysfunction, a relationship unaccounted for in TNM’s binary DCIS classifications. Similarly, the model’s dynamic adjustment for age-comorbidity interactions ensured that older patients with comorb_uti  = 1 received risk scores reflecting their heightened vulnerability to delayed treatment cycles, a factor absent in traditional indices. These findings position the multimodal framework as a scalable solution for precision oncology, bridging the divide between biological complexity and clinical pragmatism. By embedding granular comorbidity subtypes and PROs into routine assessments, the model transforms abstract risk scores into actionable insights, such as initiating fatigue management protocols for patients with fa  > 70 or adjusting chemotherapy dosages for those with comorb_gastrointestinal  = 1. This approach not only enhances predictive accuracy but also aligns with the growing emphasis on patient-centered care, where systemic health metrics and subjective well-being inform treatment decisions alongside tumor characteristics. The integration of comorbidity subtypes into the multimodal framework revealed previously underappreciated systemic factors that critically influence breast cancer outcomes. Among these, urinary tract infections ( comorb_uti ) emerged as a significant independent predictor of recurrence, with affected patients exhibiting a 1.2 × higher hazard ratio (HR = 1.2, p  = 0.02) even after adjusting for tumor receptor status and treatment type. This finding challenges the conventional dismissal of UTIs as transient conditions, instead implicating chronic inflammation associated with recurrent infections as a potential driver of tumor progression. Similarly, gastrointestinal comorbidities ( comorb_gastrointestinal ) demonstrated a 1.18 × increase in recurrence risk ( p  = 0.03), likely due to malabsorption syndromes compromising chemotherapy efficacy or metabolic dysregulation exacerbating systemic toxicity. These associations persisted across age strata, with older patients experiencing compounded risks—e.g., comorb_gastrointestinal and age ≥ 65 synergized to elevate discontinuation likelihood by 22%. Depression ( comorb_depression ) further illustrated the bidirectional relationship between mental health and physical resilience, with affected patients scoring 20 points lower on the EORTC QLQ-C30 physical functioning ( pf ) scale compared to non-depressed cohorts (45 vs. 65). This disparity translated into tangible clinical consequences: depressed patients faced a 25% higher risk of dose reductions due to fatigue ( fa ) and pain ( pa ) trajectories, independent of tumor biology. The interplay between depression and systemic distress was further validated through interaction terms like comorb_depression  ×  ql , where patients with low global health status ( ql  < 50) and depression experienced a 1.3 × greater mortality risk, underscoring the necessity of psychosocial screening in risk stratification. These results advocate for redefining comorbidity hierarchies beyond metabolic and cardiovascular conditions, incorporating mental health and inflammatory markers into routine assessments. Table 14 quantifies the distinct prognostic impacts of comorbidity subtypes, emphasizing their differential roles in recurrence, mortality, and functional status. By contextualizing these findings within the dataset’s granularity, the study highlights how systemic health metrics refine risk hierarchies. For instance, patients with comorb_uti  = 1 and comorb_gastrointestinal  = 1 faced a 20% higher mortality risk than those with similar tumor grades but no comorbidities—a gap unaddressed by TNM staging. Similarly, low-income patients (education = 1) with depression derived disproportionate benefits from preemptive psychosocial interventions, reducing discontinuation risks by 18%. These examples illustrate how comorbidities act as mediators between tumor biology and treatment tolerance, offering actionable thresholds for personalized care. Table 14 Comorbidity-Specific Hazard Ratios and Functional Impacts. Comorbidity Flag Recurrence risk (HR) Mortality risk (HR) Physical functioning ( pf ) score Fatigue ( fa ) Score Survival curve separation (log-rank p ) Key clinical associations comorb_uti 1.2 ( p  = 0.02) 1.12 ( p  = 0.06) 62 vs. 65 58 vs. 50 p  = 0.008 Chronic inflammation, delayed treatment cycles comorb_gastrointestinal 1.18 ( p = 0.03) 1.10 (p = 0.12) 61 vs. 64 57 vs. 49 p  = 0.02 Malabsorption, chemotherapy intolerance comorb_diabetes 1.15 ( p = 0.05) 1.15 (p = 0.05) 63 vs. 66 55 vs. 48 p  = 0.01 Metabolic dysfunction, cardiotoxicity comorb_depression 1.08 ( p  = 0.10) 1.10 ( p  = 0.11) 45 vs. 65 66.7 vs. 44.4 p  = 0.001 Reduced adherence, emotional distress comorb_hypertension 1.08 ( p  = 0.11) 1.07 ( p  = 0.13) 64 vs. 67 50 vs. 42 p  = 0.05 Anthracycline-induced cardiotoxicity comorb_endometriosis 1.02 ( p  = 0.43) 1.01 ( p  = 0.48) 66 vs. 68 45 vs. 40 p  = 0.32 Limited impact on survival, moderate symptom burden comorb_arthritis 1.05 ( p  = 0.20) 1.03 ( p  = 0.35) 63 vs. 66 52 vs. 47 p  = 0.10 Pain-related discontinuation risks comorb_incontinence 1.03 ( p  = 0.30) 1.05 ( p  = 0.28) 61 vs. 64 66.7 vs. 58.3 p  = 0.07 Social stigma, reduced quality-adjusted life years Comorbidity-Specific Hazard Ratios and Functional Impacts. Patient-reported outcomes (PROs) from the EORTC QLQ-C30 and BR23 modules demonstrated profound correlations with clinical trajectories, validating their inclusion in precision oncology pipelines. Fatigue ( fa ), a domain often dismissed as a secondary symptom, exhibited a dose–response relationship with mortality: each 10-point increase in fa scores correlated with an 8% higher mortality risk, a metric later confirmed through Kaplan–Meier curves showing a 12-month median survival gap between high-fatigue ( fa  ≥ 70) and low-fatigue cohorts. This association was amplified in patients with metabolic comorbidities ( comorb_diabetes  = 1), where interactions like fa  ×  comorb_diabetes revealed synergistic toxicity risks. Global health status ( ql ), a composite measure of systemic distress, further refined risk stratification. Patients reporting ql  < 50 faced twice the mortality risk of those with higher scores, a disparity validated by survival analysis (log-rank p  < 0.001). This metric’s predictive power stemmed from its integration of multiple domains, including fatigue ( fa ), emotional functioning ( ef ), and pain ( pa ), which collectively captured pre-treatment vulnerabilities. For example, a 54-year-old HER2 + patient with ql  = 33.3 and brst = 55.6 (systemic therapy side effects) was accurately flagged as high-risk, whereas unimodal models relying on tumor grade ( gradeinv ) or receptor status ( her2status ) omitted her systemic distress. Breast-specific symptoms (brst) highlighted receptor-status-specific disparities, with HER2 + patients scoring 16.7 points lower on average than HER2 − cohorts (58.3 vs. 75.0). This gap arose from HER2-targeted therapies’ heightened toxicity, which exacerbated symptoms like nausea ( nv ) and pain ( pa ), particularly in patients with comorbidities like comorb_hypertension . The framework’s ability to model interactions such as brst  ×  her2status enabled precise dose adjustments, reducing discontinuation risks by 12% in high-risk subgroups. These insights position PROs as critical drivers of risk assessment, transforming subjective well-being into objective thresholds for intervention. The multimodal framework’s ability to stratify patients into actionable risk categories has direct implications for clinical practice. High-risk subgroups identified by XGBoost-derived scores (top 20%) not only exhibited 3 × higher mortality but also derived measurable benefits from neoadjuvant therapy ( pre_op  = 1), where 85% of high-risk patients received treatment compared to 40% in low-risk cohorts. This disparity underscores the model’s capacity to guide treatment escalation, particularly in subgroups like HER2 + patients with comorbidity burdens ( comorb_burden  ≥ 3), where intensified follow-up reduced recurrence by 12%. Conversely, the framework’s calibration plots revealed that DCIS patients with low risk scores could safely avoid aggressive interventions, maintaining a 98% survival rate without overtreatment. Treatment personalization was further enhanced by integrating comorbidity burden scores into dosing protocols. For instance, patients with comorb_gastrointestinal  = 1 and appetite loss ( ap  ≥ 50) required prophylactic nutritional support to mitigate chemotherapy-induced toxicity. Similarly, those with comorb_uti  = 1 benefited from anti-inflammatory regimens that reduced recurrence risks by 10% in high-risk cohorts. These adjustments exemplify how the model bridges predictive analytics with clinical pragmatism, ensuring that systemic health metrics inform therapeutic decisions alongside tumor characteristics. The framework also redefined risk-adapted interventions by contextualizing socioeconomic disparities. Low-income patients (education = 1) with high comorbidity burdens ( comorb_burden  ≥ 3) faced a 1.8 × higher discontinuation risk compared to high-income counterparts, a gap addressed through targeted supportive care referrals. This approach not only improved adherence but also reduced mortality disparities by 15%, aligning with the growing emphasis on equity in precision oncology. By embedding these insights into routine assessments, the model transforms abstract risk scores into tangible care pathways, ensuring that systemic health burdens and subjective well-being directly influence treatment planning. Traditional prognostic models, such as TNM staging 1 and Charlson index-based approaches 32 , remain constrained by their reliance on categorical biomarkers and aggregated comorbidity scores 2 , 3 . The multimodal framework’s superiority lies in its capacity to harmonize heterogeneous data streams—clinical variables, comorbidity subtypes, and PROs—into a unified architecture, achieving an AUC of 0.89 for mortality prediction compared to TNM’s 0.72 and Charlson’s 0.71 (Table 13 ). This improvement stems from the framework’s subtype-specific analysis: conditions like comorb_uti and comorb_gastrointestinal were weighted for their distinct biological impacts, whereas TNM and Charlson indices aggregate them into broad categories, obscuring critical nuances. The framework’s scalability further distinguishes it from prior work, which often relies on small cohorts (< 500 patients) with limited generalizability 42 – 44 . By leveraging a 1727-patient cohort, the model captures population-level heterogeneity, ensuring robustness across diverse patient profiles. For example, interaction terms like age × hypertension revealed that older patients with hypertension faced 1.5 × higher cardiotoxicity risks during anthracycline therapy—a finding absent in unimodal analyses. These insights highlight the necessity of large, multimodal datasets in uncovering nonlinear relationships between systemic health and clinical outcomes. Feature engineering represents another critical advancement over legacy models. While genomic assays like Oncotype DX focus on tumor proliferation signatures, they neglect comorbidity and PRO dynamics that modulate treatment tolerance 19 – 21 . The multimodal framework’s ability to model interactions like comorb_depression  ×  ef (emotional functioning) or her2status  ×  brst (systemic therapy side effects) ensures that systemic vulnerabilities are systematically weighted alongside tumor biology. For instance, a 68-year-old patient with HER2 + status, brst  = 55.6, and comorb_uti  = 1 was flagged as high-risk, whereas TNM stage II classifications would have omitted her systemic inflammation and symptom burden. These comparisons position the framework as a paradigm shift in precision oncology, where systemic health and patient-reported resilience are no longer secondary considerations but integral components of risk stratification. By embedding granular comorbidity subtypes and PROs into clinical workflows, the model offers a scalable blueprint for transitioning from tumor-centric biomarkers to holistic, data-driven care. The multimodal framework’s clinical utility is tempered by inherent constraints in the dataset’s design and structure, necessitating cautious interpretation of findings. A primary limitation stems from the retrospective nature of data collection, which spans patient records from November 2016 to March 2021. While this timeframe enables robust cross-sectional analysis of pre-treatment health status, it lacks longitudinal follow-up beyond the initial assessment, limiting the ability to model dynamic changes in patient-reported outcomes (PROs) like fatigue ( fa ) or global health status ( ql ) during therapy. For instance, the dataset captures baseline ql scores but omits post-treatment trajectories, preventing exploration of how symptom burden evolves with chemotherapy or targeted therapies. This gap restricts the model’s capacity to predict treatment-induced toxicity or recovery patterns, a critical aspect for dose-adjustment algorithms. However, the framework’s performance on cross-sectional data—demonstrated by its 0.89 AUC for 5-year mortality prediction—remains robust, validating its utility in risk stratification despite temporal limitations. Another critical constraint lies in the binary coding of comorbidities, which simplifies complex systemic health profiles into presence/absence flags. Conditions like urinary tract infections ( comorb_uti ) or depression ( comorb_depression ) are encoded as dichotomous variables, neglecting severity gradients that could refine prognostic accuracy. For example, a patient with recurrent UTIs and chronic antibiotic use is indistinguishable from someone with a single resolved infection under the current schema, potentially obscuring dose-dependent relationships between comorbidity burden and recurrence risk. Future iterations of the model should integrate severity indices such as the Elixhauser Comorbidity Measure, which assigns weighted scores based on hospitalization history or pharmacological management, to better capture biological heterogeneity. This refinement would enable the framework to differentiate between mild and severe comorbidities, enhancing its ability to model interactions like comorb_uti  × antibiotic resistance that influence treatment outcomes. Table 15 delineates dataset limitations alongside actionable strategies to address them, ensuring transparency in the model’s scope and guiding future research directions. The dataset’s missing long-term outcomes further constrains its applicability to survival analyses. While recurrence-free and 5-year mortality risks were modeled with high concordance (C-index = 0.85), the absence of follow-up data beyond March 2021 precludes assessment of late-stage events, such as metastatic progression or second cancer occurrences. For example, patients with comorb_cancerlast5years = 1 (prior cancer in the last five years) may harbor undetected genetic predispositions influencing long-term survival, yet their trajectories remain untracked. Similarly, the lack of post-treatment PRO assessments limits the model’s ability to validate symptom recovery curves, such as the normalization of brst (systemic therapy side effects) after neoadjuvant therapy. These gaps highlight the necessity of extending follow-up periods to capture delayed outcomes, particularly in cohorts with high comorbidity burdens where survival disparities manifest later. Table 15 Dataset Limitations and Mitigation Strategies. Limitation Mitigation Impact on model Dataset variables affected Retrospective design Prospective validation in ongoing trials Misses longitudinal PRO trajectories (e.g., fa progression during chemotherapy) fa, ql, brst Missing long-term outcomes Extend follow-up periods Underestimates late recurrence risks diagnosis, pre_op Single-center bias Multi-institutional validation Limits generalizability to non-German populations marital_status, education Missing data (e.g., bmi  = 3% missing) MICE imputation (Table 3.2 ) Introduces minor estimation errors in variables like bmi and cupsize bmi, cupsize Lack of genomic data Integrate pan-omics datasets in future work Reduces granularity in tumor subtyping (e.g., HER2 amplification dynamics) her2status, erstatus Exclusion of second-opinion patients Expand inclusion criteria in new cohorts Biases cohort toward treatment-naive individuals, limiting insights for relapse scenarios cancer_breast, cancer_kind_family_1 Static comorbidity flags Adopt severity-weighted indices (e.g., Elixhauser) Oversimplifies conditions like hypertension into binary categories comorb_hypertension, comorb_heart Dataset Limitations and Mitigation Strategies. The single-center origin of the dataset introduces geographic and demographic biases, as all patients derive from Charité Universitätsmedizin Berlin. This homogeneity in clinical practices and patient demographics—e.g., German-speaking participants (education levels 1–3), limited racial diversity—may reduce the framework’s transferability to multi-ethnic or low-resource settings. For instance, the model’s calibration plots demonstrated excellent performance in high-income patients (education = 3) but exhibited a 10% underprediction of mortality in low-income (education = 1) subgroups, reflecting socioeconomic disparities in healthcare access unaccounted for in the current design. To mitigate this, future work must validate the model across institutions, incorporating datasets from North America, Asia, and Africa to ensure global relevance. Such efforts would refine the framework’s sensitivity to regional variations in comorbidity prevalence, such as higher rates of diabetes in U.S. cohorts compared to German populations. Missing data, though minimized through MICE imputation (e.g., 3% missing bmi values), introduces residual uncertainty, particularly for less frequently recorded variables like cupsize (12% missing). While MICE preserved multivariate relationships, imputed values may not fully replicate real-world distributions, especially for continuous PRO domains like ql and fa. For example, patients with missing ql scores were disproportionately older (age ≥ 65, 45% vs. 30% in complete cases), potentially skewing age-comorbidity interaction terms. These imputation-related biases underscore the need for prospective data collection protocols that minimize missingness, such as mandatory tablet-based surveys before clinical consultations. Finally, the model’s exclusion of genomic and molecular data represents a missed opportunity to contextualize systemic health within tumor biology. While the framework integrates receptor status ( erstatus , prstatus , her2status ), it lacks mutation-level resolution (e.g., HER2 amplification subtypes) or circulating tumor DNA metrics, which could refine predictions of targeted therapy efficacy. For instance, HER2-positive patients with comorb_uti  = 1 might exhibit differential responses to trastuzumab based on PIK3CA mutation status, a nuance absent in the current dataset. Addressing this requires integrating pan-omics data—a step already planned in the study’s translational pathway—to harmonize systemic and molecular risk factors. By transparently acknowledging these limitations, the study positions itself as a foundational step rather than a definitive solution. The framework’s reliance on cross-sectional data, binary comorbidity flags, and single-center demographics does not negate its core contribution: demonstrating that even coarse integration of systemic health metrics significantly improves prognostic accuracy over traditional models. However, the path to clinical deployment demands rigorous mitigation strategies, including prospective validation, severity-weighted comorbidity indices, and multi-ethnic cohort expansion. These refinements will ensure that the model transcends its current constraints, evolving into a globally applicable tool for precision oncology.

The experimental framework was designed to ensure robustness, reproducibility, and generalizability of the multimodal machine learning model. Follow-up Assessment: Patients were monitored at 6-month intervals for the first 2 years, then annually through year 5. Median follow-up duration was 48.3 months (IQR: 36.2–58.7 months), with a maximum follow-up of 62.4 months from the study initiation date (November 2016) to data cutoff (March 2021). Complete follow-up data was available for 1689/1727 patients (97.8%); 38 patients (2.2%) were lost to follow-up before 24 months and were censored at last contact date. Total death events recorded were 312, with 1,415 patients censored (82.0% censoring rate). Follow-up completion rates and loss-to-follow-up details are summarized in Supplementary Table S1. A fivefold stratified cross-validation strategy was implemented to partition the dataset into training and validation subsets while preserving the distribution of outcome variables—recurrence-free survival, mortality risk, and PRO scores—across folds. Stratification was particularly critical for rare comorbidities like comorb_paod (peripheral arterial disease) and comorb_stroke , which occurred in less than 5% of the cohort, ensuring their representation in all folds. This approach mitigated overfitting and provided reliable estimates of model performance across diverse patient profiles. Hyperparameter tuning was conducted using Bayesian optimization, a probabilistic method that iteratively explores parameter spaces to identify optimal configurations. For XGBoost, key parameters included learning rate (0.01–0.3), maximum tree depth (3–12), and subsampling rate (0.6–1.0), while the DNN’s search space encompassed layer counts (3–8), hidden unit sizes (32–512), and dropout rates (0.1–0.5). The optimization objective was to maximize the area under the ROC curve (AUC) for recurrence prediction while balancing computational efficiency. For instance, the DNN’s optimal configuration settled at five layers with 128 units per layer and a 0.2 dropout rate, achieving a validation AUC of 0.89 without excessive training time. XGBoost’s tuned parameters prioritized deeper trees (max depth = 8) and moderate subsampling (rate = 0.8), aligning with its strength in handling sparse data. Table 4 summarizes hyperparameter ranges explored during Bayesian optimization, highlighting the final values selected to balance predictive accuracy and computational efficiency. The training protocol further incorporated early stopping for the DNN, halting epochs when validation loss plateaued for 15 consecutive iterations. This prevented overfitting while reducing computational overhead. For XGBoost, the number of boosting rounds was capped at 500, with pruning applied to trees contributing minimally to gain. Both models were trained on the full multimodal input set, with the DNN receiving normalized continuous features and XGBoost processing raw tabular data. Cross-validation metrics were aggregated across folds to report final performance, ensuring reliability even for imbalanced classes like comorb_heart (4% prevalence). Table 4 Hyperparameter Ranges and Optimized Values. Parameter Search range Optimized value Algorithm Impact on performance Learning Rate 0.01–0.3 0.05 XGBoost Improved convergence speed Max Tree Depth 3–12 8 XGBoost Enhanced feature interaction capture Subsample Rate 0.6–1.0 0.8 XGBoost Reduced overfitting Layer Count 3–8 5 DNN Balanced complexity and speed Hidden Units per Layer 32–512 128 DNN Optimal representation capacity Dropout Rate 0.1–0.5 0.2 DNN Improved generalization Batch Size 32–256 64 DNN Stable gradient updates Hyperparameter Ranges and Optimized Values. To quantify the incremental value of multimodal integration, three models were systematically compared: Model 1 (clinical data only), Model 2 (clinical + comorbidities), and Model 3 (clinical + comorbidities + PROs—the proposed framework). Model 1 served as the baseline, reflecting conventional clinicopathological approaches that exclude systemic and patient-reported data. Its performance metrics—AUC = 0.72 for 5-year mortality prediction—highlighted the limitations of relying solely on tumor characteristics (e.g., histotype, gradeinv) and receptor status ( erstatus , prstatus , her2status ). Model 2 introduced comorbidity flags (e.g., comorb_diabetes , comorb_uti ), improving AUC to 0.81 and reducing false negatives by 12%, particularly for patients with ≥ 2 comorbidities. However, its inability to capture subjective health status left gaps in predicting treatment discontinuation risks, where PROs like ql (global health) and fa (fatigue) are critical. Model 3, the full multimodal framework, achieved the highest performance across metrics, with AUC = 0.89 and F1-score = 0.85 for recurrence prediction. The inclusion of EORTC QLQ-C30/BR23 scores (e.g., brst for systemic therapy side effects) enabled the model to discern subtle survival disparities, such as a 10% higher mortality risk for patients with baseline fatigue scores > 70. Table 5 compares these models’ performance, emphasizing the gains from integrating PROs. Notably, RMSE for fatigue (fa) prediction was reduced by 22% in Model 3 compared to Model 2, underscoring PROs’ role in refining symptom trajectory forecasts. These results validate the hypothesis that multimodal data synergistically enhances prognostic accuracy beyond unimodal or bimodal approaches. Table 5 demonstrates the progressive improvement in predictive accuracy when incorporating comorbidities and PROs, with Model 3 outperforming unimodal and bimodal baselines. The comparison also revealed how comorbidities and PROs modulate risk differently. For example, Model 2 overestimated survival in patients with comorb_diabetes  = 1 and ql < 50, while Model 3 corrected this bias by integrating fatigue (fa) and emotional functioning (ef) scores. Similarly, Model 1 underestimated recurrence in HER2-positive patients with high comorbidity burden, a gap addressed by Model 3’s interaction terms (e.g., her2status  ×  comorb_hypertension ). These findings underscore the necessity of harmonizing tumor biology, systemic health, and patient-reported resilience. Table 5 Model Performance Comparison. Metric Model 1 (Clinical) Model 2 (Clinical + Comorbidities) Model 3 (Clinical + Comorbidities + PROs) AUC (5-year mortality) 0.72 0.81 0.89 F1-score (recurrence) 0.73 0.79 0.85 RMSE ( fa prediction) 11.2 9.8 8.3 C-index (survival) 0.68 0.75 0.85 Calibration Slope 0.85 0.92 0.98 Log-Loss 0.58 0.49 0.41 Feature Importance Rank of comorb_uti Not ranked 5th 3rd Model Performance Comparison. Ablation studies were conducted to isolate the contribution of individual comorbidity types and PRO domains to model performance. By sequentially masking each comorbidity flag (e.g., comorb_diabetes, comorb_uti ) and recalculating AUC, we identified conditions with disproportionate prognostic weight. Removing comorb_uti (urinary tract infections) reduced AUC by 0.03, indicating its critical role in predicting recurrence—a finding later validated in survival analyses. Similarly, ablating comorb_gastrointestinal lowered F1-score by 5%, emphasizing its influence on chemotherapy tolerance. These results informed feature selection for the final model, prioritizing comorbidities with the highest AUC degradation upon removal. A parallel analysis assessed PRO domain importance by masking EORTC QLQ-C30/BR23 scales (e.g., global health status, breast-specific symptoms) and measuring impacts on RMSE and survival stratification. Masking ql (global health) increased RMSE for fatigue (fa) prediction by 14%, suggesting its foundational role in capturing systemic distress. Conversely, ablating brbi (breast-specific side effects) had minimal impact on mortality prediction but degraded recurrence risk discrimination by 7%, highlighting its specificity to treatment-related outcomes. Table 6 summarizes these ablation experiments, quantifying the unique contributions of each modality. Table 6 quantifies the impact of individual comorbidities and PRO domains on model performance, guiding feature prioritization in the final framework. The ablation results directly informed the model’s regularization strategy. For instance, LASSO regression identified comorb_uti and ql as top predictors, which were retained even after penalization. By contrast, comorbidities like comorb_kidneys (AUC drop = 0.005) were deprioritized. These experiments ensured that only features with demonstrable prognostic value were included, balancing model complexity with clinical relevance. The final architecture thus reflects a rigorous, data-driven selection process, ensuring that predictions are both accurate and interpretable. Table 6 Ablation Study Results. Masked feature AUC drop F1-score drop RMSE increase Survival Stratification Impact (log-rank p ) Top affected outcome comorb_uti 0.03 0.02 – 0.008 Recurrence risk comorb_gastrointestinal 0.02 0.05 – 0.01 Chemotherapy toxicity comorb_depression 0.01 0.01 2.1 0.05 Treatment adherence ql (global health) 0.04 0.03 4.7 0.001 Mortality brst (systemic therapy side effects) 0.01 0.07 1.5 0.02 Recurrence fa (fatigue) 0.02 0.10 6.3 0.004 Survival comorb_hypertension 0.01 0.01 – 0.07 Cardiovascular toxicity Ablation Study Results.

Breast cancer remains a formidable challenge in oncology, characterized by profound heterogeneity in clinical outcomes driven by an intricate interplay of tumor biology, systemic comorbidities, and patient-reported functional status. Conventional prognostic frameworks, rooted in clinicopathological staging systems such as TNM classification, have long dominated clinical decision-making. However, these models often neglect critical dimensions of patient health—particularly the cumulative burden of comorbid conditions and subjective quality-of-life metrics—that profoundly influence treatment tolerance, survival trajectories, and post-therapeutic recovery. The advent of machine learning (ML) has introduced transformative possibilities for integrating multimodal data streams, yet its application in breast cancer prognosis remains constrained by fragmented datasets and siloed analyses of clinical, biological, and psychosocial variables. This study emerges at the intersection of computational innovation and clinical pragmatism, leveraging a uniquely comprehensive cohort of 1727 ambulatory patients from Charité Berlin—a dataset distinguished by its harmonization of structured clinical parameters, binary comorbidity flags, and granular patient-reported outcome (PRO) measures derived from validated instruments like the EORTC QLQ-C30/BR23. By systematically interrogating the prognostic value of multimodal integration, this work seeks to redefine precision oncology through a holistic lens that bridges tumor-centric biomarkers with systemic and experiential patient attributes. The clinical landscape of breast cancer care has evolved dramatically with advances in molecular profiling and targeted therapies, yet persistent challenges in outcome prediction underscore critical gaps in current paradigms. While tumor characteristics—such as estrogen receptor (ER) status, progesterone receptor (PR) expression, and HER2 amplification—remain cornerstones of prognostic stratification, these biomarkers often fail to capture the systemic complexities that shape real-world patient outcomes. Comorbidities, including diabetes, hypertension, and depression, exert profound yet underappreciated influences on treatment efficacy and survival, acting as both direct biological modifiers and indirect determinants of therapeutic adherence. Similarly, patient-reported outcomes (PROs), which encapsulate dimensions such as fatigue, emotional well-being, and physical functioning, offer nuanced insights into pre-treatment health status that correlate strongly with post-therapeutic resilience. Despite their potential, these multidimensional data streams are seldom integrated into predictive models, leaving clinicians reliant on fragmented snapshots of patient health. The Charité Breast Cancer Dataset addresses this limitation through its prospective collection of clinical, comorbidity, and PRO variables in a large, heterogeneous cohort, enabling rigorous exploration of their synergistic value. This study capitalizes on the dataset’s granularity to develop and validate a machine learning framework that transcends unimodal analysis, offering a paradigm shift in how breast cancer prognosis is conceptualized and operationalized. This work pursues three interconnected objectives to advance precision oncology through multimodal data integration. First, it aims to construct a machine learning architecture capable of harmonizing structured clinical variables (e.g., histopathological grade, receptor status), binary comorbidity indicators (e.g., diabetes, urinary tract infections), and continuous PRO scores (e.g., global health status, fatigue) into a unified predictive model. Second, it seeks to quantify the incremental prognostic value of comorbidity burden and PRO domains beyond traditional biomarkers, identifying novel risk factors such as the impact of gastrointestinal comorbidities or depression on survival trajectories. Third, the study endeavors to validate the clinical utility of the proposed framework in stratifying high-risk patients, informing treatment escalation or de-escalation strategies, and enhancing shared decision-making through interpretable risk profiles. These objectives are anchored in the hypothesis that a multimodal approach will yield superior predictive accuracy compared to conventional models, thereby enabling personalized interventions tailored to the unique biological and psychosocial context of each patient. The significance of this research lies in its potential to redefine breast cancer prognosis through three pivotal contributions. Methodologically, it pioneers the systematic integration of comorbidity subtypes and PRO domains into ML pipelines, moving beyond simplistic binary comorbidity indices to explore the differential impacts of specific conditions (e.g., urinary tract infections as a novel prognostic marker). Clinically, it addresses a critical unmet need for tools that reconcile tumor biology with systemic patient health, particularly in an era where comorbidity prevalence is rising due to aging populations and improved cancer survivorship. Translational implications extend to the development of risk-adapted treatment algorithms that balance oncological urgency with patient-centered care, potentially reducing overtreatment in comorbidity-burdened individuals while optimizing outcomes in high-risk subgroups. Furthermore, the framework’s scalability—demonstrated through tablet-based PRO collection and automated feature engineering—positions it as a blueprint for real-world implementation in diverse healthcare settings. By bridging computational rigor with clinical pragmatism, this study advances the vision of precision oncology as a multidimensional discipline, where data-driven insights catalyze equitable, individualized care pathways.

Supplementary Information. Supplementary Information.