Impact of harmonization and oversampling methods on radiomics analysis of multi-center imbalanced datasets: Application to PET-based prediction of lung cancer subtypes

Abstract Purpose To evaluate the impact of harmonization and oversampling methods on multi-center imbalanced datasets, with specific application to PET-based radiomics modeling for histologic subtype prediction in non-small cell lung cancer (NSCLC). Methods The study included 245 patients with adenocarcinoma (ADC) and 78 patients with squamous cell carcinoma (SCC) from 4 centers. Utilizing 1502 radiomics features per patient, we trained, validated, and externally tested 4 machine-learning classifiers, to investigate the effect of no harmonization (NoH) or 4 harmonization methods, paired with no oversampling (NoO) or 5 oversampling methods on subtype prediction. Model performance was evaluated using the average area under the ROC curve (AUROC) and G-mean via 5 times 5-fold cross-validations. Statistical comparisons of the combined models against baseline (NoH+NoO) were performed for each fold of cross-validation using the DeLong test. Results The number of cross-combinations with both AUROC and G-mean outperforming baseline in internal validation and external testing was 15, 4, 2, and 7 (out of 29) for random forest (RF), linear discriminant analysis (LDA), logistic regression (LR), and support vector machine (SVM), respectively. ComBat harmonization combined with oversampling (SMOTE) via RF yielded better performance than baseline (AUROC and G-mean of internal validation: 0.725 vs. 0.608 and 0.625 vs. 0.398; external testing: 0.637 vs. 0.567 and 0.363 vs. 0.234), though statistical significances were not observed. Conclusion Applying harmonization and oversampling methods in multi-center imbalanced datasets can improve NSCLC-subtype prediction, but varies widely across classifiers. We have created open-source comparisons of harmonization and oversampling on different classifiers for comprehensive evaluations in different studies.