Predicting Central Lymph Node Metastasis in Papillary Thyroid Carcinoma Using a Fusion Model of Vision Transformer and Traditional Radiomics Based on Dynamic Dual-Modality Ultrasound

Abstract Objective This study aimed to develop a novel fusion model based on dynamic dual-modality with B-mode ultrasound and superb microvascular imaging (SMI), combining Vision Transformer (ViT) and radiomics features to predict central lymph node metastasis (CLNM) in thyroid cancer patients. Method In this retrospective diagnostic study, 310 patients with pathologically confirmed papillary thyroid carcinoma from two hospitals were included. We trained ViT models for B-mode and SMI, then extracted ViT and radiomics features from their video images. Initially, Single-modality models were developed, including the B-mode radiomics model (BMUS_RAD) and the B-mode ViT model (BMUS_ViT). Subsequently, Dual-modality models were constructed, encompassing the Dual-modality radiomics model (DMU_RAD), the Dual-modality ViT model (DMU_ViT), and finally, the integrated model DMU_RAD_ViT, to enhance the prediction of CLNM. The performance of each model was compared, and SHAP was utilized for the visual interpretation of the novel fusion model. Results Among all the models, the fusion model DMU_RAD_ViT performed the best (AUC = 0.901, p < 0.05). At the same time, the dual-modality model DMU_RAD(AUC = 0.856) and DMU_ViT(AUC = 0.832) is also higher than the single-modal model BMUS_RAD (AUC = 0.837) and BMUS_ViT (AUC = 0.789), respectively. SHAP analysis revealed that 16 radiomics and ViT features from both modalities contributed to the DMU_RAD_ViT model. Conclusions The Dual-modality fusion model, integrating both radiomics and ViT features, can be utilized to predict CLNM.