multi-teacher knowledge distillation for prostate cancer recognition

Prostate disease is one of the major diseases that endanger male life and health. Transrectal ultrasound (TRUS) imaging is an important diagnostic tool in the clinical diagnosis of prostate cancer. Due to the lack of significant differences in visual features between ultrasound images with prostate cancer and non-prostate cancer, the recognition accuracy of a single neural network model is low. Our work uses multi teacher knowledge distillation to pre-train multiple teacher networks, construct multiple teacher models, integrate the soft target outputs of multiple teachers, dynamically assign different weights to the soft target outputs of each teacher network, and distill the student models. This allows the student models to learn the advantages of different teacher models, making the model more accurate in identifying prostate cancer from TRUS images. Our experiment compares the knowledge distillation of non teacher models, single teacher models, and multiple teacher models. The accuracy of the model in predicting prostate TRUS imaging cancer increases significantly with the number of teacher models, verifying the effectiveness of this method.