Spatial transcriptome reveals histology-correlated immune signature learnt by deep learning attention mechanism on H&E–stained images for ovarian cancer prognosis

Abstract Background: The ability to predict the prognosis of patients with ovarian cancer can greatly improve disease management. However, the knowledge on the mechanism of the prediction is limited. We sought to deconvolute the attention feature learnt by a deep learning convolutional neural networks trained with whole-slide images (WSIs) of hematoxylin-and-eosin (H&E)–stained tumor samples using spatial transcriptomic data. Methods: In this study, 773 WSIs of H&E–stained tumor sections from 335 patients with high-grade serous ovarian cancer who were included in The Cancer Genome Atlas (TCGA) Pan-Cancer study were used to train and validate and to test a ResNet101 CNN model modified with attention mechanism. WSIs from patients in an independent cohort were used to further evaluate the model. Results: The prognostic value of the predicted H&E–based survival scores from the trained model on patient survival was evaluated. The attention signals learnt by the model were then examined their correlation with immune signatures using spatial transcriptome. After validating the model with the testing datasets, pathway enrichment analysis showed that the H&E—based survival score significantly correlated with certain immune signatures and this was validated spatially using spatial transcriptome data of ovarian cancer FFPE sample by correlating the selected signature and attention signal. Conclusions: In conclusion, the attention mechanism might be useful to identify regions for their specific immune activities. This could guide future pathological study for the useful immunological features that are important in modulating the prognosis of ovarian cancer patients.