A generalised vision transformer-based self-supervised model for diagnosing and grading prostate cancer using histological images

ABSTRACT Background Gleason grading remains the gold standard for prostate cancer histological classification and prognosis, yet its subjectivity leads to grade variability between pathologists, potentially impacting clinical decision-making. Herein, we trained and validated a generalised AI-driven system for diagnosing prostate cancer using diverse datasets from tissue microarray (TMA) core and whole slide images (WSIs) with Hematoxylin and Eosin staining. Methods We analysed eight prostate cancer datasets, which included 12,711 histological images from 3,648 patients, incorporating TMA core images and WSIs. The Macenko method was used to normalise colours for consistency across diverse images. Subsequently, we trained a multi-resolution (5x, 10x, 20x, and 40x) binary classifier to identify benign and malignant tissue. We then implemented a multi-class classifier for Gleason patterns (GP) sub-categorisation from malignant tissue. Finally, the models were externally validated on 11,132 histology images from 2,176 patients to determine the International Society of Urological Pathology (ISUP) grade. Models were assessed using various classification metrics, and the agreement between the model’s predictions and the ground truth was quantified using the quadratic weighted Cohen’s Kappa ( κ ) score. Results Our multi-resolution binary classifier demonstrated robust performance in distinguishing malignant from benign tissue with κ scores of 0.967 on internal validation. The model achieved κ scores ranging from 0.876 to 0.995 across four unseen testing datasets. The multi-class classifier also distinguished GP3, GP4, and GPs with an overall κ score of 0.841. This model was further tested across four datasets, obtaining κ scores ranging from 0.774 to 0.888. The models’ performance was compared against an independent pathologist’s annotation on an external dataset, achieving a κ score of 0.752 for four classes. Conclusion The self-supervised ViT-based model effectively diagnoses and grades prostate cancer using histological images, distinguishing benign and malignant tissues and classifying malignancies by aggressiveness. External validation highlights its robustness and clinical applicability in digital pathology.