Renal Cell Carcinoma Whole-Slide Image Classification and Search Using Deep Learning

Abstract Background: The size of whole slide images (WSIs) in digital pathology can vary from millions to billions of pixels. Accordingly, training state-of-the-art deep learning models with WSIs may not be feasible due to existing memory and computational constraints. In addition, not all WSI pixels may contain useful information about the scanned biopsy sample, e.g., background, debris, and artifact pixels. Furthermore, distilling expressive features from WSIs is a challenging task as there is generally no region-of-interest annotation in real-world archives. Hence, many methods focus on patch processing which can result in improper representation. Methods: Unlike the patching approach, we propose a novel framework for learning and the creation of unique, meaningful, and compact features that are critical for indexing kidney cancer WSIs. In this method, a deep convolutional neural network is trained on low magnification patches. Then, slide-level features are extracted from the feature maps at low magnification using the same deep model by considering tissue location and the corresponding filter responses. This procedure enables us to represent a large image using a small set of features. Results: We used data from the publicly available TCGA dataset to train our model, and it was assessed by both the TCGA and an additional external test cohort of 141 patients from the Ohio State University. We achieved state-of-the-art performance for WSI image search and classification in Renal Cell Carcinoma (RCC) subtypes on both datasets. Conclusions: Our study depicted that deep neural networks can be used to learn morphological patterns required for accurately representing large whole-slide images. These features can be applied to build WSI search engines to help reducing inter- and intra-observer variability.