Predicting the HER2 status in esophageal cancer from tissue microarrays using convolutional neural networks

Background Fast and accurate diagnostics are key for personalized medicine. Particularly in cancer, precise diagnosis is a prerequisite for targeted therapies which can prolong lives. In this work we focus on the automatic identification of gastroesophageal adenocarcinoma (GEA) patients that qualify for a personalized therapy targeting epidermal growth factor receptor 2 (HER2). We present a deep learning method for scoring microscopy images of GEA for the presence of HER2 overexpression. Methods Our method is based on convolutional neural networks (CNNs) trained on a rich dataset of 1,602 patient samples and tested on an independent set of 307 patient samples. We incorporated an attention mechanism in the CNN architecture to identify the tissue regions in these patient cases which the network has detected as important for the prediction outcome. Our solution allows for direct automated detection of HER2 in immunohistochemistry-stained tissue slides without the need for manual assessment and additional costly in situ hybridization (ISH) tests. Results We show accuracy of 0.94, precision of 0.97, and recall of 0.95. Importantly, our approach offers accurate predictions in cases that pathologists cannot resolve, requiring additional ISH testing. We confirmed our findings in an independent dataset collected in a different clinical center. Conclusions We demonstrate that our approach not only automates an important diagnostic process for GEA patients but also paves the way for the discovery of new morphological features that were previously unknown for GEA pathology.