Cytokeratin Immunohistochemistry-Supervised Deep Learning for Detecting Breast Cancer Lymph Node Metastases and Evaluation of its Clinical Utility

Cytokeratin Immunohistochemistry-Supervised Deep Learning for Detecting Breast Cancer Lymph Node Metastases and Evaluation of its Clinical Utility | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Help Center Sign In Submit a Preprint Cite Share Download PDF Article Cytokeratin Immunohistochemistry-Supervised Deep Learning for Detecting Breast Cancer Lymph Node Metastases and Evaluation of its Clinical Utility Yueh-Hung Chou, Chien-Hui Wu, Min-Hsiang Chang, Hsin-Hsiu Tsai, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5329122/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Lymph node status is an indispensable examination for breast cancer therapy. To detect small, inconspicuous metastatic carcinomas, pathologists usually require immunohistochemical (IHC) staining for cytokeratin (CK). Here, we proposed an IHC-supervised algorithm to create virtual CK masks in lymph node hematoxyline and eosin (H&E) images and evaluated its clinical utility. We enrolled 194 patients with breast cancer surgery-related axillary lymph nodes containing variously sized metastases. The deep learning network, Unet++ with EfficientNet-B7 as the backbone, was trained with the ground truth extracted from consecutive or re-stained CK. At the pixel level, the model had high accuracy (0.98 on average) and decent recall (0.64 on average) and performed best in macrometastasis, followed by micrometastasis and isolated tumor cells (ITC). At the whole-slide image (WSI) level, all 25 slides with macro-metastases and most micro-metastatic (15/16) were classified correctly. For ITC, 17/19 patients were identified; however, certain benign cells were misrecognized in 18/19 negative patients. In clinical settings, artificial intelligence can help pathologists detect micrometastatic carcinoma and significantly decrease reading time. IHC-supervised deep learning is robust and efficient, providing substantial, high-quality ground truth. The virtual CK masks and augmented WSI system enhanced pathologists’ ability to search for tumors in the lymph nodes. Biological sciences/Cancer Biological sciences/Computational biology and bioinformatics Health sciences/Medical research Immunohistochemistry artificial intelligence breast cancer virtual cytokeratin lymph node Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Breast cancer is the most common malignancy among women. According to the World Health Organization, there were 2.3 million women diagnosed with breast cancer and 670,000 deaths worldwide in 2022. To evaluate these patients, axillary sentinel lymph node sampling is critical for determining surgical choice, cancer staging, and treatment strategies [1]. If macrometastasis (> 2 mm) is not observed in the sentinel nodes during surgery, axillary dissection would be waived, and postoperative lymphedema could be prevented. For nodes with small metastatic foci (≦ 2 mm), such as micrometastases or isolated tumor cells (ITC), patients can be treated with radiotherapy or chemotherapy [2]. However, detecting small metastatic lesions in frozen or permanent sections is challenging for pathologists. Immunohistochemical staining (IHC) with a cytokeratin (CK) antibody is commonly used to reveal these areas, helping pathologists discover missed carcinomatous cells in 12%–29% of negative patients [3]. With deep learning and computing power development in recent decades, studies on digital pathology and image analysis with artificial intelligence (AI) have flourished [4, 5]. Detecting metastatic carcinoma in breast surgery-related lymph nodes is one of the major focuses of this field [6, 7]. In the CAMELYON 16 challenge, research teams used the most advanced neural network structures at the time (such as GoogLeNet, ResNet, and VGG-16) to create algorithms to detect metastatic carcinoma. The top five models competed with pathologists in the datasets provided, and some performed better than humans in patients with micrometastasis. However, a dataset based on human annotations must first be built to train and test these models. As only pathologists can identify lesions, experts must perform tedious labeling, requiring substantial amounts of time and money. Humans cannot precisely delineate the tumor area; because of this, the manual ground truth is not perfect, which makes it difficult for AI to learn from. To solidify the ground truth, researchers have used IHC images to assist pathologists in labeling data [8, 9]. Phosphorylated histone H3 IHC can help pathologists accurately annotate mitosis and prevent the mislabeling of mimickers. Additionally, some studies have directly extracted IHC-positive areas using color deconvolution and used them as the ground truth to train algorithms [10, 11]. IHC-based ground truth is more precise and accurate (depending on the correlation between the targets and antibodies) than handcrafted data, which mitigates the training data demand and alleviates the dependence on expertise, allowing valuable expert resources reserved for model evaluation and clinical validation. With modern hardware and programming platforms, building deep-learning neural networks is easier if datasets are available. However, an algorithm acts similarly to a car engine and is only a core part of the application software. To assess the clinical benefits, the digital pathology environment and implementation must be considered [4]. Researchers have recently deployed models in clinical workflows and seen benefits [12, 13]. In this study, we used CK IHC images as the ground truth to create a virtual CK staining model that can predict the epithelial area in breast cancer surgery-related axillary lymph node hematoxylin and eosin (H&E) images. In addition, we implemented AI in the user interface and analyzed how it improved diagnosis accuracy and efficiency. Methods This study was approved by the Research Ethics Review Committee of the Far Eastern Memorial Hospital, New Taipei City, Taiwan (No. 112230-E/DPAI-009). All slides and blocks in the study were used in a clinical diagnostic setting, retrieved from the repository, and did not contain personal information. All methods were performed in accordance with relevant guidelines and regulations. The requirement for informed patient consent was waived by the Research Ethics Review Committee. Case collection In this study, 194 patients with breast cancer surgery-related axillary lymph nodes were enrolled, and blocks were retrieved from Far Eastern Memorial Hospital (New Taipie City, Taiwan) between January 2020 and December 2023. Twenty-eight patients were benign, and 166 contained metastases of various sizes (Table 1). The patients were separated into training sets, including a consecutive CK group (Set A), a re-stained CK group (Set B), and test sets. Test Set B, a subset of Test Set A, was designed for clinical performance evaluation. Table 1. Case enrollment and ground truth diagnosis. Cases Diagnosis Macro-metastasis Micro-metastasis Isolated tumor cells Negative Training Set A (Consecutive) 108 78 12 11 7 Training Set B (Re-stained) 14 6 2 4 2 Test Set A (Re-stained) 72 25 16 12 19 Test Set B (Re-stained) 51 22 8 5 16 Specimen staining and image acquisition The retrieved formalin-fixed paraffin-embedded blocks were cut into 5-μm-thick sections and mounted on hydrophilic slides. For the consecutive CK group, the two nearby slides were stained with H&E and CK antibodies (AE1/AE3/PCK26, Ventana Medical Systems, Oro Valley, AZ, USA) on a Benchmark Ultra (Ventana Medical Systems). The slides were scanned with a Hamamatsu S210 (Hamamatsu, Iwata City, Suizuoka, Japan) at 40× magnification (0.23 μm per pixel resolution) and saved in NDPI format. In the re-stained CK group, one section was cut from each block and stained with H&E. The slides were then scanned, and the coverslips were removed. After bleaching with potassium permanganate, the slides were re-stained with CK for IHC and scanned. Model training and fine-tuning The Unet + + neural network with EfficientNet-b7 as the backbone was pre-trained on ImageNet. The first model was trained using Set A, which consisted of 108 H&E-stained whole-slide images (WSIs) and their corresponding CK images (Figure 1). The ground truth was extracted using color deconvolution after alignment. However, image deviations between consecutive slides still existed. To enhance the first model, another 14 H&E-stained WSIs and their re-stained CK images (Set B) were added for fine-tuning, resulting in the second model. Observing that the predictions of this second model on Set A were already better than the ground truths obtained from consecutive CK images, this second model was used to infer the H&E images of Set A and create pseudo-labels. Finally, the refined Set A and Set B were combined to fine-tune the final model (semi-supervised learning). Clinical performance evaluation of the lymph node model To evaluate how this AI could aid pathologists, we chose 51 lymph node slides (Set B) for two pathologists (Drs. A and B) to read in three different modalities: glass slides, WSI without AI, and WSI with AI. In the first run, the pathologists examined the glass slides using a light microscope. Once the pathologists diagnosed, the slides were scanned, bleached, and re-stained with CK to establish a definite diagnosis. After a washout period (more than 2 weeks) and case shuffling, they examined and diagnosed WSIs of the same cases again. The WSIs were then inferred using our AI within the Smart Pathology System (version 1.6.0; Quanta Computer, Taoyuan, Taiwan), where suspicious epithelial areas were highlighted with green masks; the software automatically measured mask sizes and classified the tumors into four categories: macrometastasis (> 2 mm), micrometastasis (≦ 2 mm), ITC (≦ 0.2 mm) and negative [3]. Finally, after another washout period, both doctors read the augmented WSIs using virtual epithelial masks. Diagnostic accuracy and time were recorded for performance evaluation. Statistical analysis Python (ver. 3.8.13; https://www.python.org/) was used to calculate model performance data, including the intersection of union (IoU), recall, precision, and area under the curve (AUC). Excel in Office 365 (Microsoft Corp., Redmond, WA, USA) was used for statistical analysis. The read time difference between modalities was analyzed using a paired t-test, and statistical significance was set at p < 0.05. Results Model performance at pixel-level The model was evaluated for pixel-level classification using 72 cases in Set A (Figure 2 and Table 2). Macrometastasis showed the highest performance in terms of the predicted CK-stained area (average IoU: 0.66; recall: 0.81; AUC: 0.99), followed by micrometastasis (average IoU: 0.48; recall: 0.68; AUC: 0.99). ITC displayed lower scores (average IoU: 0.15; recall: 0.24; AUC: 0.96), owing to its small size. Table 2. Model performance in pixel-level. IoU (95% CI) Recall (95% CI) Precision (95% CI) Accuracy (95% CI) AUC (95% CI) Macro-metastasis 0.6635 ± 0.051 0.8141 ± 0.047 0.7739 ± 0.035 0.9709 ± 0.007 0.9872 ± 0.003 Micro-metastasis 0.4848 ± 0.121 0.6817 ± 0.141 0.5888 ± 0.131 0.9993 ± 0.001 0.9942 ± 0.004 Isolated tumor cells 0.1475 ± 0.140 0.2445 ± 0.213 0.2546 ± 0.210 0.9998 ± 0.001 0.9558 ± 0.031 Model performance at WSI-level In the WSI-level classification (Figure 3), all 25 patients with macrometastasis were successfully categorized; 15 of the 16 patients with micrometastasis were classified correctly, while one was incorrectly identified as ITC. In the patients with ITC, 17/19 were correctly identified. However, owing to the high sensitivity of our model, scattered benign cell aggregates (macrophages and endothelial cells) were misrecognized as small tumor nests in 18/19 negative patients. Pathologist performance using different modalities Using glass slides and microscopy, both pathologists could identify all patients with macrometastasis and negative cases, whereas only half of the micrometastases were correctly diagnosed, and all five patients with ITC were missed. When reading the WSIs, they diagnosed all macrometastases and negative lymph nodes; more micrometastases (six for Dr. A and five for Dr. B) were diagnosed, but all patients with ITC were still missed. With AI assistance (Figure 4), even more micrometastatic lesions were correctly recognized (seven for Dr. A and all eight patients for Dr. B), and one patient with ITC was noticed by Dr. B. Although a few benign mimickers, such as macrophages and endothelial cells, were mistaken by the AI in the negative cases, both pathologists excluded these false-positive objectives, which prevented overdiagnosis. Overall, pathologists had better accuracy with AI assistance, particularly in detecting small metastatic foci. Both pathologists took less time to diagnose macrometastases with glass slides, followed by micrometastases and ITC (Table 3). The negative cases required the longest time because they had to check every corner of the sections. With WSIs, reading time maintained a similar pattern, but one of the pathologists (Dr. A) needed more time to check for micrometastases (22 s more on average), ITC (12 s), and negative cases (17 s), compared to using a microscope. However, when pathologists used pre-inferenced virtual CK masks (Figure 4) and automatic measurement, the time required to confirm micrometastasis dropped significantly (39 and 60 s less for Drs. A and B, respectively). The reading times for the ITC and negative cases decreased for Dr. B. Table 3: Diagnosis accuracy and average read time (sec) in different modalities. Type Glass slide WSI WSI with AI AI standalone Dr. A Dr. B Dr. A Dr. B Dr. A Dr. B Accuracy Time Accuracy Time Accuracy Time Accuracy Time Accuracy Time Accuracy Time Accuracy Macro-metastasis 100% 8.2 100% 11.5 100% 8.2 100% 14.6 100% 5.3 100% 7.7 100% Micro-metastasis 50% 26.9 50% 73.6 75% 48.6* 62% 72.3 87% 9.3** 100% 12.7** 100% ITC 0 34.6 0 136.2 0 46.4* 0 139.6 0 52.6 20% 62.0** 100%*** Negative 100% 37.9 100% 158.3 100% 54.8* 100% 137.3 100% 52.5 100% 57.3** 6%**** * WSI compared to glass slides, p < 0.05 ** WSI with AI compared to WSI only, p <0.05 *** All 5 cases were classified as ITC by AI, but tumor cells were correctly located in only 3 cases. **** In 16 negative cases, 14 and 1 were classified as ITC and micro-metastasis respectively. Discussion In histopathological examinations, it is essential to identify the epithelial component by grading dysplasia in cervical specimens [14], defining cancer areas in gastrointestinal biopsy [15], and screening for metastatic carcinoma in lymph nodes [3]. Year-long training is required for pathologists to recognize various epithelial components; however, even for experienced experts, it is still impossible to identify all types of epithelium on slides. Because of this, CK IHC is usually required to reveal these cells; it is an accurate and reliable tool for detecting specific proteins in epithelium-derived cells; however, it costs 10–20 US dollars per CK stain. Because of this, performing CK-IHC on each slide is impractical and uneconomical, increasing the financial and labor burden on the laboratory and delaying the turnaround time of reports. We wish to see if a virtual CK-stained AI could meet these demands. In this study, instead of manual annotation, we used consecutive and re-stained CK IHC to build a virtual CK algorithm that can predict metastatic carcinoma in lymph node H&E images. The IHC-supervised deep learning model achieved high accuracy in detecting variable-sized lesions; more importantly, it improved diagnosis accuracy and efficiency. Virtual CK algorithms, or AI models that can segment epithelial areas, have been studied for years. Initially, researchers used image textures such as local binary patterns to segment different areas in H&E [16]. Subsequently, deep learning was introduced and evolved from a simple convoluted neural network to a sophisticated transformer [14, 17, 18]. However, human annotations are inevitably required as ground truth to train and evaluate these models regardless of how advanced neural networks are applied. Recently, studies have begun using IHC images as the ground truth to train models [10, 11]. IHC-supervised machine learning offers several advantages. IHC highlights target cells by their nature (proteins) rather than morphology, which pathologists rely heavily on, making the ground truth extracted from IHC more accurate. Additionally, IHC images can provide extremely precise and exquisite annotations at the pixel level, far beyond what humans are capable of. Lastly, research teams can harvest ground truth from IHC images regardless of their pathology expertise, which saves time and money. As a result, IHC images can offer economical, substantial, and high-quality datasets with a limited number of patients (122 training cases in our study, less than half of 270 cases in CAMELYON16), allowing AI to learn efficiently. IHC-supervised learning is efficient and can be applied to other immunostaining and study topics [19]. For example, if we replaced CK with CD45 to highlight lymphocytes, we could build a neural network to predict tumor-infiltrating lymphocyte areas and reveal the spatial information of intratumoral immune responses. Another possibility is to use desmin and S-100 antibodies to visualize muscle and neural tissues in gastrointestinal specimens, allowing the algorithms to segment the muscle layers and ganglions to facilitate the evaluation of tumor invasion depth and perineural permeation. Using generative AI [15, 19, 20], such as generative adversarial networks, computers can generate vivid images reminiscent of genuine IHC that corresponded perfectly to the original H&E stain without additional alignment. These alternative images are cheaper than real IHCs, reducing costs to the health insurance system. Instead of performing routine IHC, precious tissues (especially small biopsy specimens) can be preserved for molecular or genetic examination. IHC-supervised AI or virtual IHC could be alternative assessment methods; however, they are not technically perfect. In this study, our model achieved a high accuracy (0.98 on average) and decent recall (0.64 on average) at the pixel level. In terms of the WSI level, the AI was oversensitive, and most negative cases were misclassified as ITCs as it mistook some macrophages and endothelial cells as tumor nests. Regardless of the size of these lesions, the AI classifies cases into ITC categories. The model had difficulty detecting discohesive tumor cells in a few patients with ITC (false negatives). For patients containing a small number of cancer cells (less than 100), the model did not find the actual lesion but only classified them as ITC by benign mimickers. We attempted an overfitting test for these difficult cases. Even if engineers simplified the dataset to force the model to be overfitted for difficult ITCs by adding target data to the training set, they still failed to detect discrete tumor cells, though this was impossible for pathologists. With the neural network structures used here, we assumed that more data might not improve performance, indicating that IHC-supervised AI or virtual IHC cannot replace the actual IHC staining. Although AI cannot function as a real CK IHC, this study demonstrated that digital pathology combined with AI enhances pathologists’ diagnostic sensitivity and efficiency. Both pathologists had similar accuracy when reading the glass slides and WSI. They made better classifications for micrometastatic lesions because the measurement was more precise for WSI viewers; however, some may require extra time (less than 1 min) to navigate the WSI and measure the lesion. Both pathologists, aided by AI, observed more micrometastatic cancers in the lymph nodes. Because the software pointed out the suspicious foci in the WSIs, they could check these suspicious areas directly. In addition, the software quantified the tumor area and classified it into four categories, saving time as the pathologists did not have to manually measure it. However, the benefits of ITC are limited. Although the model correctly predicted parts of the tumor nests, pathologists could not confirm these by morphology. In terms of negative cases, the model falsely labeled scattered non-tumor cells, but the pathologists used less time because they did not need to examine whole lymph node sections. In summary, neither humans nor standalone AI performed perfectly; however, humans performed better with AI assistance. Rather than obsessing over the accuracy of AI, it is more realistic to focus on how it can support pathologists in clinical work [13]. This study had a few limitations. Most lymph node tissues were formalin-fixed after intraoperative examination (frozen sections), meaning that their morphology differed from that of frozen sections, which are commonly used for sentinel lymph node diagnosis. Frozen section images are unstandardized and contain various artifacts, such as nuclear retraction and chatter marks, which hinder model inference [7]. Obtaining restrained CK images for IHC-supervised training is difficult. Additionally, the dataset was obtained from a single medical center. Because the quality of H&E staining differs among laboratories, the model will require fine-tuning and customization before being applied to other institutions. Finally, we analyzed the WSIs on a standalone computer, and the average inference time for one WSI file was approximately 5 min (using an NVIDIA A6000 GPU). To minimize the waiting time (from doctor’s order to complete inference), it will be necessary to integrate AI into a digital pathology information system; as a result, candidate cases can be pre-inferred before pathologists read the H&E stain. In conclusion, we developed a CK IHC-supervised algorithm to detect metastatic carcinoma in lymph node H&E-stained images. The training pipeline was efficient and economical owing to its high-quality ground truth. The virtual CK AI was highly accurate for both macro- and micro-metastases but was oversensitive. Despite this, it exhibited clinical benefits in supporting pathologists in achieving a better and quicker diagnosis. In the future, it will be necessary to incorporate diverse AI models into the digital workflow of pathologists. Declarations Acknowledgments The authors thank Dr. Chu Hsiu-Yi for her support. Author contributions Study Design: MHC, HHT and CHW. Data acquisition: MHC, CHW and YHC. Creation of software used in this study: HHT. Data analysis: HHT and MHC. Manuscript writing: YHC, MHC, HHT, and PYT. All authors have read and approved the final manuscript. Data availability statement The datasets used and analyzed in the current study are available from the corresponding author upon reasonable request. Competing interests The authors declare no competing interests. Funding The project was funded by Quanta Medical Technology Foundation (QMTF). References Zhu, H. & Doğan, B. E. American Joint Committee on Cancer's Staging System for Breast Cancer, Eighth Edition: Summary for Clinicians. Eur. J. Breast Health 17, 234–238 (2021). Luo, S. et al. Prognosis and local treatment strategies of breast cancer patients with different numbers of micrometastatic lymph nodes. World J. Surg. Oncol. 21, 202 (2023). Apple, S. K. Sentinel lymph node in breast cancer: review article from a pathologist's point of view. J. Pathol. Transl. Med. 50, 83–95 (2016). Zarella, M. D. et al. Artificial intelligence and digital pathology: clinical promise and deployment considerations. J. Med. Imaging 10, 051802 (2023). Litjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016). Ehteshami Bejnordi, B. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199–2210 (2017). Kim, Y. G. et al. Challenge for diagnostic assessment of deep learning algorithm for metastases classification in sentinel lymph nodes on frozen tissue section digital slides in women with breast cancer. Cancer Res. Treat. 52, 1103–1111 (2020). Turkki, R., Linder, N., Kovanen, P. E., Pellinen, T., & Lundin, J. Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples. J. Pathol. Inform. 7, 38 (2016). Ibrahim, A., Toss, M. S., Makhlouf, S., Miligy, I. M., Minhas, F., & Rakha, E. A. Improving mitotic cell counting accuracy and efficiency using phosphohistone-H3 (PHH3) antibody counterstained with haematoxylin and eosin as part of breast cancer grading. Histopathology 82, 393–406 (2023). Valkonen, M. et al. Cytokeratin-supervised deep learning for automatic recognition of epithelial cells in breast cancers stained for ER, PR, and Ki-67. IEEE Trans. Med. Imaging 39, 534–542 (2020). Brázdil, T., Gallo, M., Nenutil, R., Kubanda, A., Toufar, M., & Holub, P. Automated annotations of epithelial cells and stroma in hematoxylin-eosin-stained whole-slide images using cytokeratin re-staining. J. Pathol. Clin. Res. 8, 129–142 (2022). Steiner, D. F. et al. Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. Am. J. Surg. Pathol. 42, 1636–1646 (2018). Retamero, J. A. et al. Artificial intelligence helps pathologists increase diagnostic accuracy and efficiency in the detection of breast cancer lymph node metastases. Am. J. Surg. Pathol. 48, 846–854 (2024). Sornapudi, S. et al. EpithNet: Deep regression for epithelium segmentation in cervical histology images. J. Pathol. Inform. 11, 10 (2020). Hong, Y. et al. Deep learning-based virtual cytokeratin staining of gastric carcinomas to measure tumor-stroma ratio. Sci. Rep. 11, 19255 (2021). Linder, N. et al. Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagn. Pathol. 7, 22 (2012). Wu, Y. et al. A machine learning model for separating epithelial and stromal regions in oral cavity squamous cell carcinomas using H&E-stained histology images: A multi-center, retrospective study. Oral Oncol. 131, 105942 (2022). Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850–862 (2024). de Haan, K. et al. Deep learning-based transformation of H&E stained tissues into special stains. Nat. Commun. 12, 4884 (2021). Martino, F. et al. A deep learning model to predict Ki-67 positivity in oral squamous cell carcinoma. J. Pathol. Inform. 15, 100354 (2024). Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5329122","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":380768751,"identity":"5219f1b8-3935-4412-96fb-cfc76cc5bf3f","order_by":0,"name":"Yueh-Hung Chou","email":"","orcid":"","institution":"Far Eastern Memorial Hospital","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yueh-Hung","middleName":"","lastName":"Chou","suffix":""},{"id":380768753,"identity":"a64ef6f2-5f02-4696-8aeb-419b5f7229e5","order_by":1,"name":"Chien-Hui Wu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyElEQVRIiWNgGAWjYDACdgaGA0CKhx/ESSggRgszRIucZANIiwGRWkDA2ACkkYEYLfzMvA8PF7YdTtx8fnXihwcGDPL8Ygfwa5FsZjc4PBOoZduNt5slgA4znDk7Ab8Wg8NsDId5t4G0nN0A0pJgcJuAFnuYls0zzm7+QZQWA2aIFmMD/t5txNkiAbblX7qcxA3ebRYJBhKE/cLf3sb8meeMNQ9//9nNN39U2MjzSxPQgmQfWKUEscrB9h0gRfUoGAWjYBSMJAAAQs5C+i1Ie6oAAAAASUVORK5CYII=","orcid":"","institution":"Taiwan Adventist Hospital","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Chien-Hui","middleName":"","lastName":"Wu","suffix":""},{"id":380768756,"identity":"34e2b91c-dfe7-46f0-a658-77f0e7d5c111","order_by":2,"name":"Min-Hsiang Chang","email":"","orcid":"","institution":"Li Jen Pathology Clinic","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Min-Hsiang","middleName":"","lastName":"Chang","suffix":""},{"id":380768758,"identity":"507aff76-85c5-48ea-b12f-9609d70bc539","order_by":3,"name":"Hsin-Hsiu Tsai","email":"","orcid":"","institution":"Quanta Computer (Taiwan)","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Hsin-Hsiu","middleName":"","lastName":"Tsai","suffix":""},{"id":380768760,"identity":"cbf70e3c-b2ea-4ba6-adb9-c6402fdac2ad","order_by":4,"name":"Yi-Ting Peng","email":"","orcid":"","institution":"Quanta Computer (Taiwan)","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yi-Ting","middleName":"","lastName":"Peng","suffix":""}],"badges":[],"createdAt":"2024-10-25 03:08:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5329122/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5329122/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":70965436,"identity":"5b61c84f-5765-4893-a5a5-5e3226dcddc6","added_by":"auto","created_at":"2024-12-09 16:19:48","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":166803,"visible":true,"origin":"","legend":"\u003cp\u003eVirtual cytokeratin (CK) neural network training workflow. The first model was trained with aligned consecutive CK images, then boosted with more accurate re-stained CK. The last step was semi-supervised learning with Set A pseudo-labels.\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-5329122/v1/8143d6bda305077b1d19306d.jpeg"},{"id":70965435,"identity":"16c035d2-1d28-4ce4-b286-8c3e1d4f7185","added_by":"auto","created_at":"2024-12-09 16:19:48","extension":"jpeg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":416737,"visible":true,"origin":"","legend":"\u003cp\u003eMetastatic carcinoma predicted by the virtual CK. The algorithm inferred H\u0026amp;E images and detected carcinomatous cells. The prediction masks were consistent with CK-positive areas in patient with macro- and micro-metastasis. However, some isolated tumor cells (the fourth column) were missing.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-5329122/v1/296945a615b0e6134c226da5.jpeg"},{"id":70965461,"identity":"b300c1c6-66aa-48c8-bea1-ffd5d8703af9","added_by":"auto","created_at":"2024-12-09 16:19:58","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":140943,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix of model performance at whole slide image (WSI)-level. The model was over-sensitive to suspicious tumor cells; as a result, most negative cases were classified as isolated tumor cells (ITC). The red line separates macro-metastasis from other lesions. The model displayed perfect accuracy in macro-metastasis.\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-5329122/v1/5b07c51a95cc08824e9a7cb4.jpeg"},{"id":70965514,"identity":"fe9079aa-7782-45fc-b5be-d02019172246","added_by":"auto","created_at":"2024-12-09 16:20:09","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":437348,"visible":true,"origin":"","legend":"\u003cp\u003eVirtual CK masks for pathologists. The software highlighted the predicted area in light green, which users easily noticed. The patient with ITC in column 2 was missed in the glass slide and WSI reading, but Dr. B recognized the lesion with the AI-aided system. A few macrophages and endothelial cells might deceive the AI, but pathologists could exclude them correctly.\u003c/p\u003e","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-5329122/v1/12c7bfb495838128fd089750.jpeg"},{"id":75981125,"identity":"d3030c4b-6ce3-4ba8-a644-d24f6236988a","added_by":"auto","created_at":"2025-02-11 07:38:26","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1654189,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5329122/v1/5b145218-d2a3-4b33-ba3e-43d27dbccc8e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Cytokeratin Immunohistochemistry-Supervised Deep Learning for Detecting Breast Cancer Lymph Node Metastases and Evaluation of its Clinical Utility","fulltext":[{"header":"Introduction","content":"\u003cp\u003eBreast cancer is the most common malignancy among women. According to the World Health Organization, there were 2.3 million women diagnosed with breast cancer and 670,000 deaths worldwide in 2022. To evaluate these patients, axillary sentinel lymph node sampling is critical for determining surgical choice, cancer staging, and treatment strategies [1]. If macrometastasis (\u0026gt; 2 mm) is not observed in the sentinel nodes during surgery, axillary dissection would be waived, and postoperative lymphedema could be prevented. For nodes with small metastatic foci (≦ 2 mm), such as micrometastases or isolated tumor cells (ITC), patients can be treated with radiotherapy or chemotherapy [2]. However, detecting small metastatic lesions in frozen or permanent sections is challenging for pathologists. Immunohistochemical staining (IHC) with a cytokeratin (CK) antibody is commonly used to reveal these areas, helping pathologists discover missed carcinomatous cells in 12%\u0026ndash;29% of negative patients [3].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWith deep learning and computing power development in recent decades, studies on digital pathology and image analysis with artificial intelligence (AI) have flourished [4, 5]. Detecting metastatic carcinoma in breast surgery-related lymph nodes is one of the major focuses of this field [6, 7]. In the CAMELYON 16 challenge, research teams used the most advanced neural network structures at the time (such as GoogLeNet, ResNet, and VGG-16) to create algorithms to detect metastatic carcinoma. The top five models competed with pathologists in the datasets provided, and some performed better than humans in patients with micrometastasis. However, a dataset based on human annotations must first be built to train and test these models. As only pathologists can identify lesions, experts must perform tedious labeling, requiring substantial amounts of time and money. Humans cannot precisely delineate the tumor area; because of this, the manual ground truth is not perfect, which makes it difficult for AI to learn from.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo solidify the ground truth, researchers have used IHC images to assist pathologists in labeling data [8, 9]. Phosphorylated histone H3 IHC can help pathologists accurately annotate mitosis and prevent the mislabeling of mimickers. Additionally, some studies have directly extracted IHC-positive areas using color deconvolution and used them as the ground truth to train algorithms [10, 11]. IHC-based ground truth is more precise and accurate (depending on the correlation between the targets and antibodies) than handcrafted data, which mitigates the training data demand and alleviates the dependence on expertise, allowing valuable expert resources reserved for model evaluation and clinical validation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWith modern hardware and programming platforms, building deep-learning neural networks is easier if datasets are available. However, an algorithm acts similarly to a car engine and is only a core part of the application software. To assess the clinical benefits, the digital pathology environment and implementation must be considered [4]. Researchers have recently deployed models in clinical workflows and seen benefits [12, 13]. In this study, we used CK IHC images as the ground truth to create a virtual CK staining model that can predict the epithelial area in breast cancer surgery-related axillary lymph node hematoxylin and eosin (H\u0026amp;E) images. In addition, we implemented AI in the user interface and analyzed how it improved diagnosis accuracy and efficiency.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eThis study was approved by the Research Ethics Review Committee of the Far Eastern Memorial Hospital, New Taipei City, Taiwan (No. 112230-E/DPAI-009). All slides and blocks in the study were used in a clinical diagnostic setting, retrieved from the repository, and did not contain personal information. All methods were performed in accordance with relevant guidelines and regulations. The requirement for informed patient consent was waived by the Research Ethics Review Committee.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase collection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn this study, 194 patients with breast cancer surgery-related axillary lymph nodes were enrolled, and blocks were retrieved from Far Eastern Memorial Hospital (New Taipie City, Taiwan) between January 2020 and December 2023. Twenty-eight patients were benign, and 166 contained metastases of various sizes (Table 1). \u0026nbsp; The patients were separated into training sets, including a consecutive CK group (Set A), a re-stained CK group (Set B), and test sets. Test Set B, a subset of Test Set A, was designed for clinical performance evaluation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 1. Case enrollment and ground truth diagnosis.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"548\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\" style=\"width: 123px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 85px;\"\u003e\n \u003cp\u003eCases\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"4\" style=\"width: 340px;\"\u003e\n \u003cp\u003eDiagnosis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003eMacro-metastasis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003eMicro-metastasis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003eIsolated tumor cells\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003eNegative\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003eTraining Set A (Consecutive)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e108\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003eTraining Set B (Re-stained)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003eTest Set A\u003c/p\u003e\n \u003cp\u003e(Re-stained)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e25\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e19\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003eTest Set B\u003c/p\u003e\n \u003cp\u003e(Re-stained)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e16\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eSpecimen staining and image acquisition\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe retrieved formalin-fixed paraffin-embedded blocks were cut into 5-\u0026mu;m-thick sections and mounted on hydrophilic slides. For the consecutive CK group, the two nearby slides were stained with H\u0026amp;E and CK antibodies (AE1/AE3/PCK26, Ventana Medical Systems, Oro Valley, AZ, USA) on a Benchmark Ultra (Ventana Medical Systems). The slides were scanned with a Hamamatsu S210 (Hamamatsu, Iwata City, Suizuoka, Japan) at 40\u0026times; magnification (0.23 \u0026mu;m per pixel resolution) and saved in NDPI format. In the re-stained CK group, one section was cut from each block and stained with H\u0026amp;E. The slides were then scanned, and the coverslips were removed. After bleaching with potassium permanganate, the slides were re-stained with CK for IHC and scanned.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eModel training and fine-tuning\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Unet + + neural network with EfficientNet-b7 as the backbone was pre-trained on ImageNet. The first model was trained using Set A, which consisted of 108 H\u0026amp;E-stained whole-slide images (WSIs) and their corresponding CK images (Figure 1). \u0026nbsp;The ground truth was extracted using color deconvolution after alignment. However, image deviations between consecutive slides still existed. To enhance the first model, another 14 H\u0026amp;E-stained WSIs and their re-stained CK images (Set B) were added for fine-tuning, resulting in the second model. Observing that the predictions of this second model on Set A were already better than the ground truths obtained from consecutive CK images, this second model was used to infer the H\u0026amp;E images of Set A and create pseudo-labels. Finally, the refined Set A and Set B were combined to fine-tune the final model (semi-supervised learning).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical performance evaluation of the lymph node model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo evaluate how this AI could aid pathologists, we chose 51 lymph node slides (Set B) for two pathologists (Drs. A and B) to read in three different modalities: glass slides, WSI without AI, and WSI with AI. In the first run, the pathologists examined the glass slides using a light microscope. Once the pathologists diagnosed, the slides were scanned, bleached, and re-stained with CK to establish a definite diagnosis. After a washout period (more than 2 weeks) and case shuffling, they examined and diagnosed WSIs of the same cases again. The WSIs were then inferred using our AI within the Smart Pathology System (version 1.6.0; Quanta Computer, Taoyuan, Taiwan), where suspicious epithelial areas were highlighted with green masks; the software automatically measured mask sizes and classified the tumors into four categories: macrometastasis (\u0026gt; 2 mm), micrometastasis (≦ 2 mm), ITC (≦ 0.2 mm) and negative [3]. Finally, after another washout period, both doctors read the augmented WSIs using virtual epithelial masks. Diagnostic accuracy and time were recorded for performance evaluation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStatistical analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePython (ver. 3.8.13; https://www.python.org/) was used to calculate model performance data, including the intersection of union (IoU), recall, precision, and area under the curve (AUC). Excel in Office 365 (Microsoft Corp., Redmond, WA, USA) was used for statistical analysis. The read time difference between modalities was analyzed using a paired t-test, and statistical significance was set at \u003cem\u003ep\u003c/em\u003e \u0026lt; 0.05.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eModel performance at pixel-level\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe model was evaluated for pixel-level classification using 72 cases in Set A (Figure 2 and Table 2). Macrometastasis showed the highest performance in terms of the predicted CK-stained area (average IoU: 0.66; recall: 0.81; AUC: 0.99), followed by micrometastasis (average IoU: 0.48; recall: 0.68; AUC: 0.99). ITC displayed lower scores (average IoU: 0.15; recall: 0.24; AUC: 0.96), owing to its small size.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 2. Model performance in pixel-level.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"548\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eIoU (95% CI)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eRecall (95% CI)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003ePrecision (95% CI)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eAccuracy (95% CI)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eAUC (95% CI)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eMacro-metastasis\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.6635 \u0026plusmn; 0.051\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.8141 \u0026plusmn; 0.047\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.7739 \u0026plusmn; 0.035\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.9709 \u0026plusmn; 0.007\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.9872 \u0026plusmn; 0.003\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eMicro-metastasis\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.4848 \u0026plusmn; 0.121\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.6817 \u0026plusmn; 0.141\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.5888 \u0026plusmn; 0.131\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.9993 \u0026plusmn; 0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.9942 \u0026plusmn; 0.004\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003eIsolated tumor cells\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.1475 \u0026plusmn; 0.140\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.2445 \u0026plusmn; 0.213\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.2546 \u0026plusmn; 0.210\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.9998 \u0026plusmn; 0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.6667%;\"\u003e\n \u003cp\u003e0.9558 \u0026plusmn; 0.031\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eModel performance at WSI-level\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn the WSI-level classification (Figure 3), all 25 patients with macrometastasis were successfully categorized; 15 of the 16 patients with micrometastasis were classified correctly, while one was incorrectly identified as ITC. In the patients with ITC, 17/19 were correctly identified. However, owing to the high sensitivity of our model, scattered benign cell aggregates (macrophages and endothelial cells) were misrecognized as small tumor nests in 18/19 negative patients.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePathologist performance using different modalities\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eUsing glass slides and microscopy, both pathologists could identify all patients with macrometastasis and negative cases, whereas only half of the micrometastases were correctly diagnosed, and all five patients with ITC were missed. When reading the WSIs, they diagnosed all macrometastases and negative lymph nodes; more micrometastases (six for Dr. A and five for Dr. B) were diagnosed, but all patients with ITC were still missed. With AI assistance (Figure 4), even more micrometastatic lesions were correctly recognized (seven for Dr. A and all eight patients for Dr. B), and one patient with ITC was noticed by Dr. B. Although a few benign mimickers, such as macrophages and endothelial cells, were mistaken by the AI in the negative cases, both pathologists excluded these false-positive objectives, which prevented overdiagnosis. Overall, pathologists had better accuracy with AI assistance, particularly in detecting small metastatic foci.\u003c/p\u003e\n\u003cp\u003eBoth pathologists took less time to diagnose macrometastases with glass slides, followed by micrometastases and ITC (Table 3). The negative cases required the longest time because they had to check every corner of the sections. With WSIs, reading time maintained a similar pattern, but one of the pathologists (Dr. A) needed more time to check for micrometastases (22 s more on average), ITC (12 s), and negative cases (17 s), compared to using a microscope. However, when pathologists used pre-inferenced virtual CK masks (Figure 4) and automatic measurement, the time required to confirm micrometastasis dropped significantly (39 and 60 s less for Drs. A and B, respectively). The reading times for the ITC and negative cases decreased for Dr. B.\u003c/p\u003e\n\u003cp\u003eTable 3: Diagnosis accuracy and average read time (sec) in different modalities.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"560\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" style=\"width: 50px;\"\u003e\n \u003cp\u003eType\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"4\" valign=\"bottom\" style=\"width: 150px;\"\u003e\n \u003cp\u003eGlass slide\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"4\" valign=\"bottom\" style=\"width: 152px;\"\u003e\n \u003cp\u003eWSI\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"4\" valign=\"bottom\" style=\"width: 156px;\"\u003e\n \u003cp\u003eWSI with AI\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 52px;\"\u003e\n \u003cp\u003eAI standalone\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"2\" valign=\"bottom\" style=\"width: 74px;\"\u003e\n \u003cp\u003eDr. A\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" valign=\"bottom\" style=\"width: 76px;\"\u003e\n \u003cp\u003eDr. B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" valign=\"bottom\" style=\"width: 76px;\"\u003e\n \u003cp\u003eDr. A\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" valign=\"bottom\" style=\"width: 76px;\"\u003e\n \u003cp\u003eDr. B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" valign=\"bottom\" style=\"width: 74px;\"\u003e\n \u003cp\u003eDr. A\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" valign=\"bottom\" style=\"width: 82px;\"\u003e\n \u003cp\u003eDr. B\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 47px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 27px;\"\u003e\n \u003cp\u003eTime\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 47px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 29px;\"\u003e\n \u003cp\u003eTime\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 47px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 29px;\"\u003e\n \u003cp\u003eTime\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 47px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 29px;\"\u003e\n \u003cp\u003eTime\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 47px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 27px;\"\u003e\n \u003cp\u003eTime\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 47px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 35px;\"\u003e\n \u003cp\u003eTime\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 52px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 50px;\"\u003e\n \u003cp\u003eMacro-metastasis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e8.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e11.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e8.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e14.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e5.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 35px;\"\u003e\n \u003cp\u003e7.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 52px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 50px;\"\u003e\n \u003cp\u003eMicro-metastasis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e50%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e26.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e50%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e73.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e75%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e48.6*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e62%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e72.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e87%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e9.3**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 35px;\"\u003e\n \u003cp\u003e12.7**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 52px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 50px;\"\u003e\n \u003cp\u003eITC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e34.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e136.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e46.4*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e139.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e52.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e20%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 35px;\"\u003e\n \u003cp\u003e62.0**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 52px;\"\u003e\n \u003cp\u003e100%***\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 50px;\"\u003e\n \u003cp\u003eNegative\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e37.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e158.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e54.8*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 29px;\"\u003e\n \u003cp\u003e137.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 27px;\"\u003e\n \u003cp\u003e52.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 47px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 35px;\"\u003e\n \u003cp\u003e57.3**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 52px;\"\u003e\n \u003cp\u003e6%****\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e* WSI compared to glass slides, \u003cem\u003ep\u003c/em\u003e \u0026lt; 0.05\u003c/p\u003e\n\u003cp\u003e** WSI with AI compared to WSI only, \u003cem\u003ep\u003c/em\u003e \u0026lt;0.05\u003c/p\u003e\n\u003cp\u003e*** All 5 cases were classified as ITC by AI, but tumor cells were correctly located in only 3 cases.\u003c/p\u003e\n\u003cp\u003e**** In 16 negative cases, 14 and 1 were classified as ITC and micro-metastasis respectively.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn histopathological examinations, it is essential to identify the epithelial component by grading dysplasia in cervical specimens [14], defining cancer areas in gastrointestinal biopsy [15], and screening for metastatic carcinoma in lymph nodes [3]. Year-long training is required for pathologists to recognize various epithelial components; however, even for experienced experts, it is still impossible to identify all types of epithelium on slides. Because of this, CK IHC is usually required to reveal these cells; it is an accurate and reliable tool for detecting specific proteins in epithelium-derived cells; however, it costs 10\u0026ndash;20 US dollars per CK stain. Because of this, performing CK-IHC on each slide is impractical and uneconomical, increasing the financial and labor burden on the laboratory and delaying the turnaround time of reports. We wish to see if a virtual CK-stained AI could meet these demands. In this study, instead of manual annotation, we used consecutive and re-stained CK IHC to build a virtual CK algorithm that can predict metastatic carcinoma in lymph node H\u0026amp;E images. The IHC-supervised deep learning model achieved high accuracy in detecting variable-sized lesions; more importantly, it improved diagnosis accuracy and efficiency.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eVirtual CK algorithms, or AI models that can segment epithelial areas, have been studied for years. Initially, researchers used image textures such as local binary patterns to segment different areas in H\u0026amp;E [16]. Subsequently, deep learning was introduced and evolved from a simple convoluted neural network to a sophisticated transformer [14, 17, 18]. However, human annotations are inevitably required as ground truth to train and evaluate these models regardless of how advanced neural networks are applied. Recently, studies have begun using IHC images as the ground truth to train models [10, 11]. IHC-supervised machine learning offers several advantages. IHC highlights target cells by their nature (proteins) rather than morphology, which pathologists rely heavily on, making the ground truth extracted from IHC more accurate. Additionally, IHC images can provide extremely precise and exquisite annotations at the pixel level, far beyond what humans are capable of. Lastly, research teams can harvest ground truth from IHC images regardless of their pathology expertise, which saves time and money. As a result, IHC images can offer economical, substantial, and high-quality datasets with a limited number of patients (122 training cases in our study, less than half of 270 cases in CAMELYON16), allowing AI to learn efficiently.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIHC-supervised learning is efficient and can be applied to other immunostaining and study topics [19]. For example, if we replaced CK with CD45 to highlight lymphocytes, we could build a neural network to predict tumor-infiltrating lymphocyte areas and reveal the spatial information of intratumoral immune responses. Another possibility is to use desmin and S-100 antibodies to visualize muscle and neural tissues in gastrointestinal specimens, allowing the algorithms to segment the muscle layers and ganglions to facilitate the evaluation of tumor invasion depth and perineural permeation. Using generative AI [15, 19, 20], such as generative adversarial networks, computers can generate vivid images reminiscent of genuine IHC that corresponded perfectly to the original H\u0026amp;E stain without additional alignment. These alternative images are cheaper than real IHCs, reducing costs to the health insurance system. Instead of performing routine IHC, precious tissues (especially small biopsy specimens) can be preserved for molecular or genetic examination.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIHC-supervised AI or virtual IHC could be alternative assessment methods; however, they are not technically perfect. In this study, our model achieved a high accuracy (0.98 on average) and decent recall (0.64 on average) at the pixel level. In terms of the WSI level, the AI was oversensitive, and most negative cases were misclassified as ITCs as it mistook some macrophages and endothelial cells as tumor nests. Regardless of the size of these lesions, the AI classifies cases into ITC categories. The model had difficulty detecting discohesive tumor cells in a few patients with ITC (false negatives). For patients containing a small number of cancer cells (less than 100), the model did not find the actual lesion but only classified them as ITC by benign mimickers. We attempted an overfitting test for these difficult cases. Even if engineers simplified the dataset to force the model to be overfitted for difficult ITCs by adding target data to the training set, they still failed to detect discrete tumor cells, though this was impossible for pathologists. With the neural network structures used here, we assumed that more data might not improve performance, indicating that IHC-supervised AI or virtual IHC cannot replace the actual IHC staining.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAlthough AI cannot function as a real CK IHC, this study demonstrated that digital pathology combined with AI enhances pathologists\u0026rsquo; diagnostic sensitivity and efficiency. Both pathologists had similar accuracy when reading the glass slides and WSI. They made better classifications for micrometastatic lesions because the measurement was more precise for WSI viewers; however, some may require extra time (less than 1 min) to navigate the WSI and measure the lesion. Both pathologists, aided by AI, observed more micrometastatic cancers in the lymph nodes. Because the software pointed out the suspicious foci in the WSIs, they could check these suspicious areas directly. In addition, the software quantified the tumor area and classified it into four categories, saving time as the pathologists did not have to manually measure it. However, the benefits of ITC are limited. Although the model correctly predicted parts of the tumor nests, pathologists could not confirm these by morphology. In terms of negative cases, the model falsely labeled scattered non-tumor cells, but the pathologists used less time because they did not need to examine whole lymph node sections. In summary, neither humans nor standalone AI performed perfectly; however, humans performed better with AI assistance. Rather than obsessing over the accuracy of AI, it is more realistic to focus on how it can support pathologists in clinical work [13].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis study had a few limitations. Most lymph node tissues were formalin-fixed after intraoperative examination (frozen sections), meaning that their morphology differed from that of frozen sections, which are commonly used for sentinel lymph node diagnosis. Frozen section images are unstandardized and contain various artifacts, such as nuclear retraction and chatter marks, which hinder model inference [7]. Obtaining restrained CK images for IHC-supervised training is difficult. Additionally, the dataset was obtained from a single medical center. Because the quality of H\u0026amp;E staining differs among laboratories, the model will require fine-tuning and customization before being applied to other institutions. Finally, we analyzed the WSIs on a standalone computer, and the average inference time for one WSI file was approximately 5 min (using an NVIDIA A6000 GPU). To minimize the waiting time (from doctor\u0026rsquo;s order to complete inference), it will be necessary to integrate AI into a digital pathology information system; as a result, candidate cases can be pre-inferred before pathologists read the H\u0026amp;E stain.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn conclusion, we developed a CK IHC-supervised algorithm to detect metastatic carcinoma in lymph node H\u0026amp;E-stained images. The training pipeline was efficient and economical owing to its high-quality ground truth. The virtual CK AI was highly accurate for both macro- and micro-metastases but was oversensitive. Despite this, it exhibited clinical benefits in supporting pathologists in achieving a better and quicker diagnosis. In the future, it will be necessary to incorporate diverse AI models into the digital workflow of pathologists.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eAcknowledgments\u003c/p\u003e\n\u003cp\u003eThe authors thank Dr. Chu Hsiu-Yi for her support.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAuthor contributions\u003c/p\u003e\n\u003cp\u003eStudy Design: MHC, HHT and CHW. Data acquisition: MHC, CHW and YHC. Creation of software used in this study: HHT. Data analysis: HHT and MHC. Manuscript writing: YHC, MHC, HHT, and PYT. All authors have read and approved the final manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eData availability statement\u003c/p\u003e\n\u003cp\u003eThe datasets used and analyzed in the current study are available from the corresponding author upon reasonable request.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCompeting interests\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing interests.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFunding\u003c/p\u003e\n\u003cp\u003eThe project was funded by Quanta Medical Technology Foundation (QMTF).\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eZhu, H. \u0026amp; Doğan, B. E. American Joint Committee on Cancer\u0026apos;s Staging System for Breast Cancer, Eighth Edition: Summary for Clinicians. Eur. J. Breast Health 17, 234\u0026ndash;238 (2021).\u003c/li\u003e\n\u003cli\u003eLuo, S. et al. Prognosis and local treatment strategies of breast cancer patients with different numbers of micrometastatic lymph nodes. World J. Surg. Oncol. 21, 202 (2023).\u003c/li\u003e\n\u003cli\u003eApple, S. K. Sentinel lymph node in breast cancer: review article from a pathologist\u0026apos;s point of view. J. Pathol. Transl. Med. 50, 83\u0026ndash;95 (2016).\u003c/li\u003e\n\u003cli\u003eZarella, M. D. et al. Artificial intelligence and digital pathology: clinical promise and deployment considerations. J. Med. Imaging 10, 051802 (2023).\u003c/li\u003e\n\u003cli\u003eLitjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016).\u003c/li\u003e\n\u003cli\u003eEhteshami Bejnordi, B. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199\u0026ndash;2210 (2017).\u003c/li\u003e\n\u003cli\u003eKim, Y. G. et al. Challenge for diagnostic assessment of deep learning algorithm for metastases classification in sentinel lymph nodes on frozen tissue section digital slides in women with breast cancer. Cancer Res. Treat. 52, 1103\u0026ndash;1111 (2020).\u003c/li\u003e\n\u003cli\u003eTurkki, R., Linder, N., Kovanen, P. E., Pellinen, T., \u0026amp; Lundin, J. Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples. J. Pathol. Inform. 7, 38 (2016).\u003c/li\u003e\n\u003cli\u003eIbrahim, A., Toss, M. S., Makhlouf, S., Miligy, I. M., Minhas, F., \u0026amp; Rakha, E. A. Improving mitotic cell counting accuracy and efficiency using phosphohistone-H3 (PHH3) antibody counterstained with haematoxylin and eosin as part of breast cancer grading. Histopathology 82, 393\u0026ndash;406 (2023).\u003c/li\u003e\n\u003cli\u003eValkonen, M. et al. Cytokeratin-supervised deep learning for automatic recognition of epithelial cells in breast cancers stained for ER, PR, and Ki-67. IEEE Trans. Med. Imaging 39, 534\u0026ndash;542 (2020).\u003c/li\u003e\n\u003cli\u003eBr\u0026aacute;zdil, T., Gallo, M., Nenutil, R., Kubanda, A., Toufar, M., \u0026amp; Holub, P. Automated annotations of epithelial cells and stroma in hematoxylin-eosin-stained whole-slide images using cytokeratin re-staining. J. Pathol. Clin. Res. 8, 129\u0026ndash;142 (2022).\u003c/li\u003e\n\u003cli\u003eSteiner, D. F. et al. Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. Am. J. Surg. Pathol. 42, 1636\u0026ndash;1646 (2018).\u003c/li\u003e\n\u003cli\u003eRetamero, J. A. et al. Artificial intelligence helps pathologists increase diagnostic accuracy and efficiency in the detection of breast cancer lymph node metastases. Am. J. Surg. Pathol. 48, 846\u0026ndash;854 (2024).\u003c/li\u003e\n\u003cli\u003eSornapudi, S. et al. EpithNet: Deep regression for epithelium segmentation in cervical histology images. J. Pathol. Inform. 11, 10 (2020).\u003c/li\u003e\n\u003cli\u003eHong, Y. et al. Deep learning-based virtual cytokeratin staining of gastric carcinomas to measure tumor-stroma ratio. Sci. Rep. 11, 19255 (2021).\u003c/li\u003e\n\u003cli\u003eLinder, N. et al. Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagn. Pathol. 7, 22 (2012).\u003c/li\u003e\n\u003cli\u003eWu, Y. et al. A machine learning model for separating epithelial and stromal regions in oral cavity squamous cell carcinomas using H\u0026amp;E-stained histology images: A multi-center, retrospective study. Oral Oncol. 131, 105942 (2022).\u003c/li\u003e\n\u003cli\u003eChen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850\u0026ndash;862 (2024).\u003c/li\u003e\n\u003cli\u003ede Haan, K. et al. Deep learning-based transformation of H\u0026amp;E stained tissues into special stains. Nat. Commun. 12, 4884 (2021).\u003c/li\u003e\n\u003cli\u003eMartino, F. et al. A deep learning model to predict Ki-67 positivity in oral squamous cell carcinoma. J. Pathol. Inform. 15, 100354 (2024).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Immunohistochemistry, artificial intelligence, breast cancer, virtual cytokeratin, lymph node","lastPublishedDoi":"10.21203/rs.3.rs-5329122/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5329122/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Lymph node status is an indispensable examination for breast cancer therapy. To detect small, inconspicuous metastatic carcinomas, pathologists usually require immunohistochemical (IHC) staining for cytokeratin (CK). Here, we proposed an IHC-supervised algorithm to create virtual CK masks in lymph node hematoxyline and eosin (H\u0026E) images and evaluated its clinical utility. We enrolled 194 patients with breast cancer surgery-related axillary lymph nodes containing variously sized metastases. The deep learning network, Unet++ with EfficientNet-B7 as the backbone, was trained with the ground truth extracted from consecutive or re-stained CK. At the pixel level, the model had high accuracy (0.98 on average) and decent recall (0.64 on average) and performed best in macrometastasis, followed by micrometastasis and isolated tumor cells (ITC). At the whole-slide image (WSI) level, all 25 slides with macro-metastases and most micro-metastatic (15/16) were classified correctly. For ITC, 17/19 patients were identified; however, certain benign cells were misrecognized in 18/19 negative patients. In clinical settings, artificial intelligence can help pathologists detect micrometastatic carcinoma and significantly decrease reading time. IHC-supervised deep learning is robust and efficient, providing substantial, high-quality ground truth. The virtual CK masks and augmented WSI system enhanced pathologists’ ability to search for tumors in the lymph nodes.","manuscriptTitle":"Cytokeratin Immunohistochemistry-Supervised Deep Learning for Detecting Breast Cancer Lymph Node Metastases and Evaluation of its Clinical Utility","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-09 16:04:02","doi":"10.21203/rs.3.rs-5329122/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"9280109a-af68-451c-a9ee-d6a211da11fc","owner":[],"postedDate":"December 9th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":40550413,"name":"Biological sciences/Cancer"},{"id":40550414,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":40550415,"name":"Health sciences/Medical research"}],"tags":[],"updatedAt":"2025-02-11T07:38:11+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-09 16:04:02","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5329122","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5329122","identity":"rs-5329122","version":["v1"]},"buildId":"zQwnuV7TCBrMSSSToR1PI","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}