Prediction of PD-L1 tumor positive score in lung squamous cell carcinoma with H&E staining whole slide images and deep learning

Prediction of PD-L1 tumor positive score in lung squamous cell carcinoma with H&E staining whole slide images and deep learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Prediction of PD-L1 tumor positive score in lung squamous cell carcinoma with H&E staining whole slide images and deep learning Qiushi Wang, Xixiang Deng, Qiang Ma, Lianhua Zhao, Yangyang Feng, and 8 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4180916/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Detecting programmed death ligand 1 (PD-L1) expression based on immunohistochemical (IHC) staining is an important guide for the treatment of lung cancer with immune checkpoint inhibitors. However, this method has problems such as high staining costs, tumor heterogeneity, and subjective differences among pathologists. Therefore, the application of deep learning models to segment and quantitatively predict PD-L1 expression in digital sections of Hematoxylin and eosin (H&E) stained lung squamous cell carcinoma is of great significance. Methods We constructed a dataset comprising H&E-stained digital sections of lung squamous cell carcinoma and used a Transformer Unet (TransUnet) deep learning network with an encoder-decoder design to segment PD-L1 negative and positive regions and quantitatively predict the tumor cell positive score (TPS). Results The results showed that the dice similarity coefficient (DSC) and intersection overunion(IoU) of deep learning for PD-L1 expression segmentation of H&E-stained digital slides of lung squamous cell carcinoma were 80% and 72%, respectively, which were better than the other seven cutting-edge segmentation models. The root mean square error (RMSE) of quantitative prediction TPS was 26.8, and the intra-group correlation coefficients with the gold standard was 0.92 (95% CI: 0.90–0.93), which was better than the consistency between the results of five pathologists and the gold standard. Conclusion The deep learning model is capable of segmenting and quantitatively predicting PD-L1 expression in H&E-stained digital sections of lung squamous cell carcinoma, which has significant implications for the application and guidance of immune checkpoint inhibitor treatments. Trial registration: ChiCTR1900022640(2019-04-19). Lung squamous cell carcinoma PD-L1 Deep learning TransUnet H&E staining Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 1 Background Lung cancer is a malignant tumor with significant morbidity and mortality rates. Whereas non-small-cell lung cancer (NSCLC) comprises 80% of all lung cancers, the main types include adenocarcinoma (32–40%), squamous (25–30%), and large cell (8–16%)[ 1 ]. Immune checkpoint inhibitors (ICIs) have shown remarkable efficacy in the clinical treatment of NSCLC. Expression of programmed death ligand 1 (PD-L1) is significant for ICI efficacy. Therefore, immunohistochemical staining (IHC) for PD-L1 expression has been approved as a companion diagnostic marker in the clinical treatment of ICIs. However, there are disadvantages to using IHC to detect PD-L1 expression, including high detection and time costs, inconsistent interpretation standards, and the need for strong professional knowledge[ 2 ]. Furthermore, the interpretation criteria for PD-L1 vary significantly among tumor types. Meanwhile, factors including tumor heterogeneity, the complexity of the immune microenvironment, and the atypical expression of tumor cells presented by IHC staining easily cause subjective errors in the interpretation results of pathologists[ 3 ]. Hematoxylin-eosin (H&E), the most developed staining method in clinical pathology, enables the distinct visualization of cell morphology and tissue structure and is economical and easily operable. With the advancement of technology, digital high-resolution whole slide images (WSI) obtained from H&E-stained slides are used to provide a new direction for artificial intelligence (AI) assisted diagnosis using deep learning methods[ 4 ]. In recent years, researchers have found that the application of deep learning to pathological H&E-stained images can complete tasks that can be recognized by the human eye, such as tumor classification and tumor grading[ 5 , 6 ]. Moreover, it has a good predictive effect on higher-order tasks, such as gene mutation and survival period prediction[ 7 , 8 ] This method can effectively alleviate the problems of traditional PD-L1 detection technology, including its high cost, low efficiency, and subjective interpretation differences. Wu et al. proposed a high-precision AI system to automatically evaluate the tumor positive cell score (TPS) for NSCLC PD-L1 expression (22C3 and SP263), and the calculated results showed a high degree of agreement between the AI and the pathologist[ 9 ]. Mayer et al. proposed a convolutional neural network (CNN) classification model. Fusions of anaplastic lymphoma kinase (ALK), and ROS proto-oncogene 1receptor tyrosine kinase (ROS1), were predicted directly from the H&E-stained WSI of postoperative tissues of patients with NSCLC. The sensitivity and specificity of the classifier for ALK and ROS1 were 100% and 98.6%, respectively[ 10 ]. Shamai et al. developed a WSI classification dataset for H&E staining of breast cancer and predicted the PD-L1 state of H&E-stained images using deep learning technology. The area under the curve(AUC) value of the predicted result was 0.91–0.93 [ 2 ]. Sha et al.[ 11 ] used deep learning to predict the PD-L1 status of H&E-stained WSIs in NSCLC tissues, with an AUC value of 0.8. Considering the tissue structural complexity and heterogeneity of lung adenocarcinoma, lung squamous cell carcinoma was chosen as the object in this study. We constructed a WSI dataset using H&E-stained lung squamous cell carcinoma. To further quantitatively predict TPS, a deep learning model was employed for the first time to segment the PD-L1 expression region in H&E-stained WSIs of formalin-fixed and paraffin-embedded (FFPE) lung squamous cell carcinoma samples. The deep learning framework adopted the recently released deep segmentation network TransUnet, which introduced advanced transformer technology based on the classic Unet model to enhance the model's ability to acquire contextual structural information in pathological images. This solved the problem wherein traditional convolutional networks exclusively focus on local knowledge, leading to segmented regions with limited structural similarity. Based on H&E-stained WSIs, the model can assist pathologists in predicting PD-L1 TPS and achieving end-to-end prediction. 2 Methods 2.1 Datasets We enrolled surgical excision samples of lung squamous cell carcinoma. From January 2018 to December 2021, FFPE tissue samples from lung squamous cell carcinoma with PD-L1 test results from the Department of Pathology, Dapping Hospital, Army Military Medical University were enrolled in the study. Finally, 2496 H&E-stained digital images were obtained after being labeled by senior pathologists, and each image size was 959×461 pixels. The image set was randomly divided as follows: a training set comprising 1497 images was designated as the training data set for the deep learning model; a validation set comprising 499 images was used to assess the performance of PD-L1 state prediction and determine the optimal structure for the deep learning model; and a test set comprising 500 images was employed to evaluate the deep learning model's segmentation and TPS prediction capabilities compared to the TPS as assessed by the pathologist. 2.2 Slice staining, scanning, and dataset labeling Formalin fixed paraffin-embedded (FFPE) tissue samples of squamous cell carcinoma with known PD-L1 positive expression were successively sliced as 4 µm thick by technicians for H&E staining and PD-L1 IHC detection. H&E staining was performed using a fully automatic dyeing workstation Autostainer XL CV5030 (LEICA, Wetzlar, Germany). PD-L1 IHC staining was performed on the Dako Autostainer Link 48 platform (Dako, Copenhagen, Denmark) using a companion diagnostic kit (Antibody clone No. 22C3). Both H&E- and PD-L1-stained slices were generated digitally on a fully automated digital slice scanning system PRECISE 500B (UNIC-TECH, Beijing, China). The same areas on the H&E-stained digital images were synchronously labeled by trained pathologists using Adobe Photoshop CS6 software (Version 13.0), referring to the positive and negative tumor areas of PD-L1 expression on the IHC digital images. The labeled images were further confirmed by two senior pathologists as gold-standard. The region of interest (ROI) labeled by pathologists for IHC and H&E images and the mask generated by the algorithm are illustrated in Fig. 1 a. The area marked by the green line in the ROI map and the gray area of the mask generated by the algorithm are tumor regions with negative PD-L1 expression, whereas the area marked by the red line and the white area of the mask are tumor regions with positive PD-L1 expression. 2.3 Deep learning model development 2.3.1 TransUnet framework Focusing on the two problems of WSI resolution being too high for direct downsampling and patch-level classification lacking detailed representation, we proposed a more comprehensive solution for completing the recognition and segmentation of PD-L1 status in pathological images. The comprehensive detection framework is illustrated in Fig. 1 b. (1) During the data collection process, H&E and PD-L1 IHC staining were performed on successive sections of FFPE tumor tissues. WSI images were generated using an automatic section scanner. According to the PD-L1 IHC digital images, the pathologists synchronically masked the positive and negative PD-L1expression regions in the H&E images according to IHC images to obtain labeled WSI images. The data were further processed to satisfy the model. First, we used the flood fill algorithm to fill the ROI region in the WSI with the same pixel and then generated the mask label required by the training model. Using the sliding window algorithm, the original WSI images were divided into patches with no overlapping regions, and the size of the PD-L1 expression region in each patch was evaluated. Patches with no and few targets were discarded to avoid introducing many irrelevant targets, ensuring that the remaining patches had both background and target to better guide model learning. (2) During model training, the segmented patch was first enhanced with data such as translation, rotation, and flip in random proportions. Subsequently, the depth segmentation network model TransUnet was trained based on the enhanced patch, and the combined loss of dice[ 12 ] and Cross Entropy(CE) was adopted as the loss function. The former was responsible for measuring the overlap differences between the prediction results and label samples, while the latter calculated the pixel-level differences between the prediction results and label values. Combining the two losses can correctly guide model training and improve the segmentation model's convergence speed. The model output was a pixel-by-pixel classification segmentation map, and more detailed WSI segmentation results were obtained. (3) During model inference, the model input was the original patch images segmented after H&E staining, and the output was the segmentation results combined with the original patches. The segmentation results included two types of targets: the PD-L1 positive region and the PD-L1 negative region. The TPS of a patch can be quantitatively analyzed by calculating the pixel ratio of the positive area to the overall tumor area, and the TPS score of the entire WSI image can be obtained by combining the results of each patch. We designed the framework by adopting the TransUnet network model with an encoder-decoder structure to predict and segment the negative/positive expression region of PD-L1, which was built based on Unet. The encoder included the following: The backbone network adopted the ResNet-V2 model. Compared with the traditional ResNet[ 13 ] model, a pre-activation design was added to improve the performance of the residual module. Furthermore, transformer module was introduced after ResNet to enhance the modeling of context information and enrich the expression of long-range dependent information. The decoder included the following: The low-resolution feature map of the upper layer was continuously amplified by four layers of upsampling because the sampling process produced the problem of spatial structure information loss, and the shallow detailed spatial texture representation was transmitted to the upper layer with the same resolution through the shallow three skip connections. The scaling ratios of the three-hop connection were 1/2, 1/4, and 1/8. After the skip connection, the number of channels was doubled. In this case, 3×3 convolution refined information and compressed the number of channels to reduce the complexity of the model and improve the convergence speed. Multiscale features were captured by multiple sampling between the encoder and decoder. Then the complementary characteristics of deep and shallow, strong and weak were also taken into account. After fusion and refining these features, outputs feature maps with rich discriminant information. 2.3.2 Unet structure Unet[ 14 ] is a symmetric segmentation network that fuzes shallow structural features and deep semantic features with skip connections and effectively uses different features for segmentation prediction. The network mainly includes an encoder, decoder, and hop connection. As demonstrated in Fig. 1 c, the network consists of four downsampling and four upsampling modules. The downsampling module extracted sophisticated image features, reduced image resolution, and expanded information channels. The upsampling module recovered feature information and simultaneously fused the features of the previous layer to supplement the details lost in the compression process[ 15 ]. 2.3.3 Transformer structure Transformer[ 16 ] was first introduced in 2017 for natural language processing and have been widely used in computer vision in recent years. It adopts the structure of an encoder and a decoder, and the module structure is indicated in Fig. 1 d. The transformer mainly comprises position coding, layer regularization, a multihead attention mechanism, and a feedforward neural network. This uniquely designed transformer architecture can effectively capture long-distance dependencies and extract global features of images. Convolutional neural networks have significant advantages in extracting underlying features and obtaining local information. Therefore, a reasonable combination of convolutional operation and transformer modules can effectively compensate for their respective defects and fully exploit their advantages [ 17 ],[ 18 ]. 2.4 Segmentation effect comparison test and evaluation parameters We selected several deep segmentation models to participate in the construction of the PD-L1 expression detection framework, including Unet[ 14 ], AttUnet [ 19 ], TransUnet [ 20 ], FCN[ 21 ], DeepLabv3[ 22 ], DeepLabV3+ [ 23 ], and DenseASPP [ 24 ]. The Unet series models employed an encoder-decoder structure and integrated the recovered feature map with additional abundant features from the front layer. This approach facilitated the edge structure's refinement. The DeepLab series models were characterized by the introduction of atrous spatial pyramid pooling (ASPP), which can improve the representation of multiscale information. To assess the efficacy of each segmentation model and select an optimal model for the construction of the PD-L1 prediction framework. We calculated the dice similarity coefficient (DSC), intersection overunion (IoU), accuracy pixel (AP), and Hausdorff distance (HD) to evaluate each model comprehensively. The calculation principle of each parameter index is as follows: $$\begin{array}{c}Dice=\frac{2\left|{pred}_{mask}\cap mask\right|}{\left|{pred}_{mask}\right|+\left|mask\right|}\#\left(1\right)\end{array}$$ $$\begin{array}{c}IoU=\frac{{|pred}_{mask}\cap mask|}{\left|{pred}_{mask}\cup mask\right|}\#\left(2\right)\end{array}$$ In Formulas 1 and 2, \({pred}_{mask}\) represents the segmentation result predicted by the model, and the mask represents the true expression state of PD-L1 in the sample. $$\begin{array}{c}Pixel Accuracy \left(PA\right)=\frac{{\sum }_{k=0}^{n}{P}_{k}}{\sum _{i=0}^{m}{\sum }_{j=0}^{n}{P}_{ij}}\#\left(3\right)\end{array}$$ In Formula 3, \({P}_{k}\) represents the pixels predicted correctly by the model and \({P}_{ij}\) represents all pixels in the output image of the model. $${d}_{H}\left(X,Y\right)=\text{max}\left\{{d}_{XY},{d}_{YX}\right\}=$$ $$\begin{array}{c}\text{max}\left\{{max}_{x\in X}{min}_{y\in Y}d\left(x,y\right),{max}_{y\in Y}{min}_{x\in X}d\left(x,y\right)\right\}\#\left(4\right)\end{array}$$ In Formula 4, \({d}_{XY} and {d}_{YX},\) respectively, represents the distance between x set external and y set inside, and the distance between y set external and x set inside. 2.5 TPS consistency analysis Five pathologists from Daping Hospital blindly evaluated TPS using IHC slides, and AI predicted TPS based on H&E slides in the test set. The root mean square error (RMSE) between the TPS evaluated by the five pathologists and the TPS predicted by AI with the gold standard of labeling labels was calculated. In the consistency analysis, intra-group correlation coefficients (ICC) were calculated between TPS predicted by AI and the gold standard and TPS evaluated by five pathologists (3,1). A total of 453 effective samples remained after preprocessing the digital slides and excluding 39 images with PD-L1 positive and negative proportions below 5% and 14 samples without sufficient tumor cells. 3 Results 3.1 Model performance and comparison test The performance of each model on the test set is presented in Table 1 . DeepLabv3+ (ResNet101) achieves poor segmentation accuracy because it uses single upsampling to integrate low-level details into high-level information, resulting in insufficient use of details. However, the Unet network continually combines bottom feature maps of varying resolutions with top layers to maximize the utilization of local details, thereby improving prediction accuracy. FCN (ResNet101), Attention Unet, and DeepLabv3 improve the effective feature extractor, introduce an attention mechanism to refine features, and expand the global receptive field, respectively, to further improve the segmentation effect of the network while retaining detailed information. Nevertheless, a common disadvantage of these models is that they fail to enhance the significance of global features in pathological image segmentation. To achieve the best segmentation effect, the TransUnet model employed in this study not only implemented the effective fusion of multiscale features but also enhanced the significance of global discriminant features in prediction. The DSC coefficient of overlap between PD-L1 expression regions predicted by the TransUnet model and real labels was 80%, the intersection ratio IoU was 72%, the edge accuracy was 89, and the average pixel accuracy was 88%. Compared with other models, TransUnet has apparent advantages regarding object recognition integrity and structural similarity (Table 1 ). Table 1 Comparative experimental results of the different models. Model DSC/% IoU/% HD95 mPA/% Weights/Mb Parameters/10 6 TransUnet 80.0 72.0 89 88.0 403 105 Unet 77.3 68.4 101 86.3 131 34 Attention Unet 78.7 69.6 101 78.7 133 35 DenseASPP 77.3 68.6 100 86.6 136 35 FCN(ResNet101) 78.3 69.2 99 86.8 198 52 DeepLabv3(MobileNetV3) 78.9 70.1 99 87.2 42 11 DeepLabv3+(ResNet101) 75.0 65.9 104 85.2 224 58 DeepLabv3+(HRNet) 77.5 68.7 100 86.3 274 71 Figure 2 displays the pixel accuracy of the PD-L1 positive and negative regions for each model. The pixel recognition accuracy of TransUnet for negative and positive expression regions was 85% and 75%, respectively, and its accuracy was better than other models in both single and average categories. Accordingly, the expression status of PD-L1 can be identified more accurately. To visually compare the prediction accuracies of various models, we opted to visualize the model's segmentation effect. The model's prediction results are displayed in gray for PD-L1 negative regions and white for PD-L1 positive regions. The ROI represented the initial mark of the doctor, and a mask was generated by the algorithm to identify the ROI. The inaccurate segmentation of a large target area resulted from a detailed information deficiency in the prediction layer of DeepLabv3+ (ResNet101). The Attention Unet and FCN (ResNet101) greatly emphasized the extraction of detailed features, which weakened the ability to locate lesions and compromised the integrity of the segmentation target. DenseASPP and DeepLabv3+ (HRNet) exhibit improved segmentation accuracy compared with other models with less difference between the target and background (Fig. 3 ). TransUnet emphasized not only global information but also detailed information acquisition, resulting in accurate segmentation results and more detailed segmentation of PD-L1 expression regions. Regarding the extent of segmentation detail reduction, TransUnet exhibited a distinct advantage over the other two Unet-based models (Fig. 4 ). 3.2 TPS comparison test The accuracy of TPS quantitatively predicted by AI using H&E-stained slides was compared with the TPS assessed by five senior pathologists. Five pathologists performed TPS evaluation blindly on IHC slides of 453 samples in the test set and recorded the results. Additionally, the TPS predicted quantitatively by AI was approximated by the ratio of the PD-L1 positive area to the total tumor area in the TransUnet inference segmentation results. Five pathologists evaluated TPS (TPS-PATH N, N = 1–5); five pathologists evaluated mean TPS (TPS-mean PATHs); and the TPS evaluated by AI (TPS-AI) was compared to the gold standard to assess the diagnostic accuracy of pathologists and AI in TPS. The results showed that TPS-AI had the best agreement with the gold standard (TPS-AI vs. gold standard: R = 0.92), followed by TPS-PATH3 (TPS-PATH3 vs. gold standard: R = 0.9), and TPS-mean PATHs (TPS-mean PATHs vs. gold standard: R = 0.85) (Fig. 5 a). Additionally, we calculated the RMSE between TPS-PATH N (N = 1–5) and the gold standard, between TPS-mean PATHs and the gold standard, and between TPS-AI and the gold standard to reflect the accuracy of TPS evaluated by TransUnet through H&E slides. As presented in Fig. 5 b, AI had the smallest RMSE (RMSE = 19.67) with the gold standard, which was lower than the best-performing pathologist (PATH3, RMSE = 21.08) and the TPS-mean PATHs (RMSE = 29.44). 3.3 Consistency analysis For the 453 samples in the test set, we calculated the ICC (3,1) between TPS-PATH N (N = 1–5), TPS-mean PATHs, and TPS-AI, respectively, with the gold standard. The results showed that AI had the best consistency with the gold standard, ICC (3,1) value was 0.92 (95% CI: 0.90–0.93) than PATH N or mean PATHs (Fig. 6 ). 4 Discussion Under high magnification, the TPS evaluation of PD-L1 IHC-stained slides in lung squamous cell carcinoma is based on the accurate interpretation of tumor cells with positive cell membrane expression. However, the interpretation is hindered by necrotic and immune cells (e.g., lymphocytes and macrophages) that express PD-L1. Furthermore, determining the proportion of tumor cells exhibiting positive expression under low magnification is a challenging task, and pathologists' diagnostic efficacy could be significantly compromised by frequent magnification changes on the microscope[ 25 ],[ 26 ]. Therefore, accurately identifying the TPS of PD-L1 expression at global and detailed levels using deep learning models is of significant clinical application value. Results of this research demonstrated that when segmenting PD-L1 status in pathological images, the convolutional neural network-based segmentation model emphasizes the overall integrity of the target instead of accurate learning of its details; thus, the structural similarity of the segmentation results has disadvantages. Particularly, the accuracy of segmentation was reduced when differences between the object and the background were minimal. By combining the benefits of Unet and Transformer technology, the TransUnet model is capable of not only accurately identifying regions of PD-L1 expression that are positive or negative (with the highest pixel recognition accuracy) but also presenting segmentation details in the segmentation results to ensure the target's structural similarity is maintained. Considering the effects on image segmentation and recognition and the degree of restoration, some models were inaccurate in the segmentation of large areas because of the lack of attention to details. Nevertheless, certain models have laid an excessive emphasis on detailed feature extraction, resulting in compromised segmentation targets due to weakened lesion localization abilities. Compared with these models, TransUnet segmented and identified PD-L1 expression regions with more accuracy. Compared to the Unet and AttentionUnet models, segmentation and recognition at the edge of the target region are more accurate, and the reduction with mask labels is significant. Nevertheless, the clinical application of IHC staining is challenging due to its complicated process, long duration, lack of standardization of interpretation standards, and most importantly, its heavy reliance on subjective evaluation by experienced pathologists. Accurate prediction of PD-L1 status using deep learning models to analyze tissue and cell morphology in H&E-stained sections will undoubtedly facilitate the clinical application of PD-L1 expression scores. Although the TransUnet model applied to PD-L1 TPS prediction in this sdudy still needs to be trained with large number of cases, and confirmed in large-scale independent cohort studies. The current results showed that the model integrating the advantages of Unet and Transformer technology can potentially predict PD-L1 expression using H&E-stained digital slides. More importantly, compared with the subjective assessment of pathologists, the prediction results of PD-L1 TPS status using the TransUnet model are always objective and repeatable. Furthermore, this model is not only suitable for the prediction of PD-L1 in lung squamous cell carcinoma, but can also be applied to other cancers and different biomarkers based on deep learning, thus providing a new path for the description of histopathological characteristics of biomarkers and clinical treatment target screening. This study has certain limitations, and we hope to solve these problems during continuous training and clinical cohort studies. First, due to the insufficient number of images in the training group, the TransUnet model's ability to accurately predict PD-L1 expression in lung squamous cell carcinoma remains inadequately representative. Second, deep learning and testing of PD-L1 expression in lung adenocarcinoma are lacking. Compared with lung squamous cell carcinoma, lung adenocarcinoma is more complex and heterogeneous, which undoubtedly poses particular challenges for the deep learning and prediction of this model. Nevertheless, the complexity and heterogeneity of lung adenocarcinoma tissue structure are challenging and subject to interpretation differences for pathologists under light microscopy. Therefore, a head-to-head comparison of PD-L1 TPS between this model and the pathologists in the real working environment is of great clinical significance, and it is also helpful for correcting the inconsistent results of the pathologist. 5. Conclusion We proposed TransUnet, a framework for quantitative PD-L1 TPS prediction using H&E-stained digital images. The backbone of the framework is the TransUnet semantic segmentation model. TransUnet enhanced the modeling of global context information in the encoder stage and combined the representation of local features using a CNN. In the decoder stage, the feature map was amplified by multiple upsampling, and the features of the shallow and high layers were fuzed to acquire more discriminative features. The DSC coefficient and IoU of this model for PD-L1 segmentation prediction based on H&E images of lung squamous cell carcinoma were 80% and 72%, respectively, which ensured the integrity of the segmentation target and improved its structural similarity. Furthermore, when comparing the deep learning network's quantitative prediction of TPS to that of five pathologists assessment, it exhibited a greater degree of concurrence with the gold standard (ICC (3,1) = 0.92 (95% CI: 0.90–0.93) and a reduced error (RMSE = 19.67). AI not only has good accuracy in predicting TPS but also can eliminate the evaluation errors caused by subjective differences between different pathologists. In conclusion, the deep learning network proposed in this study can effectively assist pathologists in completing PD-L1 TPS prediction using H&E-stained digital images. Declarations Ethics approval and consent to participate This study was approved by the Ethics Committee of Daping Hospital, Army Medical University. Informed consent was waived by the ethics committee because the retrospective archival materials were anonymized, and the results have no impact on patient. Consent for publication Not applicable Availability of data and materials The datasets generated and analyzed in the current study are available from the corresponding author with reasonable request. Competing Interests statement The authors declare no competing interests. Authors’ contributions Q Wang contributed to the experimental design, image annotations, slice scanning and manuscript writing. X Deng contributed to the TransUnet model development, manuscript writing, and experimental design. Q Ma, L Zhao, Y Feng, Y Wang and Y Zhao performed the PD-L1 TPS based on IHC staining slides. Y Chen and P Zhong contributed to image annotations. P He and M Ma edit the manuscript and participated in the experimental design. P Feng and H Xiao conceived the study, edit the manuscript and participated in the experimental design. Fundings This work was supported by the Science and Technology Innovation Project of Army Medical University (H Xiao), the Graduate Scientific Research and Innovation Foundation of Chongqing (CYB23047, P Feng), the Chongqing Technology Innovation and Application Development Project(cstc2021jscx-gksbX0056, P Feng), Autonomous Region Science and Technology Support Xinjiang Project Plan(2021E02078, M Ma). Acknowledgements Thanks to Shirong Wei and Ping Fu for assisting with PD-L1 IHC and H&E staining. References Zarogoulidis K., Zarogoulidis P., Darwiche K., Boutsikou E., Machairiotis N., Tsakiridis K., Katsikogiannis N., Kougioumtzi I., Karapantzos I., Huang H., Spyratos D. (2013). Treatment of non-small cell lung cancer (NSCLC). Journal of Thoracic Disease, https://doi.org/10.3978/j.issn.2072-1439.2013.07.10. Shamai G., Livne A., Polónia A., Sabo E., Cretu A., Bar-Sela G., Kimmel R. (2022). Deep learning-based image analysis predicts PD-L1 status from H&E-stained histopathology images in breast cancer. Nature Communications, https://doi.org/10.1038/s41467-022-34275-9. Wang S., Zhu Y., Yu L., Chen H., Lin H., Wan X., Fan X., Heng P. (2019). RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification. Medical Image Analysis, https://doi.org/10.1016/j.media.2019.101549. Litjens G., Sánchez C.I., Timofeeva N., Hermsen M., Nagtegaal I., Kovacs I., Kaa C.H., Bult P., Ginneken B., Laak J. (2016). Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Scientific Reports, https://doi.org/10.1038/srep26286. Hu J., Cui C., Yang W., Huang L., Yu R., Liu S., Kong Y. (2021). Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images. Translational Oncology, https://doi.org/10.1016/j.tranon.2020.100921. Graham S., Vu Q.D., Raza S.E.A., Azam A., Tsang Y.W., Kwak J.T., Rajpoot N. (2019). Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical Image Analysis, https://doi.org/10.1016/j.media.2019.101563. Zeng Z., Xie W., Zhang Y., Lu Y. (2019). RIC-Unet: An improved neural network based on Unet for nuclei segmentation in histology images. IEEE Access, https://doi.org/10.1109/ACCESS.2019.2896920. Graham S. Chen H., Gamper J., Dou Q., Heng P., Snead D., Tsang Y.W. Rajpoot N. (2019). MILD-Net: minimal information loss dilated network for gland instance segmentation in colon histology images. Medical Image Analysis, https://doi.org/10.1016/j.media.2018.12.001. Wu J. Liu C., Liu X., Sun W., Li L., Gao N., Zhang Y., Yang X., Zhang J., Wang, H., Liu X., Huang X., Zhang Y., Cheng R., Chi K., Mao L., Zhou L., Lin D., Ling S. (2022). Artificial intelligence-assisted system for precision diagnosis of PD-L1 expression in non-small cell lung cancer. Modern Pathology, https://doi.org/10.1038/s41379-021-00904-9. Mayer C., Ofek E., Fridrich D.E., Molchanov Y., Yacobi R., Gazy T., Hayun I., Zalach J., Paz-Yaacov N., Barshack I. (2022). Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms. Modern Pathology, https://doi.org/10.1038/s41379-022-01141-4. Sha L., Osinski B.L., Ho I.Y., Tan T.L., Willis C., Weiss H., Beaubier N., Mahon B.M., Taxter T.J., Yip S.S.F. Multi-field-of-view deep learning model predicts non small cell lung cancer programmed death-ligand 1 status from whole-slide hematoxylin and eosin images. Journal of Pathology Informatics, https://doi.org/10.4103/jpi.jpi_24_19. Sudre C.H., Li W., Vercauteren T., Ourselin S., Cardoso M.J. (2017). Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support, https://doi.org/10.1007/978-3-319-67558-9_28. He K. Zhang X., Ren S., Sun J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR.2016.90. Ronneberger O., Fischer P., Brox T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, https://doi.org/10.1007/978-3-319-24574-4_28. Zhou Z., Siddiquee M.M.R., Tajbakhsh N., Liang J. (2020). UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation. IEEE Transactions on Medical Imaging, https://doi.org/10.1109/TMI.2019.2959609. Vaswani A., Shazeer N., Parmar N., Uszkoreit J. (2017). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, https://doi.org/10.48550/arXiv.1706.03762. Huang P. Tan X., Zhou X., Liu S., Mercaldo F., Santone A. (2022). FABNet: fusion attention block and transfer learning for laryngeal cancer tumor grading in P63 IHC histopathology images. IEEE Journal of Biomedical and Health Informatics, https://doi.org/10.1109/JBHI.2021.3108999. Huang P., He P., Tian S., Ma M., Feng P., Xiao H., Mercaldo F., Santone A., Qin J. (2023). A ViT-AMC Network With Adaptive Model Fusion and Multiobjective Optimization for Interpretable Laryngeal Tumor Grading From Histopathological Images. IEEE Transactions on Medical Imaging, https://doi.org/10.1109/TMI.2022.3202248. Oktay O., Schlemper J., Folgoc L.L., Lee M.J., Heinrich M., Misawa K., Mori K., McDonagh S.G., Hammerla N.Y., Kainz B., Glocker B., Rueckert D. (2018). Attention U-Net: Learning Where to Look for the Pancreas. ArXiv, https://doi.org/10.48550/arXiv.1804.03999. Chen J., Lu Y., Yu Q., Luo X., Adeli E., Wang Y., Lu L., Yuille A.L., Zhou Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. ArXiv, https://doi.org/10.48550/arXiv.2102.04306 Shelhamer E., Long J., Darrell T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, https://doi.org/10.1109/TPAMI.2016.2572683. Chen L.C., Papandreou G., Schroff F., Adam H. (2017). Rethinking atrous convolution for semantic image segmentation. ArXiv, https://doi.org/10.48550/arXiv.1706.05587. Chen L.C., Zhu Y., Papandreou G., Schroff F., Adam H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. European Conference on Computer Vision, https://doi.org/10.1007/978-3-030-01234-2_49. Yang M., Yu K., Zhang C., Li Z., Yang K. (2018). DenseASPP for semantic segmentation in street scenes. IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR.2018.00388. Coudray N., Ocampo P.S., Sakellaropoulos T., Narula N., Snuderl M., Fenyö D., Moreira A.L., Razavian N., Tsirigos A. (2018). Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nature Medicine, https://doi.org/10.1038/s41591-018-0177-5. Fraz M.M., Khurram S.A., Graham S., Shaban M., Hassan M., Loya A. Rajpoot N.M. (2020). FABnet: feature attention-based network for simultaneous segmentation of microvessels and nerves in routine histology images of oral cancer. Neural Computing and Applications, https://doi.org/10.1007/s00521-019-04516-y. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4180916","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":286189791,"identity":"b16fd333-b097-4fc7-b517-220fdc2a8beb","order_by":0,"name":"Qiushi Wang","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Qiushi","middleName":"","lastName":"Wang","suffix":""},{"id":286189792,"identity":"24ad46a1-3b9e-435a-9f0c-433c59b598d1","order_by":1,"name":"Xixiang Deng","email":"","orcid":"","institution":"Chongqing University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Xixiang","middleName":"","lastName":"Deng","suffix":""},{"id":286189794,"identity":"2f59b598-4900-4435-932f-238c39b1c4f7","order_by":2,"name":"Qiang Ma","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Qiang","middleName":"","lastName":"Ma","suffix":""},{"id":286189795,"identity":"88797c61-b256-4656-9724-665b45eb0f38","order_by":3,"name":"Lianhua Zhao","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Lianhua","middleName":"","lastName":"Zhao","suffix":""},{"id":286189797,"identity":"98e8787d-8467-4d83-bbe1-bc97202dec92","order_by":4,"name":"Yangyang Feng","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yangyang","middleName":"","lastName":"Feng","suffix":""},{"id":286189799,"identity":"4dbf0353-6282-4e57-aa35-054456c251a5","order_by":5,"name":"Yiying Wang","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yiying","middleName":"","lastName":"Wang","suffix":""},{"id":286189800,"identity":"ce78610c-731f-47b8-9fa0-c3727ba4df68","order_by":6,"name":"Yuan Zhao","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yuan","middleName":"","lastName":"Zhao","suffix":""},{"id":286189801,"identity":"5ebefdec-176c-4afc-89e3-6c2f6b0ea0a2","order_by":7,"name":"Yan Chen","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yan","middleName":"","lastName":"Chen","suffix":""},{"id":286189802,"identity":"01316117-eb13-46b8-a8c6-abad4776564c","order_by":8,"name":"Peng Zhong","email":"","orcid":"","institution":"Army Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Peng","middleName":"","lastName":"Zhong","suffix":""},{"id":286189803,"identity":"90e37f21-5af6-4a0a-8131-b7725dcc8271","order_by":9,"name":"Peng He","email":"","orcid":"","institution":"Chongqing University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Peng","middleName":"","lastName":"He","suffix":""},{"id":286189804,"identity":"17325414-fa88-4415-9836-f10e305134af","order_by":10,"name":"Mingrui Ma","email":"","orcid":"","institution":"Affiliated Tumor Hospital of Xinjiang Medical University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Mingrui","middleName":"","lastName":"Ma","suffix":""},{"id":286189805,"identity":"fad12c30-1037-4d8e-8a20-f3f8cb3bdcd4","order_by":11,"name":"Peng Feng","email":"","orcid":"","institution":"Chongqing University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Peng","middleName":"","lastName":"Feng","suffix":""},{"id":286189806,"identity":"b4615bcb-3e22-4b81-b18e-c3b664480529","order_by":12,"name":"Hualiang Xiao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuElEQVRIiWNgGAWjYDCCAzyMB4CkDD8z88EHxGphAGnhkWxnSzYgTYvBeR4zAaJ08N3uPXCAse0Oj/FhBjMGhhqbaIJaJO+cSzjAcOYZj9lhhrQHDMfSchsIaTG4kWNwgKHiMEjLcQPGhsPEajE4zGPczNgmQYIWoC0GzMxsxGmB+uUwj8RhNmaDBGL8Agyxgw8Y2w7L8fef//jgQ40NYS0MEgwMzH9gnASCyqFaRsEoGAWjYBTgBwCed0PIsSBzcwAAAABJRU5ErkJggg==","orcid":"","institution":"Army Medical University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Hualiang","middleName":"","lastName":"Xiao","suffix":""}],"badges":[],"createdAt":"2024-03-28 08:58:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4180916/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4180916/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53967997,"identity":"9243019d-103a-48a8-a6cc-78bec51ea07f","added_by":"auto","created_at":"2024-04-02 20:03:04","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":5737099,"visible":true,"origin":"","legend":"\u003cp\u003eAnnotated dataset generation and tumor segmentation framework construction\u003c/p\u003e\n\u003cp\u003ea, annotated pathological dataset of lung squamous carcinoma (the region of interest (ROI) labeled by pathologists based on IHC images, and synchronous generate ROI on H\u0026amp;E images and masks by the algorithm). Green line and the gray area: PD-L1 negative, red line and the white area: PD-L1 positive.\u003c/p\u003e\n\u003cp\u003eb, the entire framework of TransUnet. c, structure of Unet; d, structure of Transformer.\u003c/p\u003e","description":"","filename":"Figure1.png","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/aab27fb37ffb8d4303ef6523.png"},{"id":53967995,"identity":"3fc474f3-e2c4-4026-a1c5-8c8b1806762f","added_by":"auto","created_at":"2024-04-02 20:03:03","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":272537,"visible":true,"origin":"","legend":"\u003cp\u003ePixel accuracy of positive/negative tumor segmentation category\u003c/p\u003e","description":"","filename":"Figure2.png","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/5cb14b7b1778b13f0911319c.png"},{"id":53967996,"identity":"62c669b7-9461-440e-b516-d8a3c48bd046","added_by":"auto","created_at":"2024-04-02 20:03:03","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":10929393,"visible":true,"origin":"","legend":"\u003cp\u003eThe comparative experiment of model segmentation effect gray area: PD- L1 negative, white area: PD-L1 positive.\u003c/p\u003e","description":"","filename":"Figure3.png","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/62c6969e131ad76ab6398eff.png"},{"id":53967999,"identity":"a4791e52-d784-44ce-98bc-dac5f59bfde2","added_by":"auto","created_at":"2024-04-02 20:03:04","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":15274430,"visible":true,"origin":"","legend":"\u003cp\u003eThe comparative experiment of the segmentation details of three Unet models gray area: PD-L1 negative, white area: PD-L1 positive.\u003c/p\u003e","description":"","filename":"Figure4.png","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/ce7a2d0dd5808964a16e0f5a.png"},{"id":53967998,"identity":"8010dd5c-649b-4bb2-86c9-907b6c52be3a","added_by":"auto","created_at":"2024-04-02 20:03:04","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":192883,"visible":true,"origin":"","legend":"\u003cp\u003eTPS comparison test results\u003c/p\u003e\n\u003cp\u003e(a) The consistency of TPS assessed by five pathologists (TPS-PATH N, N=1–5), the mean TPS of five pathologists (TPS-mean PATHs), and AI-TPS with the gold standard TPS. (b) Root mean square error (RMSE) between TPS-PATH N (N=1–5), TPS-mean PATHs, and TPS-AI with the gold standard TPS.\u003c/p\u003e","description":"","filename":"Figure5.png","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/7b6d8dec6bd4aae50442f4fb.png"},{"id":53968000,"identity":"9c8e57b6-86b3-423f-899d-4f3d56438a90","added_by":"auto","created_at":"2024-04-02 20:03:04","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":140403,"visible":true,"origin":"","legend":"\u003cp\u003eIntra-group correlation coefficient (ICC) between TPS-PATH N (N=1–5), TPS-mean PATHs and TPS-AI, respectively, with the gold standard.\u003c/p\u003e","description":"","filename":"Figure6.png","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/7452c193125c040b4f09de62.png"},{"id":57319518,"identity":"c26bdd14-34a1-4084-96ad-1e24aa7ada99","added_by":"auto","created_at":"2024-05-29 05:49:43","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":44393722,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4180916/v1/874b28ba-4174-458e-8e62-d8f0247adfac.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Prediction of PD-L1 tumor positive score in lung squamous cell carcinoma with H\u0026E staining whole slide images and deep learning","fulltext":[{"header":"1 Background","content":"\u003cp\u003eLung cancer is a malignant tumor with significant morbidity and mortality rates. Whereas non-small-cell lung cancer (NSCLC) comprises 80% of all lung cancers, the main types include adenocarcinoma (32\u0026ndash;40%), squamous (25\u0026ndash;30%), and large cell (8\u0026ndash;16%)[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Immune checkpoint inhibitors (ICIs) have shown remarkable efficacy in the clinical treatment of NSCLC. Expression of programmed death ligand 1 (PD-L1) is significant for ICI efficacy. Therefore, immunohistochemical staining (IHC) for PD-L1 expression has been approved as a companion diagnostic marker in the clinical treatment of ICIs. However, there are disadvantages to using IHC to detect PD-L1 expression, including high detection and time costs, inconsistent interpretation standards, and the need for strong professional knowledge[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Furthermore, the interpretation criteria for PD-L1 vary significantly among tumor types. Meanwhile, factors including tumor heterogeneity, the complexity of the immune microenvironment, and the atypical expression of tumor cells presented by IHC staining easily cause subjective errors in the interpretation results of pathologists[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eHematoxylin-eosin (H\u0026amp;E), the most developed staining method in clinical pathology, enables the distinct visualization of cell morphology and tissue structure and is economical and easily operable. With the advancement of technology, digital high-resolution whole slide images (WSI) obtained from H\u0026amp;E-stained slides are used to provide a new direction for artificial intelligence (AI) assisted diagnosis using deep learning methods[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In recent years, researchers have found that the application of deep learning to pathological H\u0026amp;E-stained images can complete tasks that can be recognized by the human eye, such as tumor classification and tumor grading[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Moreover, it has a good predictive effect on higher-order tasks, such as gene mutation and survival period prediction[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] This method can effectively alleviate the problems of traditional PD-L1 detection technology, including its high cost, low efficiency, and subjective interpretation differences.\u003c/p\u003e \u003cp\u003eWu et al. proposed a high-precision AI system to automatically evaluate the tumor positive cell score (TPS) for NSCLC PD-L1 expression (22C3 and SP263), and the calculated results showed a high degree of agreement between the AI and the pathologist[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. Mayer et al. proposed a convolutional neural network (CNN) classification model. Fusions of anaplastic lymphoma kinase (ALK), and ROS proto-oncogene 1receptor tyrosine kinase (ROS1), were predicted directly from the H\u0026amp;E-stained WSI of postoperative tissues of patients with NSCLC. The sensitivity and specificity of the classifier for ALK and ROS1 were 100% and 98.6%, respectively[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. Shamai et al. developed a WSI classification dataset for H\u0026amp;E staining of breast cancer and predicted the PD-L1 state of H\u0026amp;E-stained images using deep learning technology. The area under the curve(AUC) value of the predicted result was 0.91\u0026ndash;0.93 [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Sha et al.[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] used deep learning to predict the PD-L1 status of H\u0026amp;E-stained WSIs in NSCLC tissues, with an AUC value of 0.8.\u003c/p\u003e \u003cp\u003eConsidering the tissue structural complexity and heterogeneity of lung adenocarcinoma, lung squamous cell carcinoma was chosen as the object in this study. We constructed a WSI dataset using H\u0026amp;E-stained lung squamous cell carcinoma. To further quantitatively predict TPS, a deep learning model was employed for the first time to segment the PD-L1 expression region in H\u0026amp;E-stained WSIs of formalin-fixed and paraffin-embedded (FFPE) lung squamous cell carcinoma samples. The deep learning framework adopted the recently released deep segmentation network TransUnet, which introduced advanced transformer technology based on the classic Unet model to enhance the model's ability to acquire contextual structural information in pathological images. This solved the problem wherein traditional convolutional networks exclusively focus on local knowledge, leading to segmented regions with limited structural similarity. Based on H\u0026amp;E-stained WSIs, the model can assist pathologists in predicting PD-L1 TPS and achieving end-to-end prediction.\u003c/p\u003e"},{"header":"2 Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Datasets\u003c/h2\u003e \u003cp\u003eWe enrolled surgical excision samples of lung squamous cell carcinoma. From January 2018 to December 2021, FFPE tissue samples from lung squamous cell carcinoma with PD-L1 test results from the Department of Pathology, Dapping Hospital, Army Military Medical University were enrolled in the study. Finally, 2496 H\u0026amp;E-stained digital images were obtained after being labeled by senior pathologists, and each image size was 959\u0026times;461 pixels. The image set was randomly divided as follows: a training set comprising 1497 images was designated as the training data set for the deep learning model; a validation set comprising 499 images was used to assess the performance of PD-L1 state prediction and determine the optimal structure for the deep learning model; and a test set comprising 500 images was employed to evaluate the deep learning model's segmentation and TPS prediction capabilities compared to the TPS as assessed by the pathologist.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Slice staining, scanning, and dataset labeling\u003c/h2\u003e \u003cp\u003eFormalin fixed paraffin-embedded (FFPE) tissue samples of squamous cell carcinoma with known PD-L1 positive expression were successively sliced as 4 \u0026micro;m thick by technicians for H\u0026amp;E staining and PD-L1 IHC detection. H\u0026amp;E staining was performed using a fully automatic dyeing workstation Autostainer XL CV5030 (LEICA, Wetzlar, Germany). PD-L1 IHC staining was performed on the Dako Autostainer Link 48 platform (Dako, Copenhagen, Denmark) using a companion diagnostic kit (Antibody clone No. 22C3). Both H\u0026amp;E- and PD-L1-stained slices were generated digitally on a fully automated digital slice scanning system PRECISE 500B (UNIC-TECH, Beijing, China). The same areas on the H\u0026amp;E-stained digital images were synchronously labeled by trained pathologists using Adobe Photoshop CS6 software (Version 13.0), referring to the positive and negative tumor areas of PD-L1 expression on the IHC digital images. The labeled images were further confirmed by two senior pathologists as gold-standard.\u003c/p\u003e \u003cp\u003eThe region of interest (ROI) labeled by pathologists for IHC and H\u0026amp;E images and the mask generated by the algorithm are illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea. The area marked by the green line in the ROI map and the gray area of the mask generated by the algorithm are tumor regions with negative PD-L1 expression, whereas the area marked by the red line and the white area of the mask are tumor regions with positive PD-L1 expression.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Deep learning model development\u003c/h2\u003e \u003cdiv id=\"Sec6\" class=\"Section3\"\u003e \u003ch2\u003e2.3.1 TransUnet framework\u003c/h2\u003e \u003cp\u003eFocusing on the two problems of WSI resolution being too high for direct downsampling and patch-level classification lacking detailed representation, we proposed a more comprehensive solution for completing the recognition and segmentation of PD-L1 status in pathological images. The comprehensive detection framework is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb.\u003c/p\u003e \u003cp\u003e(1) During the data collection process, H\u0026amp;E and PD-L1 IHC staining were performed on successive sections of FFPE tumor tissues. WSI images were generated using an automatic section scanner. According to the PD-L1 IHC digital images, the pathologists synchronically masked the positive and negative PD-L1expression regions in the H\u0026amp;E images according to IHC images to obtain labeled WSI images. The data were further processed to satisfy the model. First, we used the flood fill algorithm to fill the ROI region in the WSI with the same pixel and then generated the mask label required by the training model. Using the sliding window algorithm, the original WSI images were divided into patches with no overlapping regions, and the size of the PD-L1 expression region in each patch was evaluated. Patches with no and few targets were discarded to avoid introducing many irrelevant targets, ensuring that the remaining patches had both background and target to better guide model learning.\u003c/p\u003e \u003cp\u003e(2) During model training, the segmented patch was first enhanced with data such as translation, rotation, and flip in random proportions. Subsequently, the depth segmentation network model TransUnet was trained based on the enhanced patch, and the combined loss of dice[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] and Cross Entropy(CE) was adopted as the loss function. The former was responsible for measuring the overlap differences between the prediction results and label samples, while the latter calculated the pixel-level differences between the prediction results and label values. Combining the two losses can correctly guide model training and improve the segmentation model's convergence speed. The model output was a pixel-by-pixel classification segmentation map, and more detailed WSI segmentation results were obtained.\u003c/p\u003e \u003cp\u003e(3) During model inference, the model input was the original patch images segmented after H\u0026amp;E staining, and the output was the segmentation results combined with the original patches. The segmentation results included two types of targets: the PD-L1 positive region and the PD-L1 negative region. The TPS of a patch can be quantitatively analyzed by calculating the pixel ratio of the positive area to the overall tumor area, and the TPS score of the entire WSI image can be obtained by combining the results of each patch.\u003c/p\u003e \u003cp\u003eWe designed the framework by adopting the TransUnet network model with an encoder-decoder structure to predict and segment the negative/positive expression region of PD-L1, which was built based on Unet. The encoder included the following: The backbone network adopted the ResNet-V2 model. Compared with the traditional ResNet[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] model, a pre-activation design was added to improve the performance of the residual module. Furthermore, transformer module was introduced after ResNet to enhance the modeling of context information and enrich the expression of long-range dependent information. The decoder included the following: The low-resolution feature map of the upper layer was continuously amplified by four layers of upsampling because the sampling process produced the problem of spatial structure information loss, and the shallow detailed spatial texture representation was transmitted to the upper layer with the same resolution through the shallow three skip connections. The scaling ratios of the three-hop connection were 1/2, 1/4, and 1/8. After the skip connection, the number of channels was doubled. In this case, 3\u0026times;3 convolution refined information and compressed the number of channels to reduce the complexity of the model and improve the convergence speed. Multiscale features were captured by multiple sampling between the encoder and decoder. Then the complementary characteristics of deep and shallow, strong and weak were also taken into account. After fusion and refining these features, outputs feature maps with rich discriminant information.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section3\"\u003e \u003ch2\u003e2.3.2 Unet structure\u003c/h2\u003e \u003cp\u003eUnet[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] is a symmetric segmentation network that fuzes shallow structural features and deep semantic features with skip connections and effectively uses different features for segmentation prediction. The network mainly includes an encoder, decoder, and hop connection. As demonstrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ec, the network consists of four downsampling and four upsampling modules. The downsampling module extracted sophisticated image features, reduced image resolution, and expanded information channels. The upsampling module recovered feature information and simultaneously fused the features of the previous layer to supplement the details lost in the compression process[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section3\"\u003e \u003ch2\u003e2.3.3 Transformer structure\u003c/h2\u003e \u003cp\u003eTransformer[\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] was first introduced in 2017 for natural language processing and have been widely used in computer vision in recent years. It adopts the structure of an encoder and a decoder, and the module structure is indicated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ed. The transformer mainly comprises position coding, layer regularization, a multihead attention mechanism, and a feedforward neural network. This uniquely designed transformer architecture can effectively capture long-distance dependencies and extract global features of images. Convolutional neural networks have significant advantages in extracting underlying features and obtaining local information. Therefore, a reasonable combination of convolutional operation and transformer modules can effectively compensate for their respective defects and fully exploit their advantages [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e],[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e2.4 Segmentation effect comparison test and evaluation parameters\u003c/h2\u003e \u003cp\u003eWe selected several deep segmentation models to participate in the construction of the PD-L1 expression detection framework, including Unet[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], AttUnet [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], TransUnet [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e], FCN[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e], DeepLabv3[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e], DeepLabV3+ [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], and DenseASPP [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. The Unet series models employed an encoder-decoder structure and integrated the recovered feature map with additional abundant features from the front layer. This approach facilitated the edge structure's refinement. The DeepLab series models were characterized by the introduction of atrous spatial pyramid pooling (ASPP), which can improve the representation of multiscale information. To assess the efficacy of each segmentation model and select an optimal model for the construction of the PD-L1 prediction framework. We calculated the dice similarity coefficient (DSC), intersection overunion (IoU), accuracy pixel (AP), and Hausdorff distance (HD) to evaluate each model comprehensively. The calculation principle of each parameter index is as follows:\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}Dice=\\frac{2\\left|{pred}_{mask}\\cap mask\\right|}{\\left|{pred}_{mask}\\right|+\\left|mask\\right|}\\#\\left(1\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equb\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}IoU=\\frac{{|pred}_{mask}\\cap mask|}{\\left|{pred}_{mask}\\cup mask\\right|}\\#\\left(2\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eIn Formulas 1 and 2, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({pred}_{mask}\\)\u003c/span\u003e\u003c/span\u003e represents the segmentation result predicted by the model, and the mask represents the true expression state of PD-L1 in the sample.\u003cdiv id=\"Equc\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}Pixel Accuracy \\left(PA\\right)=\\frac{{\\sum }_{k=0}^{n}{P}_{k}}{\\sum _{i=0}^{m}{\\sum }_{j=0}^{n}{P}_{ij}}\\#\\left(3\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eIn Formula 3, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P}_{k}\\)\u003c/span\u003e\u003c/span\u003erepresents the pixels predicted correctly by the model and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P}_{ij}\\)\u003c/span\u003e\u003c/span\u003erepresents all pixels in the output image of the model.\u003cdiv id=\"Equd\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e\n$${d}_{H}\\left(X,Y\\right)=\\text{max}\\left\\{{d}_{XY},{d}_{YX}\\right\\}=$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Eque\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Eque\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}\\text{max}\\left\\{{max}_{x\\in X}{min}_{y\\in Y}d\\left(x,y\\right),{max}_{y\\in Y}{min}_{x\\in X}d\\left(x,y\\right)\\right\\}\\#\\left(4\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eIn Formula 4, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({d}_{XY} and {d}_{YX},\\)\u003c/span\u003e\u003c/span\u003e respectively, represents the distance between \u003cem\u003ex\u003c/em\u003e set external and \u003cem\u003ey\u003c/em\u003e set inside, and the distance between \u003cem\u003ey\u003c/em\u003e set external and \u003cem\u003ex\u003c/em\u003e set inside.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e2.5 TPS consistency analysis\u003c/h2\u003e \u003cp\u003eFive pathologists from Daping Hospital blindly evaluated TPS using IHC slides, and AI predicted TPS based on H\u0026amp;E slides in the test set. The root mean square error (RMSE) between the TPS evaluated by the five pathologists and the TPS predicted by AI with the gold standard of labeling labels was calculated. In the consistency analysis, intra-group correlation coefficients (ICC) were calculated between TPS predicted by AI and the gold standard and TPS evaluated by five pathologists (3,1). A total of 453 effective samples remained after preprocessing the digital slides and excluding 39 images with PD-L1 positive and negative proportions below 5% and 14 samples without sufficient tumor cells.\u003c/p\u003e \u003c/div\u003e"},{"header":"3 Results","content":"\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Model performance and comparison test\u003c/h2\u003e \u003cp\u003eThe performance of each model on the test set is presented in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. DeepLabv3+ (ResNet101) achieves poor segmentation accuracy because it uses single upsampling to integrate low-level details into high-level information, resulting in insufficient use of details. However, the Unet network continually combines bottom feature maps of varying resolutions with top layers to maximize the utilization of local details, thereby improving prediction accuracy. FCN (ResNet101), Attention Unet, and DeepLabv3 improve the effective feature extractor, introduce an attention mechanism to refine features, and expand the global receptive field, respectively, to further improve the segmentation effect of the network while retaining detailed information. Nevertheless, a common disadvantage of these models is that they fail to enhance the significance of global features in pathological image segmentation. To achieve the best segmentation effect, the TransUnet model employed in this study not only implemented the effective fusion of multiscale features but also enhanced the significance of global discriminant features in prediction. The DSC coefficient of overlap between PD-L1 expression regions predicted by the TransUnet model and real labels was 80%, the intersection ratio IoU was 72%, the edge accuracy was 89, and the average pixel accuracy was 88%. Compared with other models, TransUnet has apparent advantages regarding object recognition integrity and structural similarity (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparative experimental results of the different models.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDSC/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eIoU/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eHD95\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003emPA/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eWeights/Mb\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eParameters/10\u003csup\u003e6\u003c/sup\u003e\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTransUnet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e80.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e72.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e88.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e403\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e105\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eUnet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e77.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e68.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e101\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e86.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e131\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e34\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAttention Unet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e69.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e101\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e78.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e133\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e35\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDenseASPP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e77.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e68.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e86.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e136\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e35\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFCN(ResNet101)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e69.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e86.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e198\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e52\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDeepLabv3(MobileNetV3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e70.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e87.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e11\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDeepLabv3+(ResNet101)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e75.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e65.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e104\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e85.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e224\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e58\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDeepLabv3+(HRNet)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e77.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e68.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e86.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e274\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e71\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e displays the pixel accuracy of the PD-L1 positive and negative regions for each model. The pixel recognition accuracy of TransUnet for negative and positive expression regions was 85% and 75%, respectively, and its accuracy was better than other models in both single and average categories. Accordingly, the expression status of PD-L1 can be identified more accurately.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo visually compare the prediction accuracies of various models, we opted to visualize the model's segmentation effect. The model's prediction results are displayed in gray for PD-L1 negative regions and white for PD-L1 positive regions. The ROI represented the initial mark of the doctor, and a mask was generated by the algorithm to identify the ROI. The inaccurate segmentation of a large target area resulted from a detailed information deficiency in the prediction layer of DeepLabv3+ (ResNet101). The Attention Unet and FCN (ResNet101) greatly emphasized the extraction of detailed features, which weakened the ability to locate lesions and compromised the integrity of the segmentation target. DenseASPP and DeepLabv3+ (HRNet) exhibit improved segmentation accuracy compared with other models with less difference between the target and background (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). TransUnet emphasized not only global information but also detailed information acquisition, resulting in accurate segmentation results and more detailed segmentation of PD-L1 expression regions. Regarding the extent of segmentation detail reduction, TransUnet exhibited a distinct advantage over the other two Unet-based models (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e3.2 TPS comparison test\u003c/h2\u003e \u003cp\u003eThe accuracy of TPS quantitatively predicted by AI using H\u0026amp;E-stained slides was compared with the TPS assessed by five senior pathologists. Five pathologists performed TPS evaluation blindly on IHC slides of 453 samples in the test set and recorded the results. Additionally, the TPS predicted quantitatively by AI was approximated by the ratio of the PD-L1 positive area to the total tumor area in the TransUnet inference segmentation results.\u003c/p\u003e \u003cp\u003eFive pathologists evaluated TPS (TPS-PATH N, N\u0026thinsp;=\u0026thinsp;1\u0026ndash;5); five pathologists evaluated mean TPS (TPS-mean PATHs); and the TPS evaluated by AI (TPS-AI) was compared to the gold standard to assess the diagnostic accuracy of pathologists and AI in TPS. The results showed that TPS-AI had the best agreement with the gold standard (TPS-AI vs. gold standard: R\u0026thinsp;=\u0026thinsp;0.92), followed by TPS-PATH3 (TPS-PATH3 vs. gold standard: R\u0026thinsp;=\u0026thinsp;0.9), and TPS-mean PATHs (TPS-mean PATHs vs. gold standard: R\u0026thinsp;=\u0026thinsp;0.85) (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAdditionally, we calculated the RMSE between TPS-PATH N (N\u0026thinsp;=\u0026thinsp;1\u0026ndash;5) and the gold standard, between TPS-mean PATHs and the gold standard, and between TPS-AI and the gold standard to reflect the accuracy of TPS evaluated by TransUnet through H\u0026amp;E slides. As presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb, AI had the smallest RMSE (RMSE\u0026thinsp;=\u0026thinsp;19.67) with the gold standard, which was lower than the best-performing pathologist (PATH3, RMSE\u0026thinsp;=\u0026thinsp;21.08) and the TPS-mean PATHs (RMSE\u0026thinsp;=\u0026thinsp;29.44).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Consistency analysis\u003c/h2\u003e \u003cp\u003eFor the 453 samples in the test set, we calculated the ICC (3,1) between TPS-PATH N (N\u0026thinsp;=\u0026thinsp;1\u0026ndash;5), TPS-mean PATHs, and TPS-AI, respectively, with the gold standard. The results showed that AI had the best consistency with the gold standard, ICC (3,1) value was 0.92 (95% CI: 0.90\u0026ndash;0.93) than PATH N or mean PATHs (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"4 Discussion","content":"\u003cp\u003eUnder high magnification, the TPS evaluation of PD-L1 IHC-stained slides in lung squamous cell carcinoma is based on the accurate interpretation of tumor cells with positive cell membrane expression. However, the interpretation is hindered by necrotic and immune cells (e.g., lymphocytes and macrophages) that express PD-L1. Furthermore, determining the proportion of tumor cells exhibiting positive expression under low magnification is a challenging task, and pathologists' diagnostic efficacy could be significantly compromised by frequent magnification changes on the microscope[\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e],[\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]. Therefore, accurately identifying the TPS of PD-L1 expression at global and detailed levels using deep learning models is of significant clinical application value. Results of this research demonstrated that when segmenting PD-L1 status in pathological images, the convolutional neural network-based segmentation model emphasizes the overall integrity of the target instead of accurate learning of its details; thus, the structural similarity of the segmentation results has disadvantages. Particularly, the accuracy of segmentation was reduced when differences between the object and the background were minimal. By combining the benefits of Unet and Transformer technology, the TransUnet model is capable of not only accurately identifying regions of PD-L1 expression that are positive or negative (with the highest pixel recognition accuracy) but also presenting segmentation details in the segmentation results to ensure the target's structural similarity is maintained.\u003c/p\u003e \u003cp\u003eConsidering the effects on image segmentation and recognition and the degree of restoration, some models were inaccurate in the segmentation of large areas because of the lack of attention to details. Nevertheless, certain models have laid an excessive emphasis on detailed feature extraction, resulting in compromised segmentation targets due to weakened lesion localization abilities. Compared with these models, TransUnet segmented and identified PD-L1 expression regions with more accuracy. Compared to the Unet and AttentionUnet models, segmentation and recognition at the edge of the target region are more accurate, and the reduction with mask labels is significant.\u003c/p\u003e \u003cp\u003eNevertheless, the clinical application of IHC staining is challenging due to its complicated process, long duration, lack of standardization of interpretation standards, and most importantly, its heavy reliance on subjective evaluation by experienced pathologists. Accurate prediction of PD-L1 status using deep learning models to analyze tissue and cell morphology in H\u0026amp;E-stained sections will undoubtedly facilitate the clinical application of PD-L1 expression scores. Although the TransUnet model applied to PD-L1 TPS prediction in this sdudy still needs to be trained with large number of cases, and confirmed in large-scale independent cohort studies. The current results showed that the model integrating the advantages of Unet and Transformer technology can potentially predict PD-L1 expression using H\u0026amp;E-stained digital slides. More importantly, compared with the subjective assessment of pathologists, the prediction results of PD-L1 TPS status using the TransUnet model are always objective and repeatable. Furthermore, this model is not only suitable for the prediction of PD-L1 in lung squamous cell carcinoma, but can also be applied to other cancers and different biomarkers based on deep learning, thus providing a new path for the description of histopathological characteristics of biomarkers and clinical treatment target screening.\u003c/p\u003e \u003cp\u003eThis study has certain limitations, and we hope to solve these problems during continuous training and clinical cohort studies. First, due to the insufficient number of images in the training group, the TransUnet model's ability to accurately predict PD-L1 expression in lung squamous cell carcinoma remains inadequately representative. Second, deep learning and testing of PD-L1 expression in lung adenocarcinoma are lacking. Compared with lung squamous cell carcinoma, lung adenocarcinoma is more complex and heterogeneous, which undoubtedly poses particular challenges for the deep learning and prediction of this model. Nevertheless, the complexity and heterogeneity of lung adenocarcinoma tissue structure are challenging and subject to interpretation differences for pathologists under light microscopy. Therefore, a head-to-head comparison of PD-L1 TPS between this model and the pathologists in the real working environment is of great clinical significance, and it is also helpful for correcting the inconsistent results of the pathologist.\u003c/p\u003e"},{"header":"5. Conclusion","content":"\u003cp\u003eWe proposed TransUnet, a framework for quantitative PD-L1 TPS prediction using H\u0026amp;E-stained digital images. The backbone of the framework is the TransUnet semantic segmentation model. TransUnet enhanced the modeling of global context information in the encoder stage and combined the representation of local features using a CNN. In the decoder stage, the feature map was amplified by multiple upsampling, and the features of the shallow and high layers were fuzed to acquire more discriminative features. The DSC coefficient and IoU of this model for PD-L1 segmentation prediction based on H\u0026amp;E images of lung squamous cell carcinoma were 80% and 72%, respectively, which ensured the integrity of the segmentation target and improved its structural similarity. Furthermore, when comparing the deep learning network's quantitative prediction of TPS to that of five pathologists assessment, it exhibited a greater degree of concurrence with the gold standard (ICC (3,1)\u0026thinsp;=\u0026thinsp;0.92 (95% CI: 0.90\u0026ndash;0.93) and a reduced error (RMSE\u0026thinsp;=\u0026thinsp;19.67). AI not only has good accuracy in predicting TPS but also can eliminate the evaluation errors caused by subjective differences between different pathologists. In conclusion, the deep learning network proposed in this study can effectively assist pathologists in completing PD-L1 TPS prediction using H\u0026amp;E-stained digital images.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was approved by the Ethics Committee of Daping Hospital, Army Medical University. Informed consent was waived by the ethics committee because the retrospective archival materials were anonymized, and the results have no impact on patient.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets generated and analyzed in the current study are available from the corresponding author with reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors’ contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eQ Wang contributed to the experimental design, image annotations, slice scanning and manuscript writing. X Deng contributed to the TransUnet model development, manuscript writing, and experimental design. Q Ma, L Zhao, Y Feng, Y Wang and Y Zhao performed the PD-L1 TPS based on IHC staining slides. Y Chen and P Zhong contributed to image annotations. P He and M Ma edit the manuscript and participated in the experimental design. P Feng and H Xiao conceived the study, edit the manuscript and participated in the experimental design.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFundings\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work was supported by the Science and Technology Innovation Project of Army Medical University (H Xiao), the Graduate Scientific Research and Innovation Foundation of Chongqing (CYB23047, P Feng), the Chongqing Technology Innovation and Application Development Project(cstc2021jscx-gksbX0056, P Feng), Autonomous Region Science and Technology Support Xinjiang Project Plan(2021E02078, M Ma).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThanks to Shirong Wei and Ping Fu for assisting with PD-L1 IHC and H\u0026amp;E staining.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eZarogoulidis K., Zarogoulidis P., Darwiche K., Boutsikou E., Machairiotis N., Tsakiridis K., Katsikogiannis N., Kougioumtzi I., Karapantzos I., Huang H., Spyratos D. (2013). Treatment of non-small cell lung cancer (NSCLC). Journal of Thoracic Disease, https://doi.org/10.3978/j.issn.2072-1439.2013.07.10.\u003c/li\u003e\n\u003cli\u003eShamai G., Livne A., Pol\u0026oacute;nia A., Sabo E., Cretu A., Bar-Sela G., Kimmel R. (2022). Deep learning-based image analysis predicts PD-L1 status from H\u0026amp;E-stained histopathology images in breast cancer. Nature Communications, https://doi.org/10.1038/s41467-022-34275-9.\u003c/li\u003e\n\u003cli\u003eWang S., Zhu Y., Yu L., Chen H., Lin H., Wan X., Fan X., Heng P. (2019). RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification. Medical Image Analysis, https://doi.org/10.1016/j.media.2019.101549.\u003c/li\u003e\n\u003cli\u003eLitjens G., S\u0026aacute;nchez C.I., Timofeeva N., Hermsen M., Nagtegaal I., Kovacs I., Kaa C.H., Bult P., Ginneken B., Laak J. (2016). Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Scientific Reports, https://doi.org/10.1038/srep26286.\u003c/li\u003e\n\u003cli\u003eHu J., Cui C., Yang W., Huang L., Yu R., Liu S., Kong Y. (2021). Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images. Translational Oncology, https://doi.org/10.1016/j.tranon.2020.100921.\u003c/li\u003e\n\u003cli\u003eGraham S., Vu Q.D., Raza S.E.A., Azam A., Tsang Y.W., Kwak J.T., Rajpoot N. (2019). Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical Image Analysis, https://doi.org/10.1016/j.media.2019.101563.\u003c/li\u003e\n\u003cli\u003eZeng Z., Xie W., Zhang Y., Lu Y. (2019). RIC-Unet: An improved neural network based on Unet for nuclei segmentation in histology images. IEEE Access, https://doi.org/10.1109/ACCESS.2019.2896920.\u003c/li\u003e\n\u003cli\u003eGraham S. Chen H., Gamper J., Dou Q., Heng P., Snead D., Tsang Y.W. Rajpoot N. (2019). MILD-Net: minimal information loss dilated network for gland instance segmentation in colon histology images. Medical Image Analysis, https://doi.org/10.1016/j.media.2018.12.001.\u003c/li\u003e\n\u003cli\u003eWu J. Liu C., Liu X., Sun W., Li L., Gao N., Zhang Y., Yang X., Zhang J., Wang, H., Liu X., Huang X., Zhang Y., Cheng R., Chi K., Mao L., Zhou L., Lin D., Ling S. (2022). Artificial intelligence-assisted system for precision diagnosis of PD-L1 expression in non-small cell lung cancer. Modern Pathology, https://doi.org/10.1038/s41379-021-00904-9.\u003c/li\u003e\n\u003cli\u003eMayer C., Ofek E., Fridrich D.E., Molchanov Y., Yacobi R., Gazy T., Hayun I., Zalach J., Paz-Yaacov N., Barshack I. (2022). Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms. Modern Pathology, https://doi.org/10.1038/s41379-022-01141-4.\u003c/li\u003e\n\u003cli\u003eSha L., Osinski B.L., Ho I.Y., Tan T.L., Willis C., Weiss H., Beaubier N., Mahon B.M., Taxter T.J., Yip S.S.F. Multi-field-of-view deep learning model predicts non small cell lung cancer programmed death-ligand 1 status from whole-slide hematoxylin and eosin images. Journal of Pathology Informatics, https://doi.org/10.4103/jpi.jpi_24_19.\u003c/li\u003e\n\u003cli\u003eSudre C.H., Li W., Vercauteren T., Ourselin S., Cardoso M.J. (2017). Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support, https://doi.org/10.1007/978-3-319-67558-9_28.\u003c/li\u003e\n\u003cli\u003eHe K. Zhang X., Ren S., Sun J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR.2016.90.\u003c/li\u003e\n\u003cli\u003eRonneberger O., Fischer P., Brox T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, https://doi.org/10.1007/978-3-319-24574-4_28.\u003c/li\u003e\n\u003cli\u003eZhou Z., Siddiquee M.M.R., Tajbakhsh N., Liang J. (2020). UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation. IEEE Transactions on Medical Imaging, https://doi.org/10.1109/TMI.2019.2959609.\u003c/li\u003e\n\u003cli\u003eVaswani A., Shazeer N., Parmar N., Uszkoreit J. (2017). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, https://doi.org/10.48550/arXiv.1706.03762.\u003c/li\u003e\n\u003cli\u003eHuang P. Tan X., Zhou X., Liu S., Mercaldo F., Santone A. (2022). FABNet: fusion attention block and transfer learning for laryngeal cancer tumor grading in P63 IHC histopathology images. IEEE Journal of Biomedical and Health Informatics, https://doi.org/10.1109/JBHI.2021.3108999.\u003c/li\u003e\n\u003cli\u003eHuang P., He P., Tian S., Ma M., Feng P., Xiao H., Mercaldo F., Santone A., Qin J. (2023). A ViT-AMC Network With Adaptive Model Fusion and Multiobjective Optimization for Interpretable Laryngeal Tumor Grading From Histopathological Images. IEEE Transactions on Medical Imaging, https://doi.org/10.1109/TMI.2022.3202248.\u003c/li\u003e\n\u003cli\u003eOktay O., Schlemper J., Folgoc L.L., Lee M.J., Heinrich M., Misawa K., Mori K., McDonagh S.G., Hammerla N.Y., Kainz B., Glocker B., Rueckert D. (2018). Attention U-Net: Learning Where to Look for the Pancreas. ArXiv, https://doi.org/10.48550/arXiv.1804.03999.\u003c/li\u003e\n\u003cli\u003eChen J., Lu Y., Yu Q., Luo X., Adeli E., Wang Y., Lu L., Yuille A.L., Zhou Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. ArXiv, https://doi.org/10.48550/arXiv.2102.04306\u003c/li\u003e\n\u003cli\u003eShelhamer E., Long J., Darrell T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, https://doi.org/10.1109/TPAMI.2016.2572683.\u003c/li\u003e\n\u003cli\u003eChen L.C., Papandreou G., Schroff F., Adam H. (2017). Rethinking atrous convolution for semantic image segmentation. ArXiv, https://doi.org/10.48550/arXiv.1706.05587.\u003c/li\u003e\n\u003cli\u003eChen L.C., Zhu Y., Papandreou G., Schroff F., Adam H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. European Conference on Computer Vision, https://doi.org/10.1007/978-3-030-01234-2_49.\u003c/li\u003e\n\u003cli\u003eYang M., Yu K., Zhang C., Li Z., Yang K. (2018). DenseASPP for semantic segmentation in street scenes. IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR.2018.00388.\u003c/li\u003e\n\u003cli\u003eCoudray N., Ocampo P.S., Sakellaropoulos T., Narula N., Snuderl M., Feny\u0026ouml; D., Moreira A.L., Razavian N., Tsirigos A. (2018). Classification and mutation prediction from non\u0026ndash;small cell lung cancer histopathology images using deep learning. Nature Medicine, https://doi.org/10.1038/s41591-018-0177-5.\u003c/li\u003e\n\u003cli\u003eFraz M.M., Khurram S.A., Graham S., Shaban M., Hassan M., Loya A. Rajpoot N.M. (2020). FABnet: feature attention-based network for simultaneous segmentation of microvessels and nerves in routine histology images of oral cancer. Neural Computing and Applications, https://doi.org/10.1007/s00521-019-04516-y.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Lung squamous cell carcinoma, PD-L1, Deep learning, TransUnet, H\u0026E staining","lastPublishedDoi":"10.21203/rs.3.rs-4180916/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4180916/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eDetecting programmed death ligand 1 (PD-L1) expression based on immunohistochemical (IHC) staining is an important guide for the treatment of lung cancer with immune checkpoint inhibitors. However, this method has problems such as high staining costs, tumor heterogeneity, and subjective differences among pathologists. Therefore, the application of deep learning models to segment and quantitatively predict PD-L1 expression in digital sections of Hematoxylin and eosin (H\u0026amp;E) stained lung squamous cell carcinoma is of great significance.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eWe constructed a dataset comprising H\u0026amp;E-stained digital sections of lung squamous cell carcinoma and used a Transformer Unet (TransUnet) deep learning network with an encoder-decoder design to segment PD-L1 negative and positive regions and quantitatively predict the tumor cell positive score (TPS).\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eThe results showed that the dice similarity coefficient (DSC) and intersection overunion(IoU) of deep learning for PD-L1 expression segmentation of H\u0026amp;E-stained digital slides of lung squamous cell carcinoma were 80% and 72%, respectively, which were better than the other seven cutting-edge segmentation models. The root mean square error (RMSE) of quantitative prediction TPS was 26.8, and the intra-group correlation coefficients with the gold standard was 0.92 (95% CI: 0.90\u0026ndash;0.93), which was better than the consistency between the results of five pathologists and the gold standard.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003eThe deep learning model is capable of segmenting and quantitatively predicting PD-L1 expression in H\u0026amp;E-stained digital sections of lung squamous cell carcinoma, which has significant implications for the application and guidance of immune checkpoint inhibitor treatments.\u003c/p\u003e\u003ch2\u003eTrial registration:\u003c/h2\u003e \u003cp\u003eChiCTR1900022640(2019-04-19).\u003c/p\u003e","manuscriptTitle":"Prediction of PD-L1 tumor positive score in lung squamous cell carcinoma with H\u0026amp;E staining whole slide images and deep learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-02 20:02:58","doi":"10.21203/rs.3.rs-4180916/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"24f3f896-61a8-4e40-9384-a210ba87b5f3","owner":[],"postedDate":"April 2nd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-05-29T05:41:12+00:00","versionOfRecord":[],"versionCreatedAt":"2024-04-02 20:02:58","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4180916","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4180916","identity":"rs-4180916","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}