DST-Net: Dual Self-Integrated Transformer Network for Semi-Supervised Segmentation of Optic Disc and Optic Cup in Fundus Image

DST-Net: Dual Self-Integrated Transformer Network for Semi-Supervised Segmentation of Optic Disc and Optic Cup in Fundus Image | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article DST-Net: Dual Self-Integrated Transformer Network for Semi-Supervised Segmentation of Optic Disc and Optic Cup in Fundus Image Jing Wang, Jinke Wang, Zhipeng Yang, Min Xu, Shinichi Tamura This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4890313/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Current neural network-based optic disc (OD) and optic cup (OC) segmentation tend to prioritize the image's local edge features, thus limiting their capacity to model long-term relationships, with errors in delineating the boundaries. To address this issue, we proposed a semi-supervised Dual Self-Integrated Transformer Network (DST-Net) for joint segmentation of the OD and OC. Firstly, we construct the encoder and decoder of the self-integrated network from the mutually enhanced feature learning modules of Vision Transformer (ViT) and Convolutional Neural Networks (CNN), which are co-trained with dual views to learn the global and local features of the image adaptively. Secondly, we employed a dual self-integrated teacher-student network with a substantial amount of unlabeled data, which is utilized through semi-supervised learning to facilitate the acquisition of more refined segmentation outcomes. Finally, the Boundary Difference over Union Loss (BDoU-loss) enhances the network's focus on the boundaries. We implemented the comparative experiments on the publicly available dataset RIGA+. The Dice value of OD and OC of the proposed DST-Net reached 95.12(±)0.14 and 85.69(±)0.27, respectively, outperforming other State-Of-The-Art (SOTA) methods and proving its promising prospect in OD and OC segmentation. Physical sciences/Engineering/Biomedical engineering Biological sciences/Neuroscience/Computational neuroscience/Network models Health sciences/Health care/Medical imaging OD and OC segmentation CNN Transformer Semi-supervised Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 1 Introduction Glaucoma is one of the most severe causes of blindness worldwide. It is anticipated that by 2040, the number of individuals diagnosed with glaucoma will reach 110 million[ 1 ]. The conventional method for localizing and segmenting retinal images relies on manual expertise and is susceptible to subjective and objective variables. This process is also time-consuming and prone to errors, particularly in detecting subtle lesions. Consequently, developing efficient automated glaucoma detection technology is crucial for large-scale screening[ 2 ]. Clinically, the cup-to-disc ratio (CDR) is a pivotal diagnostic and monitoring indicator for glaucoma[ 3 ]. Accurate segmenting of the OD and OC in the retinal image is critical. Under normal conditions, the CDR is small. However, once the CDR increases, the risk of developing glaucoma increases, requiring further examination and treatment. Deep learning methods, especially convolutional neural networks (CNNs), have been widely used in several image segmentation tasks and have demonstrated satisfying performance[ 4 , 5 ]. For example, Guo et al.[ 6 ] proposed the CAFR-CNN framework for cross-domain joint OD and OC segmentation, which consists of faster R-CNN detectors, a spatial attention-based region alignment module, a pyramid ROI alignment module, and a prototype-based semantic alignment module. Luo et al.[ 7 ] proposed a new segmentation architecture, BGA-Net, which introduces auxiliary boundary branching and adversarial learning to segment OD and OC in a multi-labeled manner jointly. Yin et al.[ 8 ] proposed a deep learning-based level set approach for OD and OC segmentation in an automated retinal diagnostic system. Nevertheless, the CNN-based methods still face challenges in modeling long-range relationships due to the inherent limitations of convolutional operations[ 9 ]. With the breakthrough of Transformer architecture, semantic segmentation has ushered in a new development opportunity[ 10 ]. Dosovitskiy et al.[ 11 ] proposed Vision Transformer (ViT) in 2020, which successfully extends the Transformer model to image processing tasks and realizes the integration of natural language processing and computer vision. ViT breaks through the limitation of traditional CNNs on the input image size by serializing image pixels and capturing the global features of an image using the Transformer's self-attention mechanism. It also demonstrates excellent performance on image classification tasks. It is important to note that ViT alone does not yield good semantic segmentation results because it emphasizes global information and lacks local detail extraction. In contrast, CNN focuses on extracting local information, and thus, combining ViT and CNN enables the model to leverage local and global information in images, improving its performance and making it suitable for OD and OC segmentation tasks. However, ViT typically requires a large amount of data for practical training. Besides, fundus image samples are sparse, with many unlabeled data and few labeled data, and thus, data augmentation is not a practical solution. To address the above limitations, we developed a Dual Self-integrated Transformer Network, DST-Net, which combines CNN and ViT for joint segmentation of the OD and OC using a semi-supervised approach. The main contributions of this paper are as follows: A dual self-integrated teacher-student network is constructed using CNN and ViT, with weight updates performed via Exponential Moving Average (EMA). Through a dual-view co-training mechanism, the network fully leverages the mutual learning capabilities of CNN and ViT to learn both global and local image information adaptively. Aiming at the problem of unclear boundaries of the OD and OC, the boundary difference loss function BDoU-loss is employed to enhance the network's attention to the boundaries, improving the accuracy of boundary segmentation. A semi-supervised learning approach utilizes labeled and unlabeled data for training, in which the teacher network generates pseudo-labels to guide the student network, enhancing the model's learning process. 2 Related Work This section discusses the related work in CNN-based and Transformer-based methods. 2.1 CNN-based methods With the rapid development of deep learning technology, CNN-based models have been thoroughly studied in medical image processing. For example, Zilly et al.[ 12 ] enhanced the performance of CNN using an integrated learning technique, which employs an entropy sampling technique to identify the key information points in the image. Wang et al.[ 13 ] proposed an asymmetric segmentation network based on the U-Net model to segment the OD region, which combines the classical U-Net architecture with a unique cross-linked subnetwork to accurately localize the OD and improve sensitivity to morphological variations in ROI. Tulsani et al.[ 14 ] proposed an improved UNET + + architecture for automatically segmenting the OD and OC in glaucoma assessment, in which a clinical feature-based classification model with preprocessing and customized loss functions was introduced to address the problem of vessel and category imbalance. Pachade et al.[ 15 ] proposed a novel segmentation network called NENet, which combines EfficientNetB4 as an encoder and a series of pre-activated residual blocks, atrous space pyramid pooling (ASPP) blocks, and attention gates (AGs). Meanwhile, a modified patch-based discriminator was designed for NENet to improve local segmentation details. Guo et al.[ 16 ] proposed a joint segmentation method for OD and OC based on an improved U-Net architecture. The feature fusion module in U-Net is first added to reduce the information loss in feature extraction. Then, the channel and spatial attention mechanisms are combined to highlight the essential features related to segmentation and suppress the expression of features in irrelevant regions. Finally, multilabel loss is used to generate joint segmentation results. Experimental results show the method performs well on OD and OC segmentation tasks. In [ 17 ], Fu et al. proposed the M-Net based on a multilabel deep network with polar coordinate transformation to segment OD and OC jointly. The model uses U-Net as the backbone network, the input layer as a pyramid model for receiving feature mappings at different scales, and the side output layer outputs the corresponding local prediction results. Currently, most CNN-based methods, such as U-Net and its variants, rely on extracting and fusing multiscale features to improve segmentation accuracy. However, the fused features still have a small "effective receptive field" and are mainly concentrated in localized regions of the image, thus limiting their performance. 2.2 Transformer-based methods With the emergence of Transformer, semantic segmentation entered a brand-new era[ 10 ]. Chen et al.[ 18 ] proposed TransU-Net for medical image segmentation by combining Transformer and U-Net. Cao et al.[ 19 ] designed a pure Transformer similar to U-Net based on the translation window mechanism, Swin-U-Net, for medical image segmentation. Li et al.[ 20 ] proposed a Transformer-based medical image segmentation model Segtran. The model combines the Transformer's unrestricted sensory field at high feature resolution and the advantages of multiscale feature extraction using compressed attentional block specification of the Transformer's self-attention mechanism. It also learns diverse representations using extended blocks. In addition, the method employed a new positional coding scheme that imposes a continuum induction bias on the image. Experimental results show that compared to existing representative methods, Segtran achieves higher segmentation accuracy in the OD and OC segmentation tasks and demonstrates good cross-domain generalization capabilities. Yi et al.[ 21 ] proposed a deep learning model for joint OD and OC segmentation, C2FTFNet (Coarse-to-Fine Transformer Network). The model employs a coarse-to-fine strategy to automatically segment OD and OC in fundus images by gradually increasing the segmentation accuracy. Hussain et al.[ 22 ] proposed a new segmentation method called UT-Net, which exploits the advantages of the UNet and the Transformer in the coding layer and employs an attention-gated bilinear fusion scheme. In addition, multi-head contextual attention is introduced to augment the self-attention used in the traditional visual Transformer. In [ 23 ], Wu et al. proposed a novel Transformer-based model, SeATrans, to transfer segmentation knowledge to disease diagnosis networks. Specifically, an asymmetric multiscale interaction strategy is first proposed to associate each low-level diagnostic feature with a multiscale segmentation feature. Then an efficient strategy called SeA-block is employed to activate the diagnostic features through the associated segmentation features. Although the Transformer architecture performs well in capturing global contextual information, it is inadequate in capturing local spatial information. Besides, transformer-based models usually require much data to train their self-attention mechanism for optimal performance. In addition, there is often a lack of high-quality labeled datasets in medical image segmentation, which limits the model's training effectiveness and generalization ability. Therefore, optimizing the Transformer model under limited data conditions to balance the ability to capture global and local features remains a challenge in current research. 3 Method 3.1 Proposed DST-Net The proposed DST-Net combines the strengths of CNN and Transformer through dual-view co-training and adaptively learns an image's global and local features to achieve high-accuracy segmentation (The code is publicly available at https://github.com/ky120/DST-Net ). The network structure of DST-Net is depicted in Fig. 1 . It is a semi-supervised, self-integrated segmentation network that consists of two self-integrated networks. Each self-integrated network contains a student network, as shown in the upper half of Fig. 1 , and a teacher network, as shown in the lower half of Fig. 1 . The teacher network has the same structure as the student network. Still, its weights are updated as the EMA of the student network's weights[ 24 – 26 ]. Structurally, DST-Net builds a dual self-integration framework to learn more discriminative features. The first self-integration network consists of a U-shaped network composed of CNNs for obtaining the initial segmentation results of the OD and OC, and the second self-integration network consists of a U-shaped structured network composed of two ViTs for more accurate segmentation results. The parameters of the student network in each self-integration framework are passed to the teacher network through the EMA, which in turn updates the weights of the parameters in the teacher network. As shown in Fig. 1 , the network training process is divided into two parts. Firstly, the labeled source domain data is utilized for training, and the student network performs feature extraction by CNN and ViT to generate the predicted feature map. Then, it is compared with the ground truth, and the loss is calculated, which is noted as L1 . The teacher network also generates pseudo-labels using labeled data, compares them to the ground truth, and calculates the loss, which is L2 . Finally, the predicted feature map generated by the student network is compared with the pseudo-label generated by the teacher network. The loss is calculated and noted as L3 ; the three losses are loss fused, and backpropagation is performed on the student network. The weights are passed through the EMA to update the entire network, and the loss functions are all chosen as the BDoU-loss. Secondly, after the training using labeled data is completed, the unlabeled data is fed into the teacher network to generate pseudo-labels. The loss is computed with the predicted feature maps generated by the student network, and the loss is passed to the student network for weight updating and then passed to the teacher network via EMA. To ensure that the teacher network can effectively guide the student network, the teacher network is pre-trained using ImageNet[ 27 ]。 3.2 CNN and ViT-based Encoder-Decoder Structure To fully utilize the feature learning capabilities of CNN and ViT, this section constructs the encoder and decoder of the self-integrated network based on the CNN and the network blocks of ViT, respectively. The four encoders and decoders are connected using U-Net skip connections (shown in Fig. 2 ). Figure 2 a) shows the structure of the CNN-based network, and Fig. 2 c) shows the structure of the ViT-based network. In each CNN-based block, two 3×3 convolutional layers and two batch normalizations are used to build the structure, as shown in Fig. 2 b). The ViT-based network blocks are constructed using Swin-Transformer, as shown in Fig. 2 d). Unlike conventional Transformer blocks, Swin-Transformer employs layer normalization with offset windows, multi-head self-attention, residual connectivity, and Multilayer Perceptions (MLPs) with Gaussian Error Linear Units (GELUs) with a moving window design. This design forms the Windows Multi-head Self-Attention (WMSA) and Shifted Window-based Multi-head Self-Attention (SWMSA) mechanisms. WMSA is a windowed Multihead Self-Attention mechanism that handles the computation of Self-Attention within each window. SWMSA integrates Layer Normalization, Multihead Self-Attention, and Residual Connection to better capture the local features of an image. Through the size of the window and the sliding step, the Swin-Transformer moves over the data sequence and performs the computation. Precisely, a sliding window moves across the data sequence, selecting a continuous data segment as the window at each step. It analyzes or computes the data within the window. The window is then slid forward in fixed steps, and the process continues until the entire data sequence is covered. WMSA and SWMSA are applied in two consecutive Transformer blocks, respectively. In this way, the Transformer block maps the position of the input sequence \(\:{Z}_{0}=[{Z}_{\text{0,1}}\dots\:,{Z}_{0,N}]\) to \(\:{Z}_{L}=[{Z}_{L,1}\dots\:,{Z}_{L,N}]\) . The detailed process of feature learning of the data through the ViT network based on WMSA, SWMSA, and MLP is summarized in Eqs. ( 1 )-( 5 ), where i = [1, L] and L is the number of blocks. $$\:{Z}_{i-1}=WMSA\left(LN\left({Z}_{i-1}\right)\right)+{Z}_{i-1}$$ 1 $$\:{Z}_{i}=MLP\left(LN\left({Z}_{i}\right)\right)+{Z}_{i}\:$$ 2 $$\:{Z}_{i+1}=\text{S}\text{W}\text{M}\text{S}\text{A}\left(\text{L}\text{N}\left({Z}_{i}\right)\right)+{Z}_{i}$$ 3 $$\:{\:Z}_{i+1}=\text{M}\text{L}\text{P}\left(LN\left({Z}_{i+1}\right)\right)+{Z}_{i+1}$$ 4 $$\:\:MSA\left({Z}^{\text{*}}\right)=\text{s}\text{o}\text{f}\text{t}\text{m}\text{a}\text{x}\left(\frac{QK}{\sqrt{D}}\right)V$$ 5 Where \(\:\:Q,K,V\in\:{R}^{{M}^{2}\times\:d}\) , M 2 denotes the number of image blocks in a window, and d represents the dimension of the query and key. The self-attention mechanism consists of three point-wise linear layers that map the input tokens into intermediate representations. It then computes the significance of each element in the sequence relative to every other element. In this way, a query Q, key K, and value V are generated for each component. Then, a weighted summation is performed to obtain a self-attention representation for each element. This self-attention representation determines how much attention each element pays to the other elements. Subsequently, these self-attention representations undergo operations such as normalization and residual connections before being passed into a feedforward neural network. In the feedforward network, each element undergoes point-wise linear transformations and is enhanced through nonlinear transformations to improve its features. Finally, the output from the feedforward network is combined with the initial input through another residual connection, forming a transformed sequence that represents the output of the Transformer module. This process is repeated across multiple Transformer modules to capture hierarchical representations and dependencies within the input sequence. Unlike the basic blocks of CNNs, ViT is designed with merge and expansion layers between the encoder or decoder of each base block[ 18 , 19 ] instead of the traditional subsampling and upsampling steps. This design allows information to flow more freely between layers, preserving more details and facilitating the better capture of complex relationships in the input sequence. The Merge Layer is designed to halve the number of tokens and double the feature dimensions. It divides the input patch into four parts, then joins them together, and finally applies a linear layer to unify the dimensions by a factor of two. The expansion layer is designed to resize the input feature mapping to twice its original size and reduce the feature dimension to half the dimension of the input feature mapping. It uses a linear layer to increase the feature dimension, then employs a rearrangement operation to enlarge the size and reduce the feature dimension to a quarter of the input dimension. The size of the feature mapping in each step is briefly illustrated in Fig. 2 c), where W, H, and C represent the width, height, and channel dimensions of the feature map, respectively. The patch size is set to 4, the input channel is 3, the embedding dimension is 96, and the number of self-attention heads is 3, 6, 12, and 24, respectively, with a window size of 7 for each encoder and decoder. This design enables the ViT to capture the features and relationships within the input sequence more effectively while avoiding the subsampling and upsampling steps in traditional CNNs, thus improving model performance and efficiency. 3.3 Exponential Moving Average (EMA) EMA is commonly used to smooth time series data. The principle is to perform a weighted average of the series data, giving more weight to recent data points and less to earlier data points. It effectively smoothes the time series data, making it more continuous and stable. In deep learning, EMA is commonly used to smooth the update of model parameters. Specifically, EMA is applied to the model parameters each time they are updated, thus reducing the fluctuation of each update and making the model more stable. The formula for EMA is shown in Eq. ( 6 ): $$\:EMA\left[t\right]={\alpha\:}^{\text{*}}x\left[t\right]+(1-\alpha\:{)}^{\text{*}}EMA[t-1]$$ 6 Where t represents the time step, x ( t ) denotes the original data at the t -th time point, \(\:\alpha\:\) is the smoothing factor, which usually takes a value between 0 and 1, indicating the weight of the current sample, (1- \(\:\alpha\:\) ) represents the weight of the historical data, and EMA[ t -1] is the EMA value of the previous time point. The EMA method is employed in the teacher-student network to transfer weights and enhance the model's generalization capability. The core idea of this method is to introduce an EMA to integrate the model's predictions at different time steps. EMA assigns greater weight to recent predictions, allowing the model to focus on the most current information. During the weight transfer process, the student network performs forward propagation to compute the EMA weights, which are then applied to the corresponding layers of the teacher network. In this way, the parameters of the teacher network are updated by exponentially weighted moving averages during training, making the teacher network more robust and guiding the student network to use pseudo-labeling for learning in the context of consistency concerns. 3.4 Boundary Difference over Union Loss (BDoU-Loss) Most of the current image segmentation algorithms uniquely label the image according to specific rules, which are affected by the geometric constraints of the view. Especially in the fundus image of a lesion, since the captured fundus image is a two-dimensional (2D) planar view while the OC is a three-dimensional (3D) anatomical structure, the OD and OC region presents a circular 2D planar projection in the fundus image, which prevents the accurate representation of its true 3D shape. This leads to an uneven distribution of pixels between the OC and the background region, which reduces the boundary segmentation accuracy of the OC. The comparison between the 3D anatomical structure and the 2D planar of the OD and OC in the fundus image is shown in Fig. 3 . Current loss functions for medical image segmentation primarily focus on the overall segmentation results, and less loss is proposed for the guidance of boundary segmentation. We employ the BDoU-Loss[ 28 ], abbreviated as L BD , to address this issue to enhance boundary segmentation. L BD calculates the error region near the boundary by computing the set difference between the ground truth and prediction. The error region is then reduced by minimizing the difference ratio set to the partial intersection or union. The task of joint segmentation of OD and OC is a multilabel problem, where a specific pixel can belong to multiple categories. L BD improves the attention to the region near the boundary and effectively addresses the challenges of insufficient attention to boundary segmentation and uneven categorization. The schematic diagram of the L BD calculation is shown in Fig. 4 . Figure 4 shows the structure of the principle of BDoU-loss and the green line region on the right represents the area of the union minus the intersection of the prediction and ground truth. Below, a hyperparameter \(\:\alpha\:\) controls this union minus the intersection area. In this way, the boundary of different images can be adjusted to better guide the boundary segmentation, thus increasing the accuracy of OD and OC segmentation. The principle is illustrated in Eq. ( 7 ). $$\:{L}_{BD}=\frac{G\cup\:P-G\cap\:P}{G\cup\:P-{\alpha\:}^{\text{*}}G\cap\:P}$$ 7 Where \(\:\alpha\:\) is a hyperparameter that controls the influence of the partial joint region. \(\:\alpha\:\) is calculated as shown in Eq. ( 8 ). $$\:\alpha\:=1-2\times\:\frac{C}{S},\alpha\:\in\:\left[\text{0,1}\right)$$ 8 Where C is the boundary length, and S is the target size. 4. Experiments 4.1 Dataset The proposed network is evaluated using the RIGA + dataset. RIGA+ [ 28 , 29 ] is a composite dataset comprising five subsets: Binrush, Magrabia, BASE1, BASE2, and BASE3. Magrabia and Binrush serve as the source domain, while BASE1, BASE2, and BASE3 are used as the target domain 1, target domain 2, and target domain 3. Binrush and Magrabia have 195 and 95 labeled fundus retinal images for semi-supervised training, respectively. Labeled and unlabeled retinal fundus images are present in BASE1, BASE2, and BASE3, where unlabeled images are used for semi-supervised training and labeled images are used to test the segmentation performance of the model. BASE1 consists of 227 unlabeled images and 35 labeled images, BASE2 includes 238 unlabeled images and 30 labeled images, and BASE3 comprises 252 unlabeled images and 27 labeled images. In RIGA+, all images are cropped to an 800 × 800 OD region. Table 1 details the RIGA + dataset information, and Fig. 5 shows the fundus images of the source and target domains. Table 1 Introduction to the RIGA + dataset Domain Dataset Names Numbers (Training + Test) Image Size Source BinRushed 195(195 + 0) 800 \(\times\) 800 Source Magrabia 95(95 + 0) 800 \(\times\) 800 Target 1 MESSIDOR-BASE1 173(138 + 35) 800 \(\times\) 800 Target 2 MESSIDOR-BASE2 148(118 + 30) 800 \(\times\) 800 Target 3 MESSIDOR-BASE3 133(106 + 27) 800 \(\times\) 800 4.2 Evaluation Metrics Dice and Hausdorff Distance are used as OD and OC segmentation evaluation metrics. The dice coefficient is employed as an evaluation metric to compare the segmentation results of different methods. Higher coefficient values indicate better segmentation results for both OD and OC. The expression of Dice is shown in Eq. (9). In most datasets, classic methods yield Dice values for OD segmentation in the range of 70–80% and for OC segmentation in the range of 65–70%. Improving the Dice values for segmentation is crucial for future OD and OC segmentation research. Dice \(\:=\frac{2\times\:TP}{2\times\:TP+FP+FN}\) (9) Hausdorff Distance is a metric used to determine the degree of resemblance between two point sets. It is beneficial in comparing images or evaluating the similarity between image segmentation results. Given two point sets \(\:A=\{a1,\dots\:,ap\},B=\{b1,\dots\:,bq\}\) , the Hausdorff Distance H(A, B) between these two sets is defined as shown in Eqs. ( 10 )-( 12 ): $$\:H(A,B)=\text{m}\text{a}\text{x}\left(h\right(A,B),h(B,A\left)\right)$$ 10 $$\:h(A,B)=\underset{a\in\:A}{\text{m}\text{a}\text{x}}\:\left\{\underset{b\in\:B}{\text{m}\text{i}\text{n}}\parallel\:a-b\parallel\:\right\}$$ 11 $$\:h(B,A)=\underset{b\in\:B}{\text{m}\text{a}\text{x}}\:\left\{\underset{a\in\:A}{\text{m}\text{i}\text{n}}\parallel\:b-a\parallel\:\right\}$$ 12 h ( A , B ) and h ( B , A ) are the directed Hausdorff distance from A to B and B to A, respectively. In image segmentation, a variant of the Hausdorff distance, Hausdorff Distance at 95th percentile (HD95), is often used to assess the quality of the segmentation results, measuring the distance between two sets. It measures the discrepancy between the segmentation result and the ground truth by calculating the distances between all point pairs in the two sets and finding the 95th percentile of these distances. HD95 is commonly used to assess the accuracy of segmentation results, where a smaller value indicates a higher degree of agreement with the ground truth and, consequently, better segmentation quality. The Hausdorff Distance is highly sensitive to outliers, but HD95 mitigates this sensitivity by using the percentile of the distances, providing a more robust metric. 4.3 Implementation details 4.3.1 Experimental Setup We implemented the experiment using Ubuntu 18.04, Pytorch version 1.7, Cuda version 11.4, and Python version 3.7 throughout the OD and OC segmentation comparison, ablation, and generalization experiments. The network training hyperparameters are listed in Table 2 . Table 2 Network training hyperparameters Parameter name Value Epoch 200 Batch Size 16 Learning Rate 0.005 Optimizer Adam Weight Decay 0 4.3.2 Image Preprocessing The overall preprocessing process is divided into two stages. The first stage operates on a single fundus image, including grayscale conversion, standardization, Contrast Limited Adaptive Histogram Equalization (CLAHE), and gamma correction. The objective is to enhance the image's contrast, reduce the network training burden, and accelerate convergence. The second stage focuses on data augmentation, including random cropping, image rigid transformation, and elastic transformation, as shown in Fig. 6 . In the first stage, to speed up the convergence of the training network and reduce the network training burden, three-channel color images are converted into single-channel grayscale images. The grayscale images are standardized to improve the model's generalization ability and achieve the unity of the metrics. The CLAHE algorithm is used to enhance the local contrast. Finally, a nonlinear Gamma correction is applied to adjust the light intensity of the input retinal image, performing nonlinear operations on the intensity values to establish an exponential relationship between the input and output image intensities. After the preprocessing stage, the OD is localized and cropped to enlarge the area occupied by the OD in preparation for subsequent data augmentation and OD and OC segmentation. To enhance the limited medical image dataset, data augmentation is an effective strategy to mitigate insufficient training data and reduce overfitting. Various rigid and elastic transformations are applied to the images in the dataset, including scaling, rotation, mirroring, and B-spline elastic deformation. Finally, the dataset images are randomly cropped, and the cropped image blocks of 256×256 are then used for network training. 4.4 Ablation Analysis We conducted ablation experiments to validate the effectiveness and contribution of each module within the proposed network for OD and OC segmentation. For the teacher-student network structure, various combinations of CNN and ViT were tested to evaluate the effectiveness of their integration in the model, with the ablation results presented in Table 3 . Table 3 Results of ablation experiments on the BASE1 dataset Model Dice Disc (%) Dice Cup (%) Student Teacher CNN CNN 76.32 ± 0.98 61.35 ± 1.01 ViT ViT 76.21 ± 1.41 63.81 ± 1.56 CNN + CNN CNN + CNN 82.65 ± 0.38 73.07 ± 0.27 ViT + ViT ViT + ViT 80.76 ± 0.65 70.58 ± 0.49 ViT + CNN ViT + CNN 92.60 ± 0.12 82.62 ± 0.19 CNN + ViT CNN + ViT 95.12 ± 0.14 85.69 ± 0.27 Table 3 shows that the combination of CNN and ViT achieves the best OD and OC segmentation performance, with Dice scores of 95.12 ± 0.14 for the OD and 85.69 ± 0.27 for OC. In contrast, the combinations of ViT + ViT and CNN + CNN result in the poorest segmentation performance. The superior performance of CNN + ViT can be attributed to the strengths of both components: CNN's spatial locality allows for the effective capture of local features, and its translation invariance helps the model handle variations in the positions of the OD and OC within the image. Meanwhile, ViT's global awareness facilitates the establishment of connections across the entire image, making it well-suited for capturing the global relationship and overall structural features between OD and OC. The ViT + ViT and CNN + CNN combinations perform poorly due to information redundancy. Using two CNNs can result in redundant extraction of similar features, while two ViTs may lead to redundancy in global perception. The ViT + CNN combination may underperform if the feature extraction in the ViT stage is not robust enough, limiting the CNN stage's ability to classify with rich features, thus constraining the model's performance. Also, the Transformer network may lose some spatial information when processing images because it does not explicitly consider spatial localization as CNN does, which may impact segmentation tasks such as OD and OC that require spatial information. In summary, the combination of using CNN to perform local feature extraction first and then establishing global association through ViT allows the model to maintain spatial information while being able to globally associate and understand semantic features in different regions of the image, improving the flexibility and versatility of the model and maximizing the advantages of both. The visualization results of the ablation experiments are shown in Fig. 7 , and the visualization comparison map can more intuitively demonstrate the segmentation differences between different combinations. To better illustrate the segmentation results of varying module combinations, the segmentation of the OD and OC is extracted and shown in green. In contrast, the ground truth is shown in red. By overlaying these two colors, the resulting visualization indicates segmentation quality. In this way, the segmentation effect can be visualized. From Fig. 7 , it can be seen that the CNN + CNN combination has obvious over-segmentation and under-segmentation when processing the OC region. For the segmentation of the OD, the over-segmentation is particularly prominent. This is because the combination pays too much attention to local feature extraction, resulting in strong segmentation ability in the local region and many over-segmentation phenomena due to the weak performance in the segmentation judgment of the boundary part. This indicates that relying solely on local feature extraction can lead to over-extraction of features, thereby degrading segmentation performance. The combination of ViT + ViT performs well in overall control but is deficient in detailed feature extraction. As a result, this combination tends to experience under-segmentation issues in OD and OC segmentation. The significant presence of red areas in the figure indicates that under-segmentation is a prevalent problem. Secondly, the ViT + CNN combination has certain deficiencies in feature extraction. Due to the poor effect of feature extraction in the ViT stage, the impact of feature extraction in the CNN stage is also affected. As can be seen from the figure, although there is an improvement compared to the first two combinations, there are still more under-segmentation cases. Finally, the CNN + ViT combination demonstrates excellent performance. After feature extraction by CNN, the model obtains rich local feature information, while ViT excels in overall detail management. This combination effectively combines the local and global information, significantly improving the model's over-segmentation and under-segmentation. As can be seen from the figure, the red and green regions in OD and OC regions are the least in all comparisons. 4.5 Quantitative Comparison with SOTA Methods In this section, the proposed network is compared in detail with SOTA methods. Table 4 details the experimental results of the various methods on the BASE1, BASE2, and BASE3 datasets. The evaluation metric used is the mean Dice score, with the standard deviation indicated and the best results highlighted in bold. Table 4 Performance of methods on the BASE1, BASE2, and BASE3 dataset Methods BASE1 BASE2 BASE3 Dice Disc (%) Dice Cup (%) Dice Disc (%) Dice Cup (%) Dice Disc (%) Dice Cup (%) AdaEnt[32] 94.46 ± 0.03 82.71 ± 0.06 92.77 ± 0.02 77.79 ± 0.03 93.72 ± 0.03 81.87 ± 0.04 AdaMI[33] 94.50 ± 0.06 82.80 ± 0.19 92.72 ± 0.02 78.86 ± 0.19 93.65 ± 0.06 82.71 ± 0.11 FSM[34] 94.96 ± 0.63 84.30 ± 1.47 93.10 ± 0.32 81.39 ± 0.91 94.41 ± 0.98 83.21 ± 1.92 pOSAL[30] 94.01 ± 0.23 83.37 ± 0.09 95.09 ± 0.12 84.28 ± 0.17 94.77 ± 0.12 84.06 ± 0.15 BEAL[31] 95.31 ± 0.35 85.44 ± 0.21 95.57 ± 0.34 83.18 ± 0.25 94.79 ± 0.19 83.98 ± 0.22 ProFSDA[35] 95.29 ± 0.12 85.61 ± 0.24 94.71 ± 0.01 85.33 ± 0.08 95.47 ± 0.01 85.53 ± 0.15 HPFG[36] 93.11 ± 0.16 84.43 ± 0.21 93.12 ± 0.43 83.88 ± 0.31 93.23 ± 0.23 84.01 ± 0.21 DST-Net 95.12 ± 0.14 85.69 ± 0.27 95.97 ± 0.31 85.37 ± 0.23 95.49 ± 0.17 85.62 ± 0.11 As shown in Table 4 , DST-Net performs best segmentation on the BASE2 and BASE3 datasets. However, in the BASE1 dataset, the OD Dice score is slightly lower than that of the BEAL network due to the presence of images with lesions. The BEAL network generates more precise boundaries and reduces high-uncertainty predictions in the segmentation of the OD and OC by encouraging similarity between boundary predictions and mask probability entropy maps (uncertainty maps) in the target and source domains. This approach provides an advantage in OD boundary segmentation and enhances accuracy in identifying pathological regions. Figure 8 displays the segmentation results of several networks on the BASE1, BASE2, and BASE3 datasets. It can be observed that DST-Net performs well in segmenting both OD and OC. BEAL performs better than pOSAL for OD segmentation, showing its advantage in boundary segmentation, especially in cases where the OD has a more regular boundary. However, for the case of irregular boundaries such as the OC, the BEAL model has more obvious under-segmentation. In contrast, DST-Net achieves better segmentation of irregular boundaries by leveraging transfer learning through the pre-training of the teacher network, allowing it to better adapt to the OD and OC segmentation tasks. With the cooperation of CNN and ViT, DST-Net has significantly improved its segmentation effect in dealing with irregular boundaries. To more intuitively demonstrate the proposed network's focus on different regions of fundus images, heatmaps were generated to illustrate the network's attention across various areas, as shown in Fig. 9 . It can be observed from Fig. 9 that the blue region indicates that the network pays less attention to the region, while the red region indicates that the network pays more attention to the region. As each layer's weights accumulate, the attention to the OD and OC regions gradually increases, indicating that the segmentation network is more focused on these areas, resulting in improved performance. In the heatmap of the BASE2 dataset, the OD region is predominantly covered by red, showing relatively good segmentation. However, some black shadowed areas are at the bottom, probably due to the interference of blood vessels in the early stages of training, attracting the model's attention. In the heatmap of the BASE3 dataset, the focus is mainly on the OD and OC regions, with other areas showing a deep blue, indicating that the segmentation of the OD and OC has been successful. Despite some black shadow regions affecting the segmentation performance, the model shows great competitiveness overall. This finding provides direction for subsequent optimization and improvement for challenging regions. Meanwhile, the combined analysis with the heatmap offers a clearer understanding of the regions of interest of the network for the fundus image, which will help to further optimize the network structure and parameters for more accurate OD and OC segmentation. 4.6 Loss Function Analysis on the RIGA + Dataset To verify the effectiveness of the boundary difference loss function selected in this paper. This section will use four loss functions, IoU-loss, Dice-loss, MSE-loss, and BDoU-loss, to conduct comparative experiments on the BASE1 dataset of RIGA+. The Dice similarity coefficient is used to measure the effectiveness of the boundary differential loss function, and the HD95 is used to evaluate boundary attention. The experimental results are shown in Table 5 . Table 5 Comparative experiments of different loss functions Loss Function Dice Disc (%) Dice Cup (%) HD95 IoU-loss 94.76 ± 0.78 83.59 ± 0.66 14 Dice-loss 95.44 ± 0.55 83.94 ± 0.63 17 MSE-loss 87.14 ± 0.25 76.90 ± 0.23 19 BDoU-loss 95.12 ± 0.14 85.69 ± 0.27 11 As can be seen in Table 5 in the comparison of the metrics, the MSE-loss performs the worst across various metrics. Its principle involves penalizing model errors by calculating the squared error between prediction and the ground truth, making it highly sensitive to outliers. Furthermore, the MSE loss is less effective in dealing with classification problems, contributing to poor segmentation. Compared with IoU-loss, BDoU-loss achieves higher Dice values for both OD and Dice. The main reason is that for triple classification problems like OD and OC segmentation, the BDoU-loss pays more attention to capturing the detailed information of the segmentation boundary, especially for the OD boundary and the OD and OC boundary, thereby improving the model's segmentation accuracy. In terms of OD segmentation, BDoU-loss performs slightly worse than Dice-loss. However, it achieves a higher average Dice value overall. Dice-Loss usually has better sensitivity to boundary prediction and thus has higher accuracy in segmentation tasks with regular boundaries, which is better reflected in OD segmentation. Nevertheless, due to the low contrast of the OD and OC boundary and the relatively irregular boundary of the OC, boundary incompleteness might occur, leading to slightly inferior segmentation performance. In contrast, BDoU-loss is made more tolerant to some slight boundary incompleteness by introducing a limiting factor, which mitigates the negative impact of these incompleteness and irregular shapes on the segmentation of the OD and OC, resulting in better segmentation of the OC region. Overall, BDoU-loss demonstrates the best comprehensive performance among the compared loss functions, yielding the most optimal segmentation results. The superiority of BDoU-loss is further illustrated by the variation in loss values during the training process with different loss functions, as shown in Fig. 10 . As shown in Fig. 10 , the loss curve of BDoU-loss during the training process is smoother, with more minor training fluctuations. This indicates that BDoU-loss provides superior stability against outliers or noise during training. In datasets with noise or anomalies, BDoU-loss helps the model better adapt to these conditions. A comparison of training speeds reveals that BDoU-loss tends to converge more quickly to local minima during training due to the introduction of a constraint factor. This means it has a faster convergence speed, reducing training time and improving efficiency. To show the segmentation differences between different loss functions more intuitively, the segmentation results of the ground truth and different loss functions are drawn on the original graph with blue lines for visualization, as shown in Fig. 11 . Since the boundaries of the vast majority of the OD and OC segmentation results are continuous, the segmentation results can be observed more intuitively by extracting the boundaries of the segmentation results of different loss functions and overlaying them into the original image, as shown in Fig. 11 . From the visualization comparison. It can be seen that the Dice-loss and BDoU-loss yield almost identical segmentation results for the OD. However, BDoU-loss shows better performance for OC segmentation, which is closer to the ground truth. IoU-loss performs poorly for the OC segmentation and introduces many irregular areas in the OD segmentation. This is because IoU-loss is more sensitive to minor deviations in boundary predictions, and such sensitivity might penalize the model when slight irregularities appear in predicted boundaries, resulting in inferior performance compared to Dice-loss and BDoU-loss, which shows the importance of boundary segmentation for OD and OC segmentation. MSE-loss is less effective for both OD and OC segmentation. The reason is that it does not consider the relative positional relationship between pixels and thus may not be able to handle the boundary ambiguity in the segmentation task effectively. If there is an imbalance in pixel distribution between the OD and OC, such as when one category has significantly more pixels than the other, the MSE loss function may guide the model to favor the category with a higher number, resulting in the model being more inclined to learn the category with a higher number while ignoring the one with a lower number. 4.7 Parameters and FLOPs We compare our DST-Net with SOTA. Table 6 lists the Giga Floating-point Operations Per Second (GFLOPs) and the Dice scores for OD and OC for the different methods on the BASE2 dataset. In general, the number of parameters of a network is proportional to its computational complexity, and smaller parameters tend to degrade the network's performance. Table 6 shows that the proposed DST-Net performs best regarding OD and OC segmentation accuracy compared to the listed methods. The main reason is that the teacher model is trained by semi-supervised learning, and the generated pseudo-labels are used to train the student model, which improves the model's performance and reduces the need for expensive expert labeling. Table 6 Training results for each model trained on the BASE2 dataset Method Parameters GFLOPs BASE2 Dice Disc (%) Dice Cup (%) AdaEnt[32] 40774656 3.621 92.77 ± 0.02 77.79 ± 0.03 FSM[34] 34273776 3.736 93.10 ± 0.32 81.39 ± 0.91 pOSAL[30] 53118960 4.608 95.09 ± 0.12 84.28 ± 0.17 BEAL[31] 63247099 6.410 95.57 ± 0.34 83.18 ± 0.25 ProFSDA[35] 42513479 4.531 94.71 ± 0.01 85.33 ± 0.08 HPFG[36] 140641904 12.528 93.12 ± 0.43 83.88 ± 0.31 Proposed DST-Net 81672536 8.26 95.97 ± 0.31 85.37 ± 0.23 In addition, the proposed method is the second to last in terms of the number of parameters and GLOPs. This is mainly because Transformer methods typically have more parameters than multiscale CNN methods regarding parameter comparison. However, the proposed DST-Net comprises both CNN and Transformer modules, only using high-level features extracted by CNNs for global feature modeling in the Transformer, which can significantly reduce computational load to a certain extent. 5 Conclusions In this paper, we proposed a Dual Self-Integrated Transformer Network (DST-Net) for semi-supervised segmentation of OD and OC. The proposed DST-Net fully leverages the characteristics of both ViT and CNN networks to balance local and global information. A semi-supervised approach effectively utilizes large amounts of unlabeled data for network training. Aiming at the problem that the real 3D morphology of the OD and OC cannot be displayed in the 2D fundus images, which leads to uneven pixel distribution and unclear boundaries, we introduce BDoU-loss. This loss function focuses on boundary segmentation, improving accuracy at the irregular boundaries between the OD and OC. Ablation experiments verify the effectiveness of the CNN and ViT combination. In comparison experiments, DST-Net demonstrated superior performance on the BASE1, BASE2, and BASE3 datasets, with OD Dice scores of 0.9512, 0.9597, and 0.9549, and OC Dice scores of 0.8569, 0.8537, and 0.8562, respectively. The analysis of loss functions confirms the effectiveness of the BDoU-loss selected in this paper. Future research could further optimize the network structure of DST-Net to reduce the number of parameters while maintaining the segmentation accuracy to enhance model performance and optimize computational efficiency further. Declarations Additional Information The authors declare that they have no conflict of interest. Author Contribution Jing Wang: Writing–original draft, Validation. Jinke Wang: Conceptualization, Methodology. Zhipeng Yang: Investigation, Visualization. Min Xu: Data enhancement. Shinichi Tamura: Writing – review & editing. Data Availability The datasets used in this paper are publicly available as follows: https://deepblue.lib.umich.edu/data/concern/data_sets/3b591905z, and https://www.adcis.net/en/third-party/messidor2/ References Y.-C. Tham, X. Li, T.Y. Wong, H.A. Quigley, T. Aung, C.-Y. Cheng, Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis, Ophthalmology 121 (2014) 2081–2090. Thompson A C, Jammal A A, Medeiros F A. A Review of Deep Learning for Screening, Diagnosis, and Detection of Glaucoma Progression[J]. Translational Vision Science & Technology, 2020, 9(2): 42–42. Zhao R, Chen X, Liu X, et al. Direct cup-to-disc ratio estimation for glaucoma screening via semi-supervised learning[J]. IEEE journal of biomedical and health informatics, 2019, 24(4): 1104–1113. Jia C, Shi F, Zhao M, et al. Semantic segmentation with light field imaging and convolutional neural networks[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1–14. Hassan T, Hassan B, Akram M U, et al. Incremental cross-domain adaptation for robust retinopathy screening via Bayesian deep learning[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1–14. Guo Y, Peng Y, Zhang B. CAFR-CNN: coarse-to-fine adaptive faster R-CNN for cross-domain joint optic disc and cup segmentation[J]. Applied Intelligence, 2021, 51: 5701–5725. Luo L, Xue D, Pan F, et al. Joint optic disc and optic cup segmentation based on boundary prior and adversarial learning[J]. International Journal of Computer Assisted Radiology and Surgery, 2021, 16(6): 905–914. Yin P, Xu Y, Zhu J, et al. Deep level set learning for optic disc and cup segmentation[J]. Neurocomputing, 2021, 464: 330–341. Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arxiv preprint arxiv:2102.04306, 2021. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arxiv preprint arxiv:2010.11929, 2020. Zilly J, Buhmann J M, Mahapatra D. Glaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation[J]. Computerized Medical Imaging and Graphics, 2017, 55: 28–41. Wang L, Gu J, Chen Y, et al. Automated segmentation of the optic disc from fundus images using an asymmetric deep learning network[J]. Pattern recognition, 2021, 112: 107810. Tulsani A, Kumar P, Pathan S. Automated segmentation of optic disc and optic cup for glaucoma assessment using improved UNET + + architecture[J]. Biocybernetics and Biomedical Engineering, 2021, 41(2): 819–832. Pachade S, Porwal P, Kokare M, et al. NENet: Nested EfficientNet and adversarial learning for joint optic disc and cup segmentation[J]. Medical Image Analysis, 2021, 74: 102253. Guo X, Li J, Lin Q, et al. Joint optic disc and cup segmentation using feature fusion and attention[J]. Computers in Biology and Medicine, 2022, 150: 106094. Fu H, Cheng J, Xu Y, et al. Joint optic disc and cup segmentation based on multilabel deep network and polar transformation[J]. IEEE transactions on medical imaging, 2018, 37(7): 1597–1605. Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arxiv preprint arxiv:2102.04306, 2021. Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2022: 205–218. Li S, Sui X, Luo X, et al. Medical image segmentation using squeeze-and-expansion transformers[J]. arxiv preprint arxiv:2105.09511, 2021. Yi Y, Jiang Y, Zhou B, et al. C2FTFNet: Coarse-to-fine transformer network for joint optic disc and cup segmentation[J]. Computers in Biology and Medicine, 2023, 164: 107215. Hussain R, Basak H. UT-Net: Combining U-Net and Transformer for Joint Optic Disc and Cup Segmentation and Glaucoma Detection[J]. arxiv preprint arxiv:2303.04939, 2023. Wu J, Fang H, Shang F, et al. SeATrans: learning segmentation-assisted diagnosis model via transformer[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 677–687. Han B, Yao Q, Yu X, et al. Co-teaching: Robust training of deep neural networks with extremely noisy labels[J]. Advances in neural information processing systems, 2018, 31. Tarvainen A, Valpola H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[J]. Advances in neural information processing systems, 2017, 30. Laine S, Aila T. Temporal ensembling for semi-supervised learning[J]. arxiv preprint arxiv:1610.02242, 2016. Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248–255. Decencière E, Zhang X, Cazuguel G, et al. Feedback on a publicly distributed image database: the Messidor database[J]. Image Analysis and Stereology, 2014, 33(3): 231–234. Almazroa A, Alodhayb S, Osman E, et al. Retinal fundus images for glaucoma analysis: the RIGA dataset[C]//Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. SPIE, 2018, 10579: 55–62. Wang S, Yu L, Yang X, et al. Patch-based output space adversarial learning for joint optic disc and cup segmentation[J]. IEEE transactions on medical imaging, 2019, 38(11): 2485–2495. Wang S, Yu L, Li K, et al. Boundary and entropy-driven adversarial learning for fundus image segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22. Springer International Publishing, 2019: 102–110. Bateson M, Kervadec H, Dolz J, et al. Source-relaxed domain adaptation for image segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part I 23. Springer International Publishing, 2020: 490–499. Bateson M, Kervadec H, Dolz J, et al. Source-free domain adaptation for image segmentation[J]. Medical Image Analysis, 2022, 82: 102617. Yang C, Guo X, Chen Z, et al. Source free domain adaptation for medical image segmentation with fourier style mining[J]. Medical Image Analysis, 2022, 79: 102457. Hu S, Liao Z, Xia Y. ProSFDA: prompt learning based source-free domain adaptation for medical image segmentation[J]. arXiv preprint arXiv:2211.11514, 2022. Li F, Jiang A, Li M, et al. HPFG: semi-supervised medical image segmentation framework based on hybrid pseudo-label and feature-guiding[J]. Medical & Biological Engineering & Computing, 2024, 62(2): 405–421. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4890313","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":354695520,"identity":"1dc378f9-718d-41ef-a71f-382899240211","order_by":0,"name":"Jing Wang","email":"","orcid":"","institution":"Harbin University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jing","middleName":"","lastName":"Wang","suffix":""},{"id":354695521,"identity":"f770c805-c1a7-4756-981e-97fbfe461113","order_by":1,"name":"Jinke Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABUklEQVRIie3QMUsDMRQH8ISDdKl2fUfBfoUcgdRiRT+G45VCu1wdi0MpASEuLa4e6HfoJDp5Urgup64HdigIt3jCuUjFIubaUmx73QXvT3gkIT9eEoTSpPmDyYnF1IwLodP5uwCE1EDZdQJOEsG2ANhE6OEyQVOibQm0mWS6QRRNhjvFTDUYjU+2GR089F/3b3bb9Fng0ZtEheIKyQ6YbcuAlTpB0eh4hFPvuLbX8ADo0NGMK4mMW7FMoKapa/QrPd/keSxJmTsWZw0JwH2T5LckMqmzTiaTmNQ/8vhbkaeQs9KMZL6SyJEiiMTEUl0E4dy32Aued9ESCHiuhrvqLdQLm3rHJezAD7naAf3Or5zql49g9JZJ7kxqaKx+jA7q1zBuuYZ9brHoU7Zz+kX1Pgqb5cJKl3kWm25cCMwWOP4qSDr/m7TiokUbjqVJkybN/8wP9e55nzS6XoIAAAAASUVORK5CYII=","orcid":"","institution":"Harbin University of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Jinke","middleName":"","lastName":"Wang","suffix":""},{"id":354695522,"identity":"9f4fe34f-1c86-4650-b40c-d48c1ea54a8b","order_by":2,"name":"Zhipeng Yang","email":"","orcid":"","institution":"Harbin University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Zhipeng","middleName":"","lastName":"Yang","suffix":""},{"id":354695523,"identity":"5f1d6b3b-e532-483c-ad23-263a59cfcf69","order_by":3,"name":"Min Xu","email":"","orcid":"","institution":"Weihai Municipal Hospital, Affiliated to Shandong University","correspondingAuthor":false,"prefix":"","firstName":"Min","middleName":"","lastName":"Xu","suffix":""},{"id":354695524,"identity":"8e8f33d2-eb1c-4e75-a442-7e66113e3015","order_by":4,"name":"Shinichi Tamura","email":"","orcid":"","institution":"SANKEN, Osaka University","correspondingAuthor":false,"prefix":"","firstName":"Shinichi","middleName":"","lastName":"Tamura","suffix":""}],"badges":[],"createdAt":"2024-08-10 06:38:49","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4890313/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4890313/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":66682974,"identity":"4c081b2e-93d5-453b-a575-048ca2b359c3","added_by":"auto","created_at":"2024-10-15 12:36:30","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":701780,"visible":true,"origin":"","legend":"\u003cp\u003eDST-Net network architecture\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/c9181dc3a5ccfdd32e114041.png"},{"id":66683908,"identity":"3d8bcd42-85b7-4ede-b9a5-3c454529c969","added_by":"auto","created_at":"2024-10-15 12:44:30","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":847571,"visible":true,"origin":"","legend":"\u003cp\u003eCNN and ViT-based encoding and decoding structure\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/82a8fb35cb7a4349166da515.png"},{"id":66682594,"identity":"6b5baac4-8394-4968-a9f8-312655e58c5f","added_by":"auto","created_at":"2024-10-15 12:28:30","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":1655181,"visible":true,"origin":"","legend":"\u003cp\u003eStereoscopic and planar comparison of OD and OC(a) 3D anatomical structure (b) 2D planar structure\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/c47f6d221ac0aabf5ff646e6.png"},{"id":66682597,"identity":"c00447b8-ffdd-4809-b8be-ef70e61f5764","added_by":"auto","created_at":"2024-10-15 12:28:30","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":173788,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic diagram of Boundry-DoU Loss calculation\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/cbab9a8d7b8c12a3bbb30b1c.png"},{"id":66682602,"identity":"047d87a0-4b67-4b84-b05c-656a47006706","added_by":"auto","created_at":"2024-10-15 12:28:30","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":2865265,"visible":true,"origin":"","legend":"\u003cp\u003eFundus images of the source and target domains of the RIGA+ dataset\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/d0bf103f3c9a395bd6292f60.png"},{"id":66684164,"identity":"06148778-b0f5-4ac0-967b-dd8afeb12ced","added_by":"auto","created_at":"2024-10-15 12:52:30","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":965720,"visible":true,"origin":"","legend":"\u003cp\u003eImage preprocessing process\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/81076eea92edc9f382e1be99.png"},{"id":66682976,"identity":"203b23be-26aa-4b71-9e2d-db00d4f03e27","added_by":"auto","created_at":"2024-10-15 12:36:30","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":997002,"visible":true,"origin":"","legend":"\u003cp\u003eVisualization of ablation experiment on BASE1 datasets (green is the prediction result, red denotes the ground truth, and yellow represents the overlaying region)\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/b9fc92eb0b1cb295f57df247.png"},{"id":66682978,"identity":"fd513d34-fcae-454d-bd82-2ec3d19c390e","added_by":"auto","created_at":"2024-10-15 12:36:30","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":3058461,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of visualization of different network segmentation results ( blue line represents ground truth)\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/b4343d1c1e2391bb13c4fff2.png"},{"id":66682600,"identity":"6af01f8f-c6fd-4f1e-a3ba-a68290991d69","added_by":"auto","created_at":"2024-10-15 12:28:30","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":6335300,"visible":true,"origin":"","legend":"\u003cp\u003eHeat map displaying the attention of different weight layers of the network\u003c/p\u003e","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/94fbfa3959956049d76bbb09.png"},{"id":66682979,"identity":"8d38c815-8387-4ffb-a15c-8051d9bc88a9","added_by":"auto","created_at":"2024-10-15 12:36:30","extension":"png","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":103884,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of training loss curves with different loss functions\u003c/p\u003e","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/53b909862c9d2a523345896a.png"},{"id":66682980,"identity":"4afa1e32-003e-4eca-8a43-89442f4b5d3e","added_by":"auto","created_at":"2024-10-15 12:36:30","extension":"png","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":2919687,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of different loss function training visualizations\u003c/p\u003e","description":"","filename":"floatimage11.png","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/81263f6c7b3f59322d6e24ba.png"},{"id":69358337,"identity":"f44cfe03-3670-437a-89e4-7632b2c8acbe","added_by":"auto","created_at":"2024-11-19 14:02:31","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":37947496,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4890313/v1/0bd086c1-fecd-4d14-a97f-fab877c439d2.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"DST-Net: Dual Self-Integrated Transformer Network for Semi-Supervised Segmentation of Optic Disc and Optic Cup in Fundus Image","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eGlaucoma is one of the most severe causes of blindness worldwide. It is anticipated that by 2040, the number of individuals diagnosed with glaucoma will reach 110 million[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. The conventional method for localizing and segmenting retinal images relies on manual expertise and is susceptible to subjective and objective variables. This process is also time-consuming and prone to errors, particularly in detecting subtle lesions. Consequently, developing efficient automated glaucoma detection technology is crucial for large-scale screening[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Clinically, the cup-to-disc ratio (CDR) is a pivotal diagnostic and monitoring indicator for glaucoma[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Accurate segmenting of the OD and OC in the retinal image is critical. Under normal conditions, the CDR is small. However, once the CDR increases, the risk of developing glaucoma increases, requiring further examination and treatment. Deep learning methods, especially convolutional neural networks (CNNs), have been widely used in several image segmentation tasks and have demonstrated satisfying performance[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. For example, Guo et al.[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e] proposed the CAFR-CNN framework for cross-domain joint OD and OC segmentation, which consists of faster R-CNN detectors, a spatial attention-based region alignment module, a pyramid ROI alignment module, and a prototype-based semantic alignment module. Luo et al.[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] proposed a new segmentation architecture, BGA-Net, which introduces auxiliary boundary branching and adversarial learning to segment OD and OC in a multi-labeled manner jointly. Yin et al.[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] proposed a deep learning-based level set approach for OD and OC segmentation in an automated retinal diagnostic system. Nevertheless, the CNN-based methods still face challenges in modeling long-range relationships due to the inherent limitations of convolutional operations[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWith the breakthrough of Transformer architecture, semantic segmentation has ushered in a new development opportunity[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. Dosovitskiy et al.[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] proposed Vision Transformer (ViT) in 2020, which successfully extends the Transformer model to image processing tasks and realizes the integration of natural language processing and computer vision. ViT breaks through the limitation of traditional CNNs on the input image size by serializing image pixels and capturing the global features of an image using the Transformer's self-attention mechanism. It also demonstrates excellent performance on image classification tasks. It is important to note that ViT alone does not yield good semantic segmentation results because it emphasizes global information and lacks local detail extraction. In contrast, CNN focuses on extracting local information, and thus, combining ViT and CNN enables the model to leverage local and global information in images, improving its performance and making it suitable for OD and OC segmentation tasks. However, ViT typically requires a large amount of data for practical training. Besides, fundus image samples are sparse, with many unlabeled data and few labeled data, and thus, data augmentation is not a practical solution.\u003c/p\u003e \u003cp\u003eTo address the above limitations, we developed a Dual Self-integrated Transformer Network, DST-Net, which combines CNN and ViT for joint segmentation of the OD and OC using a semi-supervised approach. The main contributions of this paper are as follows:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eA dual self-integrated teacher-student network is constructed using CNN and ViT, with weight updates performed via Exponential Moving Average (EMA). Through a dual-view co-training mechanism, the network fully leverages the mutual learning capabilities of CNN and ViT to learn both global and local image information adaptively.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eAiming at the problem of unclear boundaries of the OD and OC, the boundary difference loss function BDoU-loss is employed to enhance the network's attention to the boundaries, improving the accuracy of boundary segmentation.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eA semi-supervised learning approach utilizes labeled and unlabeled data for training, in which the teacher network generates pseudo-labels to guide the student network, enhancing the model's learning process.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e"},{"header":"2 Related Work","content":"\u003cp\u003eThis section discusses the related work in CNN-based and Transformer-based methods.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 CNN-based methods\u003c/h2\u003e \u003cp\u003eWith the rapid development of deep learning technology, CNN-based models have been thoroughly studied in medical image processing. For example, Zilly et al.[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] enhanced the performance of CNN using an integrated learning technique, which employs an entropy sampling technique to identify the key information points in the image. Wang et al.[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] proposed an asymmetric segmentation network based on the U-Net model to segment the OD region, which combines the classical U-Net architecture with a unique cross-linked subnetwork to accurately localize the OD and improve sensitivity to morphological variations in ROI. Tulsani et al.[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] proposed an improved UNET\u0026thinsp;+\u0026thinsp;+\u0026thinsp;architecture for automatically segmenting the OD and OC in glaucoma assessment, in which a clinical feature-based classification model with preprocessing and customized loss functions was introduced to address the problem of vessel and category imbalance. Pachade et al.[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] proposed a novel segmentation network called NENet, which combines EfficientNetB4 as an encoder and a series of pre-activated residual blocks, atrous space pyramid pooling (ASPP) blocks, and attention gates (AGs). Meanwhile, a modified patch-based discriminator was designed for NENet to improve local segmentation details. Guo et al.[\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] proposed a joint segmentation method for OD and OC based on an improved U-Net architecture. The feature fusion module in U-Net is first added to reduce the information loss in feature extraction. Then, the channel and spatial attention mechanisms are combined to highlight the essential features related to segmentation and suppress the expression of features in irrelevant regions. Finally, multilabel loss is used to generate joint segmentation results. Experimental results show the method performs well on OD and OC segmentation tasks. In [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e], Fu et al. proposed the M-Net based on a multilabel deep network with polar coordinate transformation to segment OD and OC jointly. The model uses U-Net as the backbone network, the input layer as a pyramid model for receiving feature mappings at different scales, and the side output layer outputs the corresponding local prediction results.\u003c/p\u003e \u003cp\u003eCurrently, most CNN-based methods, such as U-Net and its variants, rely on extracting and fusing multiscale features to improve segmentation accuracy. However, the fused features still have a small \"effective receptive field\" and are mainly concentrated in localized regions of the image, thus limiting their performance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Transformer-based methods\u003c/h2\u003e \u003cp\u003eWith the emergence of Transformer, semantic segmentation entered a brand-new era[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. Chen et al.[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] proposed TransU-Net for medical image segmentation by combining Transformer and U-Net. Cao et al.[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] designed a pure Transformer similar to U-Net based on the translation window mechanism, Swin-U-Net, for medical image segmentation. Li et al.[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] proposed a Transformer-based medical image segmentation model Segtran. The model combines the Transformer's unrestricted sensory field at high feature resolution and the advantages of multiscale feature extraction using compressed attentional block specification of the Transformer's self-attention mechanism. It also learns diverse representations using extended blocks. In addition, the method employed a new positional coding scheme that imposes a continuum induction bias on the image. Experimental results show that compared to existing representative methods, Segtran achieves higher segmentation accuracy in the OD and OC segmentation tasks and demonstrates good cross-domain generalization capabilities. Yi et al.[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] proposed a deep learning model for joint OD and OC segmentation, C2FTFNet (Coarse-to-Fine Transformer Network). The model employs a coarse-to-fine strategy to automatically segment OD and OC in fundus images by gradually increasing the segmentation accuracy. Hussain et al.[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] proposed a new segmentation method called UT-Net, which exploits the advantages of the UNet and the Transformer in the coding layer and employs an attention-gated bilinear fusion scheme. In addition, multi-head contextual attention is introduced to augment the self-attention used in the traditional visual Transformer. In [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], Wu et al. proposed a novel Transformer-based model, SeATrans, to transfer segmentation knowledge to disease diagnosis networks. Specifically, an asymmetric multiscale interaction strategy is first proposed to associate each low-level diagnostic feature with a multiscale segmentation feature. Then an efficient strategy called SeA-block is employed to activate the diagnostic features through the associated segmentation features.\u003c/p\u003e \u003cp\u003eAlthough the Transformer architecture performs well in capturing global contextual information, it is inadequate in capturing local spatial information. Besides, transformer-based models usually require much data to train their self-attention mechanism for optimal performance. In addition, there is often a lack of high-quality labeled datasets in medical image segmentation, which limits the model's training effectiveness and generalization ability. Therefore, optimizing the Transformer model under limited data conditions to balance the ability to capture global and local features remains a challenge in current research.\u003c/p\u003e \u003c/div\u003e"},{"header":"3 Method","content":"\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Proposed DST-Net\u003c/h2\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe proposed DST-Net combines the strengths of CNN and Transformer through dual-view co-training and adaptively learns an image's global and local features to achieve high-accuracy segmentation (The code is publicly available at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/ky120/DST-Net\u003c/span\u003e\u003cspan address=\"https://github.com/ky120/DST-Net\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e). The network structure of DST-Net is depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. It is a semi-supervised, self-integrated segmentation network that consists of two self-integrated networks. Each self-integrated network contains a student network, as shown in the upper half of Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, and a teacher network, as shown in the lower half of Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The teacher network has the same structure as the student network. Still, its weights are updated as the EMA of the student network's weights[\u003cspan additionalcitationids=\"CR25\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]. Structurally, DST-Net builds a dual self-integration framework to learn more discriminative features. The first self-integration network consists of a U-shaped network composed of CNNs for obtaining the initial segmentation results of the OD and OC, and the second self-integration network consists of a U-shaped structured network composed of two ViTs for more accurate segmentation results. The parameters of the student network in each self-integration framework are passed to the teacher network through the EMA, which in turn updates the weights of the parameters in the teacher network.\u003c/p\u003e \u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, the network training process is divided into two parts. Firstly, the labeled source domain data is utilized for training, and the student network performs feature extraction by CNN and ViT to generate the predicted feature map. Then, it is compared with the ground truth, and the loss is calculated, which is noted as \u003cem\u003eL1\u003c/em\u003e. The teacher network also generates pseudo-labels using labeled data, compares them to the ground truth, and calculates the loss, which is \u003cem\u003eL2\u003c/em\u003e. Finally, the predicted feature map generated by the student network is compared with the pseudo-label generated by the teacher network. The loss is calculated and noted as \u003cem\u003eL3\u003c/em\u003e; the three losses are loss fused, and backpropagation is performed on the student network. The weights are passed through the EMA to update the entire network, and the loss functions are all chosen as the BDoU-loss. Secondly, after the training using labeled data is completed, the unlabeled data is fed into the teacher network to generate pseudo-labels. The loss is computed with the predicted feature maps generated by the student network, and the loss is passed to the student network for weight updating and then passed to the teacher network via EMA. To ensure that the teacher network can effectively guide the student network, the teacher network is pre-trained using ImageNet[\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]。\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.2 CNN and ViT-based Encoder-Decoder Structure\u003c/h2\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo fully utilize the feature learning capabilities of CNN and ViT, this section constructs the encoder and decoder of the self-integrated network based on the CNN and the network blocks of ViT, respectively. The four encoders and decoders are connected using U-Net skip connections (shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eFigure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ea) shows the structure of the CNN-based network, and Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ec) shows the structure of the ViT-based network. In each CNN-based block, two 3\u0026times;3 convolutional layers and two batch normalizations are used to build the structure, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eb). The ViT-based network blocks are constructed using Swin-Transformer, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ed). Unlike conventional Transformer blocks, Swin-Transformer employs layer normalization with offset windows, multi-head self-attention, residual connectivity, and Multilayer Perceptions (MLPs) with Gaussian Error Linear Units (GELUs) with a moving window design. This design forms the Windows Multi-head Self-Attention (WMSA) and Shifted Window-based Multi-head Self-Attention (SWMSA) mechanisms. WMSA is a windowed Multihead Self-Attention mechanism that handles the computation of Self-Attention within each window. SWMSA integrates Layer Normalization, Multihead Self-Attention, and Residual Connection to better capture the local features of an image. Through the size of the window and the sliding step, the Swin-Transformer moves over the data sequence and performs the computation. Precisely, a sliding window moves across the data sequence, selecting a continuous data segment as the window at each step. It analyzes or computes the data within the window. The window is then slid forward in fixed steps, and the process continues until the entire data sequence is covered. WMSA and SWMSA are applied in two consecutive Transformer blocks, respectively. In this way, the Transformer block maps the position of the input sequence \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{Z}_{0}=[{Z}_{\\text{0,1}}\\dots\\:,{Z}_{0,N}]\\)\u003c/span\u003e\u003c/span\u003e to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{Z}_{L}=[{Z}_{L,1}\\dots\\:,{Z}_{L,N}]\\)\u003c/span\u003e\u003c/span\u003e. The detailed process of feature learning of the data through the ViT network based on WMSA, SWMSA, and MLP is summarized in Eqs.\u0026nbsp;(\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e)-(\u003cspan refid=\"Equ5\" class=\"InternalRef\"\u003e5\u003c/span\u003e), where \u003cem\u003ei\u003c/em\u003e = [1, L] and L is the number of blocks.\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\:{Z}_{i-1}=WMSA\\left(LN\\left({Z}_{i-1}\\right)\\right)+{Z}_{i-1}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\:{Z}_{i}=MLP\\left(LN\\left({Z}_{i}\\right)\\right)+{Z}_{i}\\:$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$\\:{Z}_{i+1}=\\text{S}\\text{W}\\text{M}\\text{S}\\text{A}\\left(\\text{L}\\text{N}\\left({Z}_{i}\\right)\\right)+{Z}_{i}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e\n$$\\:{\\:Z}_{i+1}=\\text{M}\\text{L}\\text{P}\\left(LN\\left({Z}_{i+1}\\right)\\right)+{Z}_{i+1}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ5\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ5\" name=\"EquationSource\"\u003e\n$$\\:\\:MSA\\left({Z}^{\\text{*}}\\right)=\\text{s}\\text{o}\\text{f}\\text{t}\\text{m}\\text{a}\\text{x}\\left(\\frac{QK}{\\sqrt{D}}\\right)V$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e5\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\:Q,K,V\\in\\:{R}^{{M}^{2}\\times\\:d}\\)\u003c/span\u003e\u003c/span\u003e, M\u003csup\u003e2\u003c/sup\u003e denotes the number of image blocks in a window, and \u003cem\u003ed\u003c/em\u003e represents the dimension of the query and key.\u003c/p\u003e \u003cp\u003eThe self-attention mechanism consists of three point-wise linear layers that map the input tokens into intermediate representations. It then computes the significance of each element in the sequence relative to every other element. In this way, a query Q, key K, and value V are generated for each component. Then, a weighted summation is performed to obtain a self-attention representation for each element. This self-attention representation determines how much attention each element pays to the other elements. Subsequently, these self-attention representations undergo operations such as normalization and residual connections before being passed into a feedforward neural network. In the feedforward network, each element undergoes point-wise linear transformations and is enhanced through nonlinear transformations to improve its features. Finally, the output from the feedforward network is combined with the initial input through another residual connection, forming a transformed sequence that represents the output of the Transformer module. This process is repeated across multiple Transformer modules to capture hierarchical representations and dependencies within the input sequence.\u003c/p\u003e \u003cp\u003eUnlike the basic blocks of CNNs, ViT is designed with merge and expansion layers between the encoder or decoder of each base block[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] instead of the traditional subsampling and upsampling steps. This design allows information to flow more freely between layers, preserving more details and facilitating the better capture of complex relationships in the input sequence. The Merge Layer is designed to halve the number of tokens and double the feature dimensions. It divides the input patch into four parts, then joins them together, and finally applies a linear layer to unify the dimensions by a factor of two. The expansion layer is designed to resize the input feature mapping to twice its original size and reduce the feature dimension to half the dimension of the input feature mapping. It uses a linear layer to increase the feature dimension, then employs a rearrangement operation to enlarge the size and reduce the feature dimension to a quarter of the input dimension. The size of the feature mapping in each step is briefly illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ec), where W, H, and C represent the width, height, and channel dimensions of the feature map, respectively. The patch size is set to 4, the input channel is 3, the embedding dimension is 96, and the number of self-attention heads is 3, 6, 12, and 24, respectively, with a window size of 7 for each encoder and decoder. This design enables the ViT to capture the features and relationships within the input sequence more effectively while avoiding the subsampling and upsampling steps in traditional CNNs, thus improving model performance and efficiency.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Exponential Moving Average (EMA)\u003c/h2\u003e \u003cp\u003eEMA is commonly used to smooth time series data. The principle is to perform a weighted average of the series data, giving more weight to recent data points and less to earlier data points. It effectively smoothes the time series data, making it more continuous and stable. In deep learning, EMA is commonly used to smooth the update of model parameters. Specifically, EMA is applied to the model parameters each time they are updated, thus reducing the fluctuation of each update and making the model more stable. The formula for EMA is shown in Eq.\u0026nbsp;(\u003cspan refid=\"Equ6\" class=\"InternalRef\"\u003e6\u003c/span\u003e):\u003cdiv id=\"Equ6\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ6\" name=\"EquationSource\"\u003e\n$$\\:EMA\\left[t\\right]={\\alpha\\:}^{\\text{*}}x\\left[t\\right]+(1-\\alpha\\:{)}^{\\text{*}}EMA[t-1]$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e6\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u003cem\u003et\u003c/em\u003e represents the time step, \u003cem\u003ex\u003c/em\u003e(\u003cem\u003et\u003c/em\u003e) denotes the original data at the \u003cem\u003et\u003c/em\u003e-th time point, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\alpha\\:\\)\u003c/span\u003e\u003c/span\u003e is the smoothing factor, which usually takes a value between 0 and 1, indicating the weight of the current sample, (1-\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\alpha\\:\\)\u003c/span\u003e\u003c/span\u003e) represents the weight of the historical data, and EMA[\u003cem\u003et\u003c/em\u003e-1] is the EMA value of the previous time point.\u003c/p\u003e \u003cp\u003eThe EMA method is employed in the teacher-student network to transfer weights and enhance the model's generalization capability. The core idea of this method is to introduce an EMA to integrate the model's predictions at different time steps. EMA assigns greater weight to recent predictions, allowing the model to focus on the most current information. During the weight transfer process, the student network performs forward propagation to compute the EMA weights, which are then applied to the corresponding layers of the teacher network. In this way, the parameters of the teacher network are updated by exponentially weighted moving averages during training, making the teacher network more robust and guiding the student network to use pseudo-labeling for learning in the context of consistency concerns.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Boundary Difference over Union Loss (BDoU-Loss)\u003c/h2\u003e \u003cp\u003eMost of the current image segmentation algorithms uniquely label the image according to specific rules, which are affected by the geometric constraints of the view. Especially in the fundus image of a lesion, since the captured fundus image is a two-dimensional (2D) planar view while the OC is a three-dimensional (3D) anatomical structure, the OD and OC region presents a circular 2D planar projection in the fundus image, which prevents the accurate representation of its true 3D shape. This leads to an uneven distribution of pixels between the OC and the background region, which reduces the boundary segmentation accuracy of the OC. The comparison between the 3D anatomical structure and the 2D planar of the OD and OC in the fundus image is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eCurrent loss functions for medical image segmentation primarily focus on the overall segmentation results, and less loss is proposed for the guidance of boundary segmentation. We employ the BDoU-Loss[\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e], abbreviated as \u003cem\u003eL\u003c/em\u003e\u003csub\u003eBD\u003c/sub\u003e, to address this issue to enhance boundary segmentation. \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eBD\u003c/em\u003e\u003c/sub\u003e calculates the error region near the boundary by computing the set difference between the ground truth and prediction. The error region is then reduced by minimizing the difference ratio set to the partial intersection or union. The task of joint segmentation of OD and OC is a multilabel problem, where a specific pixel can belong to multiple categories. \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eBD\u003c/em\u003e\u003c/sub\u003e improves the attention to the region near the boundary and effectively addresses the challenges of insufficient attention to boundary segmentation and uneven categorization. The schematic diagram of the \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eBD\u003c/em\u003e\u003c/sub\u003e calculation is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFigure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e shows the structure of the principle of BDoU-loss and the green line region on the right represents the area of the union minus the intersection of the prediction and ground truth. Below, a hyperparameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\alpha\\:\\)\u003c/span\u003e\u003c/span\u003e controls this union minus the intersection area. In this way, the boundary of different images can be adjusted to better guide the boundary segmentation, thus increasing the accuracy of OD and OC segmentation. The principle is illustrated in Eq.\u0026nbsp;(\u003cspan refid=\"Equ7\" class=\"InternalRef\"\u003e7\u003c/span\u003e).\u003cdiv id=\"Equ7\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ7\" name=\"EquationSource\"\u003e\n$$\\:{L}_{BD}=\\frac{G\\cup\\:P-G\\cap\\:P}{G\\cup\\:P-{\\alpha\\:}^{\\text{*}}G\\cap\\:P}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e7\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\alpha\\:\\)\u003c/span\u003e\u003c/span\u003e is a hyperparameter that controls the influence of the partial joint region. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\alpha\\:\\)\u003c/span\u003e\u003c/span\u003e is calculated as shown in Eq.\u0026nbsp;(\u003cspan refid=\"Equ8\" class=\"InternalRef\"\u003e8\u003c/span\u003e).\u003cdiv id=\"Equ8\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ8\" name=\"EquationSource\"\u003e\n$$\\:\\alpha\\:=1-2\\times\\:\\frac{C}{S},\\alpha\\:\\in\\:\\left[\\text{0,1}\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e8\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u003cem\u003eC\u003c/em\u003e is the boundary length, and \u003cem\u003eS\u003c/em\u003e is the target size.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Experiments","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Dataset\u003c/h2\u003e \u003cp\u003eThe proposed network is evaluated using the RIGA\u0026thinsp;+\u0026thinsp;dataset. RIGA+ [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] is a composite dataset comprising five subsets: Binrush, Magrabia, BASE1, BASE2, and BASE3. Magrabia and Binrush serve as the source domain, while BASE1, BASE2, and BASE3 are used as the target domain 1, target domain 2, and target domain 3. Binrush and Magrabia have 195 and 95 labeled fundus retinal images for semi-supervised training, respectively. Labeled and unlabeled retinal fundus images are present in BASE1, BASE2, and BASE3, where unlabeled images are used for semi-supervised training and labeled images are used to test the segmentation performance of the model. BASE1 consists of 227 unlabeled images and 35 labeled images, BASE2 includes 238 unlabeled images and 30 labeled images, and BASE3 comprises 252 unlabeled images and 27 labeled images. In RIGA+, all images are cropped to an 800 \u0026times; 800 OD region. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e details the RIGA\u0026thinsp;+\u0026thinsp;dataset information, and Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e shows the fundus images of the source and target domains.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eIntroduction to the RIGA\u0026thinsp;+\u0026thinsp;dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"+\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDomain\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDataset Names\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNumbers (Training\u0026thinsp;+\u0026thinsp;Test)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eImage Size\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSource\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBinRushed\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"+\" colname=\"c3\"\u003e \u003cp\u003e195(195\u0026thinsp;+\u0026thinsp;0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e800\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times\\)\u003c/span\u003e\u003c/span\u003e800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSource\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMagrabia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"+\" colname=\"c3\"\u003e \u003cp\u003e95(95\u0026thinsp;+\u0026thinsp;0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e800\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times\\)\u003c/span\u003e\u003c/span\u003e800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTarget 1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMESSIDOR-BASE1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"+\" colname=\"c3\"\u003e \u003cp\u003e173(138\u0026thinsp;+\u0026thinsp;35)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e800\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times\\)\u003c/span\u003e\u003c/span\u003e800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTarget 2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMESSIDOR-BASE2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"+\" colname=\"c3\"\u003e \u003cp\u003e148(118\u0026thinsp;+\u0026thinsp;30)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e800\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times\\)\u003c/span\u003e\u003c/span\u003e800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTarget 3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMESSIDOR-BASE3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"+\" colname=\"c3\"\u003e \u003cp\u003e133(106\u0026thinsp;+\u0026thinsp;27)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e800\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times\\)\u003c/span\u003e\u003c/span\u003e800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Evaluation Metrics\u003c/h2\u003e \u003cp\u003eDice and Hausdorff Distance are used as OD and OC segmentation evaluation metrics. The dice coefficient is employed as an evaluation metric to compare the segmentation results of different methods. Higher coefficient values indicate better segmentation results for both OD and OC. The expression of Dice is shown in Eq.\u0026nbsp;(9). In most datasets, classic methods yield Dice values for OD segmentation in the range of 70\u0026ndash;80% and for OC segmentation in the range of 65\u0026ndash;70%. Improving the Dice values for segmentation is crucial for future OD and OC segmentation research.\u003c/p\u003e \u003cp\u003e \u003cem\u003eDice\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:=\\frac{2\\times\\:TP}{2\\times\\:TP+FP+FN}\\)\u003c/span\u003e\u003c/span\u003e (9)\u003c/p\u003e \u003cp\u003eHausdorff Distance is a metric used to determine the degree of resemblance between two point sets. It is beneficial in comparing images or evaluating the similarity between image segmentation results. Given two point sets \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:A=\\{a1,\\dots\\:,ap\\},B=\\{b1,\\dots\\:,bq\\}\\)\u003c/span\u003e\u003c/span\u003e, the Hausdorff Distance H(A, B) between these two sets is defined as shown in Eqs.\u0026nbsp;(\u003cspan refid=\"Equ9\" class=\"InternalRef\"\u003e10\u003c/span\u003e)-(\u003cspan refid=\"Equ11\" class=\"InternalRef\"\u003e12\u003c/span\u003e):\u003cdiv id=\"Equ9\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ9\" name=\"EquationSource\"\u003e\n$$\\:H(A,B)=\\text{m}\\text{a}\\text{x}\\left(h\\right(A,B),h(B,A\\left)\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e10\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ10\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ10\" name=\"EquationSource\"\u003e\n$$\\:h(A,B)=\\underset{a\\in\\:A}{\\text{m}\\text{a}\\text{x}}\\:\\left\\{\\underset{b\\in\\:B}{\\text{m}\\text{i}\\text{n}}\\parallel\\:a-b\\parallel\\:\\right\\}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e11\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ11\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ11\" name=\"EquationSource\"\u003e\n$$\\:h(B,A)=\\underset{b\\in\\:B}{\\text{m}\\text{a}\\text{x}}\\:\\left\\{\\underset{a\\in\\:A}{\\text{m}\\text{i}\\text{n}}\\parallel\\:b-a\\parallel\\:\\right\\}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e12\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cem\u003eh\u003c/em\u003e(\u003cem\u003eA\u003c/em\u003e, \u003cem\u003eB\u003c/em\u003e) and \u003cem\u003eh\u003c/em\u003e(\u003cem\u003eB\u003c/em\u003e, \u003cem\u003eA\u003c/em\u003e) are the directed Hausdorff distance from A to B and B to A, respectively.\u003c/p\u003e \u003cp\u003eIn image segmentation, a variant of the Hausdorff distance, Hausdorff Distance at 95th percentile (HD95), is often used to assess the quality of the segmentation results, measuring the distance between two sets. It measures the discrepancy between the segmentation result and the ground truth by calculating the distances between all point pairs in the two sets and finding the 95th percentile of these distances. HD95 is commonly used to assess the accuracy of segmentation results, where a smaller value indicates a higher degree of agreement with the ground truth and, consequently, better segmentation quality. The Hausdorff Distance is highly sensitive to outliers, but HD95 mitigates this sensitivity by using the percentile of the distances, providing a more robust metric.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Implementation details\u003c/h2\u003e \u003cdiv id=\"Sec14\" class=\"Section3\"\u003e \u003ch2\u003e4.3.1 Experimental Setup\u003c/h2\u003e \u003cp\u003eWe implemented the experiment using Ubuntu 18.04, Pytorch version 1.7, Cuda version 11.4, and Python version 3.7 throughout the OD and OC segmentation comparison, ablation, and generalization experiments. The network training hyperparameters are listed in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eNetwork training hyperparameters\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eParameter name\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValue\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEpoch\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e200\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBatch Size\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e16\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLearning Rate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.005\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOptimizer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdam\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWeight Decay\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section3\"\u003e \u003ch2\u003e4.3.2 Image Preprocessing\u003c/h2\u003e \u003cp\u003eThe overall preprocessing process is divided into two stages. The first stage operates on a single fundus image, including grayscale conversion, standardization, Contrast Limited Adaptive Histogram Equalization (CLAHE), and gamma correction. The objective is to enhance the image's contrast, reduce the network training burden, and accelerate convergence. The second stage focuses on data augmentation, including random cropping, image rigid transformation, and elastic transformation, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eIn the first stage, to speed up the convergence of the training network and reduce the network training burden, three-channel color images are converted into single-channel grayscale images. The grayscale images are standardized to improve the model's generalization ability and achieve the unity of the metrics. The CLAHE algorithm is used to enhance the local contrast. Finally, a nonlinear Gamma correction is applied to adjust the light intensity of the input retinal image, performing nonlinear operations on the intensity values to establish an exponential relationship between the input and output image intensities.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAfter the preprocessing stage, the OD is localized and cropped to enlarge the area occupied by the OD in preparation for subsequent data augmentation and OD and OC segmentation. To enhance the limited medical image dataset, data augmentation is an effective strategy to mitigate insufficient training data and reduce overfitting. Various rigid and elastic transformations are applied to the images in the dataset, including scaling, rotation, mirroring, and B-spline elastic deformation. Finally, the dataset images are randomly cropped, and the cropped image blocks of 256\u0026times;256 are then used for network training.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Ablation Analysis\u003c/h2\u003e \u003cp\u003eWe conducted ablation experiments to validate the effectiveness and contribution of each module within the proposed network for OD and OC segmentation. For the teacher-student network structure, various combinations of CNN and ViT were tested to evaluate the effectiveness of their integration in the model, with the ablation results presented in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eResults of ablation experiments on the BASE1 dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eDice\u003csub\u003eDisc\u003c/sub\u003e (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eDice\u003csub\u003eCup\u003c/sub\u003e (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStudent\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTeacher\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e76.32\u0026thinsp;\u0026plusmn;\u0026thinsp;0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e61.35\u0026thinsp;\u0026plusmn;\u0026thinsp;1.01\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eViT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eViT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e76.21\u0026thinsp;\u0026plusmn;\u0026thinsp;1.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e63.81\u0026thinsp;\u0026plusmn;\u0026thinsp;1.56\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCNN\u0026thinsp;+\u0026thinsp;CNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCNN\u0026thinsp;+\u0026thinsp;CNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e82.65\u0026thinsp;\u0026plusmn;\u0026thinsp;0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e73.07\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eViT\u0026thinsp;+\u0026thinsp;ViT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eViT\u0026thinsp;+\u0026thinsp;ViT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e80.76\u0026thinsp;\u0026plusmn;\u0026thinsp;0.65\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e70.58\u0026thinsp;\u0026plusmn;\u0026thinsp;0.49\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eViT\u0026thinsp;+\u0026thinsp;CNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eViT\u0026thinsp;+\u0026thinsp;CNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e92.60\u0026thinsp;\u0026plusmn;\u0026thinsp;0.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e82.62\u0026thinsp;\u0026plusmn;\u0026thinsp;0.19\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCNN\u0026thinsp;+\u0026thinsp;ViT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCNN\u0026thinsp;+\u0026thinsp;ViT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e95.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.14\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e85.69\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows that the combination of CNN and ViT achieves the best OD and OC segmentation performance, with Dice scores of 95.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.14 for the OD and 85.69\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27 for OC. In contrast, the combinations of ViT\u0026thinsp;+\u0026thinsp;ViT and CNN\u0026thinsp;+\u0026thinsp;CNN result in the poorest segmentation performance. The superior performance of CNN\u0026thinsp;+\u0026thinsp;ViT can be attributed to the strengths of both components: CNN's spatial locality allows for the effective capture of local features, and its translation invariance helps the model handle variations in the positions of the OD and OC within the image. Meanwhile, ViT's global awareness facilitates the establishment of connections across the entire image, making it well-suited for capturing the global relationship and overall structural features between OD and OC. The ViT\u0026thinsp;+\u0026thinsp;ViT and CNN\u0026thinsp;+\u0026thinsp;CNN combinations perform poorly due to information redundancy. Using two CNNs can result in redundant extraction of similar features, while two ViTs may lead to redundancy in global perception. The ViT\u0026thinsp;+\u0026thinsp;CNN combination may underperform if the feature extraction in the ViT stage is not robust enough, limiting the CNN stage's ability to classify with rich features, thus constraining the model's performance. Also, the Transformer network may lose some spatial information when processing images because it does not explicitly consider spatial localization as CNN does, which may impact segmentation tasks such as OD and OC that require spatial information.\u003c/p\u003e \u003cp\u003eIn summary, the combination of using CNN to perform local feature extraction first and then establishing global association through ViT allows the model to maintain spatial information while being able to globally associate and understand semantic features in different regions of the image, improving the flexibility and versatility of the model and maximizing the advantages of both. The visualization results of the ablation experiments are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e, and the visualization comparison map can more intuitively demonstrate the segmentation differences between different combinations. To better illustrate the segmentation results of varying module combinations, the segmentation of the OD and OC is extracted and shown in green. In contrast, the ground truth is shown in red. By overlaying these two colors, the resulting visualization indicates segmentation quality. In this way, the segmentation effect can be visualized.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFrom Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e, it can be seen that the CNN\u0026thinsp;+\u0026thinsp;CNN combination has obvious over-segmentation and under-segmentation when processing the OC region. For the segmentation of the OD, the over-segmentation is particularly prominent. This is because the combination pays too much attention to local feature extraction, resulting in strong segmentation ability in the local region and many over-segmentation phenomena due to the weak performance in the segmentation judgment of the boundary part. This indicates that relying solely on local feature extraction can lead to over-extraction of features, thereby degrading segmentation performance. The combination of ViT\u0026thinsp;+\u0026thinsp;ViT performs well in overall control but is deficient in detailed feature extraction. As a result, this combination tends to experience under-segmentation issues in OD and OC segmentation. The significant presence of red areas in the figure indicates that under-segmentation is a prevalent problem. Secondly, the ViT\u0026thinsp;+\u0026thinsp;CNN combination has certain deficiencies in feature extraction. Due to the poor effect of feature extraction in the ViT stage, the impact of feature extraction in the CNN stage is also affected. As can be seen from the figure, although there is an improvement compared to the first two combinations, there are still more under-segmentation cases. Finally, the CNN\u0026thinsp;+\u0026thinsp;ViT combination demonstrates excellent performance. After feature extraction by CNN, the model obtains rich local feature information, while ViT excels in overall detail management. This combination effectively combines the local and global information, significantly improving the model's over-segmentation and under-segmentation. As can be seen from the figure, the red and green regions in OD and OC regions are the least in all comparisons.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section2\"\u003e \u003ch2\u003e4.5 Quantitative Comparison with SOTA Methods\u003c/h2\u003e \u003cp\u003eIn this section, the proposed network is compared in detail with SOTA methods. Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e details the experimental results of the various methods on the BASE1, BASE2, and BASE3 datasets. The evaluation metric used is the mean Dice score, with the standard deviation indicated and the best results highlighted in bold.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePerformance of methods on the BASE1, BASE2, and BASE3 dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eMethods\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eBASE1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003eBASE2\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003eBASE3\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDice\u003csub\u003eDisc\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDice\u003csub\u003eCup\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDice\u003csub\u003eDisc\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eDice\u003csub\u003eCup\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eDice\u003csub\u003eDisc\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eDice\u003csub\u003eCup\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdaEnt[32]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e94.46\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e82.71\u0026thinsp;\u0026plusmn;\u0026thinsp;0.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e92.77\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e77.79\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e93.72\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e81.87\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdaMI[33]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e94.50\u0026thinsp;\u0026plusmn;\u0026thinsp;0.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e82.80\u0026thinsp;\u0026plusmn;\u0026thinsp;0.19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e92.72\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e78.86\u0026thinsp;\u0026plusmn;\u0026thinsp;0.19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e93.65\u0026thinsp;\u0026plusmn;\u0026thinsp;0.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e82.71\u0026thinsp;\u0026plusmn;\u0026thinsp;0.11\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFSM[34]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e94.96\u0026thinsp;\u0026plusmn;\u0026thinsp;0.63\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e84.30\u0026thinsp;\u0026plusmn;\u0026thinsp;1.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e93.10\u0026thinsp;\u0026plusmn;\u0026thinsp;0.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e81.39\u0026thinsp;\u0026plusmn;\u0026thinsp;0.91\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e94.41\u0026thinsp;\u0026plusmn;\u0026thinsp;0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e83.21\u0026thinsp;\u0026plusmn;\u0026thinsp;1.92\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003epOSAL[30]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e94.01\u0026thinsp;\u0026plusmn;\u0026thinsp;0.23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e83.37\u0026thinsp;\u0026plusmn;\u0026thinsp;0.09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e95.09\u0026thinsp;\u0026plusmn;\u0026thinsp;0.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e84.28\u0026thinsp;\u0026plusmn;\u0026thinsp;0.17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e94.77\u0026thinsp;\u0026plusmn;\u0026thinsp;0.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e84.06\u0026thinsp;\u0026plusmn;\u0026thinsp;0.15\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBEAL[31]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e95.31\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e85.44\u0026thinsp;\u0026plusmn;\u0026thinsp;0.21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e95.57\u0026thinsp;\u0026plusmn;\u0026thinsp;0.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e83.18\u0026thinsp;\u0026plusmn;\u0026thinsp;0.25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e94.79\u0026thinsp;\u0026plusmn;\u0026thinsp;0.19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e83.98\u0026thinsp;\u0026plusmn;\u0026thinsp;0.22\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eProFSDA[35]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e95.29\u0026thinsp;\u0026plusmn;\u0026thinsp;0.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e85.61\u0026thinsp;\u0026plusmn;\u0026thinsp;0.24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e94.71\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e85.33\u0026thinsp;\u0026plusmn;\u0026thinsp;0.08\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e95.47\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e85.53\u0026thinsp;\u0026plusmn;\u0026thinsp;0.15\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHPFG[36]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e93.11\u0026thinsp;\u0026plusmn;\u0026thinsp;0.16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e84.43\u0026thinsp;\u0026plusmn;\u0026thinsp;0.21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e93.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e83.88\u0026thinsp;\u0026plusmn;\u0026thinsp;0.31\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e93.23\u0026thinsp;\u0026plusmn;\u0026thinsp;0.23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e84.01\u0026thinsp;\u0026plusmn;\u0026thinsp;0.21\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDST-Net\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e95.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e85.69\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e95.97\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.31\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e85.37\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e95.49\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e85.62\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.11\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAs shown in Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e, DST-Net performs best segmentation on the BASE2 and BASE3 datasets. However, in the BASE1 dataset, the OD Dice score is slightly lower than that of the BEAL network due to the presence of images with lesions. The BEAL network generates more precise boundaries and reduces high-uncertainty predictions in the segmentation of the OD and OC by encouraging similarity between boundary predictions and mask probability entropy maps (uncertainty maps) in the target and source domains. This approach provides an advantage in OD boundary segmentation and enhances accuracy in identifying pathological regions.\u003c/p\u003e \u003cp\u003eFigure\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e displays the segmentation results of several networks on the BASE1, BASE2, and BASE3 datasets. It can be observed that DST-Net performs well in segmenting both OD and OC. BEAL performs better than pOSAL for OD segmentation, showing its advantage in boundary segmentation, especially in cases where the OD has a more regular boundary. However, for the case of irregular boundaries such as the OC, the BEAL model has more obvious under-segmentation. In contrast, DST-Net achieves better segmentation of irregular boundaries by leveraging transfer learning through the pre-training of the teacher network, allowing it to better adapt to the OD and OC segmentation tasks. With the cooperation of CNN and ViT, DST-Net has significantly improved its segmentation effect in dealing with irregular boundaries.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo more intuitively demonstrate the proposed network's focus on different regions of fundus images, heatmaps were generated to illustrate the network's attention across various areas, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eIt can be observed from Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e that the blue region indicates that the network pays less attention to the region, while the red region indicates that the network pays more attention to the region. As each layer's weights accumulate, the attention to the OD and OC regions gradually increases, indicating that the segmentation network is more focused on these areas, resulting in improved performance. In the heatmap of the BASE2 dataset, the OD region is predominantly covered by red, showing relatively good segmentation. However, some black shadowed areas are at the bottom, probably due to the interference of blood vessels in the early stages of training, attracting the model's attention. In the heatmap of the BASE3 dataset, the focus is mainly on the OD and OC regions, with other areas showing a deep blue, indicating that the segmentation of the OD and OC has been successful.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eDespite some black shadow regions affecting the segmentation performance, the model shows great competitiveness overall. This finding provides direction for subsequent optimization and improvement for challenging regions. Meanwhile, the combined analysis with the heatmap offers a clearer understanding of the regions of interest of the network for the fundus image, which will help to further optimize the network structure and parameters for more accurate OD and OC segmentation.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec18\" class=\"Section2\"\u003e \u003ch2\u003e4.6 Loss Function Analysis on the RIGA\u0026thinsp;+\u0026thinsp;Dataset\u003c/h2\u003e \u003cp\u003eTo verify the effectiveness of the boundary difference loss function selected in this paper. This section will use four loss functions, IoU-loss, Dice-loss, MSE-loss, and BDoU-loss, to conduct comparative experiments on the BASE1 dataset of RIGA+. The Dice similarity coefficient is used to measure the effectiveness of the boundary differential loss function, and the HD95 is used to evaluate boundary attention. The experimental results are shown in Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparative experiments of different loss functions\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLoss Function\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDice\u003csub\u003eDisc\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDice\u003csub\u003eCup\u003c/sub\u003e(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eHD95\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eIoU-loss\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e94.76\u0026thinsp;\u0026plusmn;\u0026thinsp;0.78\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e83.59\u0026thinsp;\u0026plusmn;\u0026thinsp;0.66\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e14\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDice-loss\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e95.44\u0026thinsp;\u0026plusmn;\u0026thinsp;0.55\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e83.94\u0026thinsp;\u0026plusmn;\u0026thinsp;0.63\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e17\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMSE-loss\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e87.14\u0026thinsp;\u0026plusmn;\u0026thinsp;0.25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e76.90\u0026thinsp;\u0026plusmn;\u0026thinsp;0.23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e19\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBDoU-loss\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e \u003cp\u003e95.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e85.69\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e11\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAs can be seen in Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e in the comparison of the metrics, the MSE-loss performs the worst across various metrics. Its principle involves penalizing model errors by calculating the squared error between prediction and the ground truth, making it highly sensitive to outliers. Furthermore, the MSE loss is less effective in dealing with classification problems, contributing to poor segmentation. Compared with IoU-loss, BDoU-loss achieves higher Dice values for both OD and Dice. The main reason is that for triple classification problems like OD and OC segmentation, the BDoU-loss pays more attention to capturing the detailed information of the segmentation boundary, especially for the OD boundary and the OD and OC boundary, thereby improving the model's segmentation accuracy. In terms of OD segmentation, BDoU-loss performs slightly worse than Dice-loss. However, it achieves a higher average Dice value overall. Dice-Loss usually has better sensitivity to boundary prediction and thus has higher accuracy in segmentation tasks with regular boundaries, which is better reflected in OD segmentation. Nevertheless, due to the low contrast of the OD and OC boundary and the relatively irregular boundary of the OC, boundary incompleteness might occur, leading to slightly inferior segmentation performance.\u003c/p\u003e \u003cp\u003eIn contrast, BDoU-loss is made more tolerant to some slight boundary incompleteness by introducing a limiting factor, which mitigates the negative impact of these incompleteness and irregular shapes on the segmentation of the OD and OC, resulting in better segmentation of the OC region. Overall, BDoU-loss demonstrates the best comprehensive performance among the compared loss functions, yielding the most optimal segmentation results. The superiority of BDoU-loss is further illustrated by the variation in loss values during the training process with different loss functions, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e, the loss curve of BDoU-loss during the training process is smoother, with more minor training fluctuations. This indicates that BDoU-loss provides superior stability against outliers or noise during training. In datasets with noise or anomalies, BDoU-loss helps the model better adapt to these conditions. A comparison of training speeds reveals that BDoU-loss tends to converge more quickly to local minima during training due to the introduction of a constraint factor. This means it has a faster convergence speed, reducing training time and improving efficiency.\u003c/p\u003e \u003cp\u003eTo show the segmentation differences between different loss functions more intuitively, the segmentation results of the ground truth and different loss functions are drawn on the original graph with blue lines for visualization, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eSince the boundaries of the vast majority of the OD and OC segmentation results are continuous, the segmentation results can be observed more intuitively by extracting the boundaries of the segmentation results of different loss functions and overlaying them into the original image, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e. From the visualization comparison. It can be seen that the Dice-loss and BDoU-loss yield almost identical segmentation results for the OD. However, BDoU-loss shows better performance for OC segmentation, which is closer to the ground truth. IoU-loss performs poorly for the OC segmentation and introduces many irregular areas in the OD segmentation. This is because IoU-loss is more sensitive to minor deviations in boundary predictions, and such sensitivity might penalize the model when slight irregularities appear in predicted boundaries, resulting in inferior performance compared to Dice-loss and BDoU-loss, which shows the importance of boundary segmentation for OD and OC segmentation.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eMSE-loss is less effective for both OD and OC segmentation. The reason is that it does not consider the relative positional relationship between pixels and thus may not be able to handle the boundary ambiguity in the segmentation task effectively. If there is an imbalance in pixel distribution between the OD and OC, such as when one category has significantly more pixels than the other, the MSE loss function may guide the model to favor the category with a higher number, resulting in the model being more inclined to learn the category with a higher number while ignoring the one with a lower number.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec19\" class=\"Section2\"\u003e \u003ch2\u003e4.7 Parameters and FLOPs\u003c/h2\u003e \u003cp\u003eWe compare our DST-Net with SOTA. Table\u0026nbsp;\u003cspan refid=\"Tab6\" class=\"InternalRef\"\u003e6\u003c/span\u003e lists the Giga Floating-point Operations Per Second (GFLOPs) and the Dice scores for OD and OC for the different methods on the BASE2 dataset. In general, the number of parameters of a network is proportional to its computational complexity, and smaller parameters tend to degrade the network's performance. Table\u0026nbsp;\u003cspan refid=\"Tab6\" class=\"InternalRef\"\u003e6\u003c/span\u003e shows that the proposed DST-Net performs best regarding OD and OC segmentation accuracy compared to the listed methods. The main reason is that the teacher model is trained by semi-supervised learning, and the generated pseudo-labels are used to train the student model, which improves the model's performance and reduces the need for expensive expert labeling.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab6\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 6\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTraining results for each model trained on the BASE2 dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eMethod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eParameters\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eGFLOPs\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c6\" namest=\"c4\"\u003e \u003cp\u003eBASE2\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003eDice\u003csub\u003eDisc\u003c/sub\u003e (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eDice\u003csub\u003eCup\u003c/sub\u003e (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdaEnt[32]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e40774656\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e3.621\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e92.77\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e77.79\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFSM[34]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e34273776\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.736\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e93.10\u0026thinsp;\u0026plusmn;\u0026thinsp;0.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e81.39\u0026thinsp;\u0026plusmn;\u0026thinsp;0.91\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003epOSAL[30]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e53118960\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.608\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e95.09\u0026thinsp;\u0026plusmn;\u0026thinsp;0.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e84.28\u0026thinsp;\u0026plusmn;\u0026thinsp;0.17\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBEAL[31]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e63247099\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6.410\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e95.57\u0026thinsp;\u0026plusmn;\u0026thinsp;0.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e83.18\u0026thinsp;\u0026plusmn;\u0026thinsp;0.25\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eProFSDA[35]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e42513479\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.531\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e94.71\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e85.33\u0026thinsp;\u0026plusmn;\u0026thinsp;0.08\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHPFG[36]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e140641904\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12.528\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e93.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e83.88\u0026thinsp;\u0026plusmn;\u0026thinsp;0.31\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eProposed DST-Net\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e81672536\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8.26\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e95.97\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.31\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e\u003cb\u003e85.37\u003c/b\u003e\u0026thinsp;\u0026plusmn;\u0026thinsp;0.23\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn addition, the proposed method is the second to last in terms of the number of parameters and GLOPs. This is mainly because Transformer methods typically have more parameters than multiscale CNN methods regarding parameter comparison. However, the proposed DST-Net comprises both CNN and Transformer modules, only using high-level features extracted by CNNs for global feature modeling in the Transformer, which can significantly reduce computational load to a certain extent.\u003c/p\u003e \u003c/div\u003e"},{"header":"5 Conclusions","content":"\u003cp\u003eIn this paper, we proposed a Dual Self-Integrated Transformer Network (DST-Net) for semi-supervised segmentation of OD and OC. The proposed DST-Net fully leverages the characteristics of both ViT and CNN networks to balance local and global information. A semi-supervised approach effectively utilizes large amounts of unlabeled data for network training. Aiming at the problem that the real 3D morphology of the OD and OC cannot be displayed in the 2D fundus images, which leads to uneven pixel distribution and unclear boundaries, we introduce BDoU-loss. This loss function focuses on boundary segmentation, improving accuracy at the irregular boundaries between the OD and OC. Ablation experiments verify the effectiveness of the CNN and ViT combination.\u003c/p\u003e \u003cp\u003eIn comparison experiments, DST-Net demonstrated superior performance on the BASE1, BASE2, and BASE3 datasets, with OD Dice scores of 0.9512, 0.9597, and 0.9549, and OC Dice scores of 0.8569, 0.8537, and 0.8562, respectively. The analysis of loss functions confirms the effectiveness of the BDoU-loss selected in this paper. Future research could further optimize the network structure of DST-Net to reduce the number of parameters while maintaining the segmentation accuracy to enhance model performance and optimize computational efficiency further.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e \u003ch2\u003eAdditional Information\u003c/h2\u003e \u003cp\u003eThe authors declare that they have no conflict of interest.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eJing Wang: Writing\u0026ndash;original draft, Validation. Jinke Wang: Conceptualization, Methodology. Zhipeng Yang: Investigation, Visualization. Min Xu: Data enhancement. Shinichi Tamura: Writing \u0026ndash; review \u0026amp; editing.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe datasets used in this paper are publicly available as follows: https://deepblue.lib.umich.edu/data/concern/data_sets/3b591905z, and https://www.adcis.net/en/third-party/messidor2/\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eY.-C. Tham, X. Li, T.Y. Wong, H.A. Quigley, T. Aung, C.-Y. Cheng, Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis, Ophthalmology 121 (2014) 2081\u0026ndash;2090.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThompson A C, Jammal A A, Medeiros F A. A Review of Deep Learning for Screening, Diagnosis, and Detection of Glaucoma Progression[J]. Translational Vision Science \u0026amp; Technology, 2020, 9(2): 42\u0026ndash;42.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao R, Chen X, Liu X, et al. Direct cup-to-disc ratio estimation for glaucoma screening via semi-supervised learning[J]. IEEE journal of biomedical and health informatics, 2019, 24(4): 1104\u0026ndash;1113.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJia C, Shi F, Zhao M, et al. Semantic segmentation with light field imaging and convolutional neural networks[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHassan T, Hassan B, Akram M U, et al. Incremental cross-domain adaptation for robust retinopathy screening via Bayesian deep learning[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuo Y, Peng Y, Zhang B. CAFR-CNN: coarse-to-fine adaptive faster R-CNN for cross-domain joint optic disc and cup segmentation[J]. Applied Intelligence, 2021, 51: 5701\u0026ndash;5725.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLuo L, Xue D, Pan F, et al. Joint optic disc and optic cup segmentation based on boundary prior and adversarial learning[J]. International Journal of Computer Assisted Radiology and Surgery, 2021, 16(6): 905\u0026ndash;914.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYin P, Xu Y, Zhu J, et al. Deep level set learning for optic disc and cup segmentation[J]. Neurocomputing, 2021, 464: 330\u0026ndash;341.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arxiv preprint arxiv:2102.04306, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arxiv preprint arxiv:2010.11929, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZilly J, Buhmann J M, Mahapatra D. Glaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation[J]. Computerized Medical Imaging and Graphics, 2017, 55: 28\u0026ndash;41.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang L, Gu J, Chen Y, et al. Automated segmentation of the optic disc from fundus images using an asymmetric deep learning network[J]. Pattern recognition, 2021, 112: 107810.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTulsani A, Kumar P, Pathan S. Automated segmentation of optic disc and optic cup for glaucoma assessment using improved UNET\u0026thinsp;+\u0026thinsp;+\u0026thinsp;architecture[J]. Biocybernetics and Biomedical Engineering, 2021, 41(2): 819\u0026ndash;832.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePachade S, Porwal P, Kokare M, et al. NENet: Nested EfficientNet and adversarial learning for joint optic disc and cup segmentation[J]. Medical Image Analysis, 2021, 74: 102253.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuo X, Li J, Lin Q, et al. Joint optic disc and cup segmentation using feature fusion and attention[J]. Computers in Biology and Medicine, 2022, 150: 106094.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFu H, Cheng J, Xu Y, et al. Joint optic disc and cup segmentation based on multilabel deep network and polar transformation[J]. IEEE transactions on medical imaging, 2018, 37(7): 1597\u0026ndash;1605.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arxiv preprint arxiv:2102.04306, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2022: 205\u0026ndash;218.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi S, Sui X, Luo X, et al. Medical image segmentation using squeeze-and-expansion transformers[J]. arxiv preprint arxiv:2105.09511, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYi Y, Jiang Y, Zhou B, et al. C2FTFNet: Coarse-to-fine transformer network for joint optic disc and cup segmentation[J]. Computers in Biology and Medicine, 2023, 164: 107215.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHussain R, Basak H. UT-Net: Combining U-Net and Transformer for Joint Optic Disc and Cup Segmentation and Glaucoma Detection[J]. arxiv preprint arxiv:2303.04939, 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu J, Fang H, Shang F, et al. SeATrans: learning segmentation-assisted diagnosis model via transformer[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 677\u0026ndash;687.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHan B, Yao Q, Yu X, et al. Co-teaching: Robust training of deep neural networks with extremely noisy labels[J]. Advances in neural information processing systems, 2018, 31.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTarvainen A, Valpola H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[J]. Advances in neural information processing systems, 2017, 30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLaine S, Aila T. Temporal ensembling for semi-supervised learning[J]. arxiv preprint arxiv:1610.02242, 2016.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248\u0026ndash;255.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDecenci\u0026egrave;re E, Zhang X, Cazuguel G, et al. Feedback on a publicly distributed image database: the Messidor database[J]. Image Analysis and Stereology, 2014, 33(3): 231\u0026ndash;234.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlmazroa A, Alodhayb S, Osman E, et al. Retinal fundus images for glaucoma analysis: the RIGA dataset[C]//Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. SPIE, 2018, 10579: 55\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang S, Yu L, Yang X, et al. Patch-based output space adversarial learning for joint optic disc and cup segmentation[J]. IEEE transactions on medical imaging, 2019, 38(11): 2485\u0026ndash;2495.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang S, Yu L, Li K, et al. Boundary and entropy-driven adversarial learning for fundus image segmentation[C]//Medical Image Computing and Computer Assisted Intervention\u0026ndash;MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13\u0026ndash;17, 2019, Proceedings, Part I 22. Springer International Publishing, 2019: 102\u0026ndash;110.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBateson M, Kervadec H, Dolz J, et al. Source-relaxed domain adaptation for image segmentation[C]//Medical Image Computing and Computer Assisted Intervention\u0026ndash;MICCAI 2020: 23rd International Conference, Lima, Peru, October 4\u0026ndash;8, 2020, Proceedings, Part I 23. Springer International Publishing, 2020: 490\u0026ndash;499.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBateson M, Kervadec H, Dolz J, et al. Source-free domain adaptation for image segmentation[J]. Medical Image Analysis, 2022, 82: 102617.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang C, Guo X, Chen Z, et al. Source free domain adaptation for medical image segmentation with fourier style mining[J]. Medical Image Analysis, 2022, 79: 102457.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu S, Liao Z, Xia Y. ProSFDA: prompt learning based source-free domain adaptation for medical image segmentation[J]. arXiv preprint arXiv:2211.11514, 2022.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi F, Jiang A, Li M, et al. HPFG: semi-supervised medical image segmentation framework based on hybrid pseudo-label and feature-guiding[J]. Medical \u0026amp; Biological Engineering \u0026amp; Computing, 2024, 62(2): 405\u0026ndash;421.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"OD and OC, segmentation, CNN, Transformer, Semi-supervised","lastPublishedDoi":"10.21203/rs.3.rs-4890313/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4890313/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCurrent neural network-based optic disc (OD) and optic cup (OC) segmentation tend to prioritize the image's local edge features, thus limiting their capacity to model long-term relationships, with errors in delineating the boundaries. To address this issue, we proposed a semi-supervised Dual Self-Integrated Transformer Network (DST-Net) for joint segmentation of the OD and OC. Firstly, we construct the encoder and decoder of the self-integrated network from the mutually enhanced feature learning modules of Vision Transformer (ViT) and Convolutional Neural Networks (CNN), which are co-trained with dual views to learn the global and local features of the image adaptively. Secondly, we employed a dual self-integrated teacher-student network with a substantial amount of unlabeled data, which is utilized through semi-supervised learning to facilitate the acquisition of more refined segmentation outcomes. Finally, the Boundary Difference over Union Loss (BDoU-loss) enhances the network's focus on the boundaries. We implemented the comparative experiments on the publicly available dataset RIGA+. The Dice value of OD and OC of the proposed DST-Net reached 95.12(±)0.14 and 85.69(±)0.27, respectively, outperforming other State-Of-The-Art (SOTA) methods and proving its promising prospect in OD and OC segmentation.\u003c/p\u003e","manuscriptTitle":"DST-Net: Dual Self-Integrated Transformer Network for Semi-Supervised Segmentation of Optic Disc and Optic Cup in Fundus Image","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-15 12:28:25","doi":"10.21203/rs.3.rs-4890313/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"bd9ee196-bd2a-49c2-8bbe-0310a6a99bc0","owner":[],"postedDate":"October 15th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":37697319,"name":"Physical sciences/Engineering/Biomedical engineering"},{"id":37697320,"name":"Biological sciences/Neuroscience/Computational neuroscience/Network models"},{"id":37697321,"name":"Health sciences/Health care/Medical imaging"}],"tags":[],"updatedAt":"2024-11-19T13:53:23+00:00","versionOfRecord":[],"versionCreatedAt":"2024-10-15 12:28:25","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4890313","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4890313","identity":"rs-4890313","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}