ResTr: A Joint Framework for Retinal Vein Occlusion Image Classification

ResTr: A Joint Framework for Retinal Vein Occlusion Image Classification | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article ResTr: A Joint Framework for Retinal Vein Occlusion Image Classification Xiaochen Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2225149/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The classification method based on convolutional neural network can improve the performance of the pattern recognition system by automatically extracting and self-learning features. At present, the conventional image classification network, including the network method designed in this paper, directly uses the common depth convolution network to directly extract and classify features. Therefore, the core of optimizing the experimental results is how to better extract the features in the image, that is, to extract more advanced and richer features. The advantage of Transformer is to capture the global context information in the way of attention, so as to establish a long-distance dependence on the target, so as to extract more powerful features. Figures Figure 1 Figure 2 Figure 3 Introduction Over the past few years, transformer has demonstrated superior performance in the field of natural language processing[ 1 ]. Different from the convolutional neural network (CNN), which focuses on the local receptive field of convolutional layer, transformer mainly extracts the intrinsic features of text data based on self-attention mechanism. Due to its powerful feature extraction and representation capabilities, researchers have been trying to apply transformer to computer vision tasks. The good news is that transformer has also made significant breakthroughs in the field of computer vision during the last two years. A large number of transformer-based methods have been applied to image processing tasks, such as DETR[ 2 ] for object detection, ViT[ 3 ] and DeiT[ 4 ] for image classification, and SETR[ 5 ] for semantic segmentation. However, compared to natural images, medical images are usually limited in number due to the high cost of data collection and annotation. And they are larger in size, with more noise and redundant information. Therefore, how to optimize the performance of transformer-based models in medical image tasks is a problem that needs to be addressed continuously. ResNet and its variant structures are currently the basic building blocks of most methods proposed for image classification. The convolution kernels of these networks basically focus on only a local subset of pixels in the whole image and then strictly follow the process of refining global features from local features. But as thus, the networks will lack the ability to model the long-range correlations present in the image. Nevertheless, in medical images, such as the retinal images that are the focus of this paper, there are many explicit sequences that contain important long-range dependencies and semantic information. As shown in Figure, most visual representations in retinal images are ordered, and disruption of these sequences would significantly reduce the accuracy of the model classification. The previously mentioned transformer is an encoder that handles serial relationships, which enables global or remote interaction between all embedded entries. In Paper [ 6 ], Google examined some key layers of ResNet and ViT using Centered Kernel Alignment (CKA) and demonstrated that ViT retains more spatial information than ResNet. This shows that transformer has a powerful ability to model sequence correlation and long-range correlation of semantic information. And it is our motivation for applying transformer to retinal image classification. As described in other articles[ 7 – 17 ], convolutional neural network has more or less many shortcomings. For instance, Jeya et al. proposed a gated axial-attention model, which extended the existing architectures by introducing an additional control mechanism in the self-attention module. Shuang Yu et al. proposed a multiple instance learning (MIL) based ‘MIL head’, which can be conveniently attached to the ViT in a plug-and-play manner. Hu Cao et al. used hierarchical Swin Transformer with shifted windows as the encoder to extract context features. These works basically revolve around pure transformer structures, but transformer-based models only work well when trained on large-scale datasets. In addition, due to the small size of the retinal image dataset, it would be a problem that the network lacks sufficient information to establish relationships between low-level semantic features. To overcome the above limitations, we designed a joint framework that can combine the transformer with the CNN structure. Two major contributions are achieved with this paper: 1) We propose a joint model ResTr for retinal image classification. we introduce transformer to explore the long-range dependence of explicit sequences of images and employ CNN to compensate for the loss of local structural information. 2) In the ResTr method, we design a joint loss function to optimize its dual flow and increase the intra-group aggregation. Method In this paper, we propose ResTR, and its framework is provided in Figure. As shown in Figure, ResTr consists of two different branches to extract semantic features and structural features, respectively. Furthermore, we design a joint loss function to optimize the end-to-end model. As shown in the figure below: 2.1 CNN Branch As a CNN model widely used in image classification tasks, ResNet is used to extract local features. We used the core of ResNet's architecture, the residual structure other than the fully connected layer. The residual structure can solve the gradient and performance degradation problems while maintaining network complexity and depth, and achieve very significant classification results. Unlike other traditional CNNs, ResNet uses stride = 2 convolution for downsampling and replaces the fully connected layer with a global average pool layer. Feature extraction is then performed by stacking 3X3 convolutions, and training is accelerated using Batch Normalization (i.e., dropout is discarded). Figure 3 shows the network structure of ResNet, which is conventionally divided into 5 parts. 2.2 Transformer Branch We use the transformer to extract global features thanks to its ability to model the long-range dependencies between the input sequence elements. Unlike CNN branches, the input image needs to be divided into separate patches with size of P × P[ViT]. An image can be obtained with 16 patches, which are further flattened into 1D format and then embedded into the D dimension by linear layers, which are further flattened into 1D format and then get embedded via a linear layer into D dimensions. For the patch embedding the position information of the image sequence is added using the standard learnable 1D position embedding. The resulting vector sequence is then fed into the Transformer encoder. More specifically, the encoder consists of alternating Multihead Self-Attention (MSA) and MLP block layers. LayerNorm (LN) is applied before each block and residual concatenation is applied after each block. 2.3 Self-Attention Overview The core of the Transformer structure is the SA mechanism, which is similar to the idea of attention. In the image classification task of this paper, SA can be understood as computing the correlation between each pixel point. The specific computation process is following: an input feature map with height H, weight W and channels C. The output of a self-attention layer is computed using the following equation: actual process of getting such affinities, the calculation is very expensive. Inspired by axial attention[Axial-DeepLab], self-attention is decomposed into two self-attention modules. The first module performs self-attention on the feature map height axis and the second one operates on the width axis. Axial attention can effectively capture non-local contextual information and is computationally more efficient. It can also effectively embed location information and capture long-range dependencies between feature mappings. After that, the updated self-attention mechanism along with width axis can be written as: Eq. 2 describes the axial attention applied along the height axis of the tensor, and a similar formulation is also used to apply axial attention along the width axis after. Figure 2 illustrates the calculation process of axial attention along two directions. Dataset Details The main manifestations of RVO fundus are tortuous dilatation of the affected veins, flame-like haemorrhage along the retinal veins[ 7 ]. Each eye of the patient is treated as an independent classification sample. Multicolor(MC) image is one of the new technologies based on confocal scanning laser ophthalmoscope (cSLO), which can obtain multiple modal images with different wavelengths. We collected the results of the MC image examination from 29 patients as an internal dataset with a total of 220 images (some cases with missing images). The number was increased to 1320 by image enhancement strategies, including random cropping, rotation, horizontal flipping and color jitting. The ophthalmologist manually labelled them as RVO or not. These labels are treated as the ground truth for validating our algorithm. As shown in the figure below: Experimental Setting We implement the proposed framework and perform experiments using the PyTorch library. All experiments are performed on a GPU cluster with four NVIDIA GeForce RTX 3090 GPUs and each with 24 GB of memory. The optimization process is run for 50 epochs. We set the learning rate as 0.0001, and the batch size is set as 12. All methods are optimized by the Adam optimizer. The cross-entropy function is selected as the classification loss function. Conclusion One of the research difficulties is that Transformer mechanism is a relatively new image processing technology introduced from the field of natural language processing, which is not as mature as the application of convolutional neural network in the field of computer vision. Moreover, the reason why Transformer has strong ability to learn long-term semantic information is that it does not assume to start from local information, but can get global information from the beginning, so it will be more difficult to learn. In view of these difficulties, the improved network structure in this paper requires a lot of experiments to constantly adjust the parameters to optimize the classification results. Less pathological sample data is a major difficulty in medical image analysis task, which is also inevitable in this study. Although the number of data sets can be expanded through data enhancement, in order to avoid over fitting of classification model training, it is also necessary to balance data expansion and diversity. In addition, in order to improve the generalization ability of the model, we need to continue to find large public datasets for comparative experiments. References Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need[J].Advances in neural information processing systems,30 Carion N, Massa F, Synnaeve G et al (2020) End-to-end object detection with transformers[C]//European conference on computer vision. Springer, Cham, : 213–229 Dosovitskiy A, Beyer L, Kolesnikov A et al An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv:2010.11929, 2020. Touvron H, Cord M, Douze M et al (2021) Training data-efficient image transformers & distillation through attention[C]//International Conference on Machine Learning. PMLR, : 10347–10357 Zheng S, Lu J, Zhao H et al (2021) Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. : 6881–6890 Raghu M, Unterthiner T, Kornblith S et al (2021) Do vision transformers see like convolutional neural networks?[J]. Adv Neural Inf Process Syst 34:12116–12128 Usman M, Fraz MM, Barman SA (2017) Computer vision techniques applied for diagnostic analysis of retinal OCT images: a review[J]. Arch Comput Methods Eng 24(3):449–465 Singh A, Dutta MK, ParthaSarathi M et al (2016) Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image[J]. Comput Methods Programs Biomed 124:108–120 Fu H, Cheng J, Xu Y et al (2018) Disc-aware ensemble network for glaucoma screening from fundus image[J]. IEEE Trans Med Imaging 37(11):2493–2501 Singh A, Dutta MK, ParthaSarathi M et al (2016) Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image[J]. Comput Methods Programs Biomed 124:108–120 Li Z, He Y, Keel S et al (2018) Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs[J]. Ophthalmology 125(8):1199–1206 Li A, Cheng J, Wong DWK et al (2016) Integrating holistic and local deep features for glaucoma classification[C]// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, :1328–1331 Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. : 2818–2826 Karri SPK, Chakraborty D, Chatterjee J (2017) Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration[J]. Biomedical Opt express 8(2):579–592 Wang J, Wang Z, Li F et al (2019) Joint retina segmentation and classification for early glaucoma diagnosis[J]. Biomedical Opt express 10(5):2639–2656 Acharya UR, Chua CK, Ng EYK et al (2008) Application of Higher Order Spectra for the Identification of Diabetes Retinopathy Stages[J]. J Med Syst 32(6):481–488 Du N, Li Y (2013) Automated identification of diabetic retinopathy stages using support vector machine[C]// Control Conference (CCC), 2013 32nd Chinese. IEEE, : 3882–3886 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2225149","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":148549199,"identity":"2ed306aa-13ac-4dc3-b62b-dc9057e67542","order_by":0,"name":"Xiaochen Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzUlEQVRIiWNgGAWjYDACCTiL+cCBDz9I08KWeHBmD2laeIwPc7ARoYN/dvOxBx9qDsuZ86/5cJiBh0GeX+wAAUvuHEs3nHHssLHljLcbDhdYMBjOnJ2AX4uBRI6ZNA/b7cQNN85uODyDhyHB4DZBLfnfpHn+gbSceXCYh40oLTls0rxtQC3nexiI0yJxI81Mcmbff2ODG2wGwECWIOwX/hnJzyQ+fEuTMzh/+PGHDz9s5PmlCWhBsg+sUoKAKlT7DpCiehSMglEwCkYSAADWPUkXdHryNwAAAABJRU5ErkJggg==","orcid":"","institution":"Shandong Normal University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Xiaochen","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2022-11-01 08:59:27","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2225149/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2225149/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":28629896,"identity":"86748ec3-a999-47a9-accf-1ea86eea8d31","added_by":"auto","created_at":"2022-11-03 19:28:11","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":62555,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version.\u003c/p\u003e","description":"","filename":"Fig1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-2225149/v1/159e8c93cb8cc40607dfc5e5.jpg"},{"id":28629894,"identity":"51307fc3-7ae8-40c2-b228-187e99a9ca97","added_by":"auto","created_at":"2022-11-03 19:28:11","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":84252,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version.\u003c/p\u003e","description":"","filename":"Fig2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-2225149/v1/412d67a07882174992c14977.jpg"},{"id":28630753,"identity":"b178bb5a-c414-4750-afd8-3a33cc8fcbee","added_by":"auto","created_at":"2022-11-03 19:36:11","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":70485,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version.\u003c/p\u003e","description":"","filename":"Fig3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-2225149/v1/e269e4c6308ae5f49d1d8011.jpg"},{"id":28776975,"identity":"3799ac44-cfd1-4d15-ad64-810977a76644","added_by":"auto","created_at":"2022-11-08 00:14:30","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":344016,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2225149/v1/1c50fb25-77ff-4850-ba20-6bc68b542f2f.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"ResTr: A Joint Framework for Retinal Vein Occlusion Image Classification","fulltext":[{"header":"Introduction","content":"\u003cp\u003eOver the past few years, transformer has demonstrated superior performance in the field of natural language processing[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Different from the convolutional neural network (CNN), which focuses on the local receptive field of convolutional layer, transformer mainly extracts the intrinsic features of text data based on self-attention mechanism. Due to its powerful feature extraction and representation capabilities, researchers have been trying to apply transformer to computer vision tasks.\u003c/p\u003e \u003cp\u003eThe good news is that transformer has also made significant breakthroughs in the field of computer vision during the last two years. A large number of transformer-based methods have been applied to image processing tasks, such as DETR[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] for object detection, ViT[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] and DeiT[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] for image classification, and SETR[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] for semantic segmentation. However, compared to natural images, medical images are usually limited in number due to the high cost of data collection and annotation. And they are larger in size, with more noise and redundant information. Therefore, how to optimize the performance of transformer-based models in medical image tasks is a problem that needs to be addressed continuously.\u003c/p\u003e \u003cp\u003eResNet and its variant structures are currently the basic building blocks of most methods proposed for image classification. The convolution kernels of these networks basically focus on only a local subset of pixels in the whole image and then strictly follow the process of refining global features from local features. But as thus, the networks will lack the ability to model the long-range correlations present in the image. Nevertheless, in medical images, such as the retinal images that are the focus of this paper, there are many explicit sequences that contain important long-range dependencies and semantic information. As shown in Figure, most visual representations in retinal images are ordered, and disruption of these sequences would significantly reduce the accuracy of the model classification. The previously mentioned transformer is an encoder that handles serial relationships, which enables global or remote interaction between all embedded entries. In Paper [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e], Google examined some key layers of ResNet and ViT using Centered Kernel Alignment (CKA) and demonstrated that ViT retains more spatial information than ResNet. This shows that transformer has a powerful ability to model sequence correlation and long-range correlation of semantic information. And it is our motivation for applying transformer to retinal image classification.\u003c/p\u003e \u003cp\u003eAs described in other articles[\u003cspan additionalcitationids=\"CR8 CR9 CR10 CR11 CR12 CR13 CR14 CR15 CR16\" citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e], convolutional neural network has more or less many shortcomings.\u003c/p\u003e \u003cp\u003eFor instance, Jeya et al. proposed a gated axial-attention model, which extended the existing architectures by introducing an additional control mechanism in the self-attention module. Shuang Yu et al. proposed a multiple instance learning (MIL) based \u0026lsquo;MIL head\u0026rsquo;, which can be conveniently attached to the ViT in a plug-and-play manner. Hu Cao et al. used hierarchical Swin Transformer with shifted windows as the encoder to extract context features. These works basically revolve around pure transformer structures, but transformer-based models only work well when trained on large-scale datasets. In addition, due to the small size of the retinal image dataset, it would be a problem that the network lacks sufficient information to establish relationships between low-level semantic features.\u003c/p\u003e \u003cp\u003eTo overcome the above limitations, we designed a joint framework that can combine the transformer with the CNN structure. Two major contributions are achieved with this paper:\u003c/p\u003e \u003cp\u003e1) We propose a joint model ResTr for retinal image classification. we introduce transformer to explore the long-range dependence of explicit sequences of images and employ CNN to compensate for the loss of local structural information.\u003c/p\u003e \u003cp\u003e2) In the ResTr method, we design a joint loss function to optimize its dual flow and increase the intra-group aggregation.\u003c/p\u003e "},{"header":"Method","content":"\u003cp\u003eIn this paper, we propose ResTR, and its framework is provided in Figure. As shown in Figure, ResTr consists of two different branches to extract semantic features and structural features, respectively. Furthermore, we design a joint loss function to optimize the end-to-end model. As shown in the figure below:\u003c/p\u003e\n\u003cp\u003e2.1 CNN Branch\u003c/p\u003e\n\u003cp\u003eAs a CNN model widely used in image classification tasks, ResNet is used to extract local features. We used the core of ResNet\u0026apos;s architecture, the residual structure other than the fully connected layer. The residual structure can solve the gradient and performance degradation problems while maintaining network complexity and depth, and achieve very significant classification results. Unlike other traditional CNNs, ResNet uses stride\u0026thinsp;=\u0026thinsp;2 convolution for downsampling and replaces the fully connected layer with a global average pool layer. Feature extraction is then performed by stacking 3X3 convolutions, and training is accelerated using Batch Normalization (i.e., dropout is discarded). Figure\u0026nbsp;3 shows the network structure of ResNet, which is conventionally divided into 5 parts.\u003c/p\u003e\n\u003cp\u003e2.2 Transformer Branch\u003c/p\u003e\n\u003cp\u003eWe use the transformer to extract global features thanks to its ability to model the long-range dependencies between the input sequence elements. Unlike CNN branches, the input image \u003cimg src=\"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/4RD6RXhpZgAATU0AKgAAAAgABAE7AAIAAAAQAAAISodpAAQAAAABAAAIWpydAAEAAAAgAAAQ0uocAAcAAAgMAAAAPgAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFNhY2hpbiBNYWhhcm51cgAABZADAAIAAAAUAAAQqJAEAAIAAAAUAAAQvJKRAAIAAAADOTgAAJKSAAIAAAADOTgAAOocAAcAAAgMAAAInAAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADIwMjI6MTE6MDMgMjA6Mjc6MzAAMjAyMjoxMTowMyAyMDoyNzozMAAAAFMAYQBjAGgAaQBuACAATQBhAGgAYQByAG4AdQByAAAA/+ELImh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8APD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4NCjx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iPjxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+PHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9InV1aWQ6ZmFmNWJkZDUtYmEzZC0xMWRhLWFkMzEtZDMzZDc1MTgyZjFiIiB4bWxuczpkYz0iaHR0cDovL3B1cmwub3JnL2RjL2VsZW1lbnRzLzEuMS8iLz48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyI+PHhtcDpDcmVhdGVEYXRlPjIwMjItMTEtMDNUMjA6Mjc6MzAuOTgwPC94bXA6Q3JlYXRlRGF0ZT48L3JkZjpEZXNjcmlwdGlvbj48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyI+PGRjOmNyZWF0b3I+PHJkZjpTZXEgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj48cmRmOmxpPlNhY2hpbiBNYWhhcm51cjwvcmRmOmxpPjwvcmRmOlNlcT4NCgkJCTwvZGM6Y3JlYXRvcj48L3JkZjpEZXNjcmlwdGlvbj48L3JkZjpSREY+PC94OnhtcG1ldGE+DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgIDw/eHBhY2tldCBlbmQ9J3cnPz7/2wBDAAcFBQYFBAcGBQYIBwcIChELCgkJChUPEAwRGBUaGRgVGBcbHichGx0lHRcYIi4iJSgpKywrGiAvMy8qMicqKyr/2wBDAQcICAoJChQLCxQqHBgcKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKir/wAARCAAdAFcDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6RoryHxpbX/w08WWXjWHVNZ1Lw3vMOoaXcarNILdnGFljV3wwz1Vsheo9V7LwV4auNNSXV9U1i9v73UCZjEdSnntLZHIYRxK7EEDj5yMnnG0HbRHVX/q/9ag9HY6yisTUta1Eak+neH9Mhv7mGNZJ3uro28MQbO1d4R2LHBOAuAOpGRm5pM2rzQOddsbKzlDYRbO9e5Vlx1JaKPBz2wfrQBforzf4n31rB4j8MW13e61awNJcXF6dJnuwfs0cfO9Lf5sb2j+bHAzyM0/wENavJtcvdIvL5PDl5FE+hya3I92+4qd0oDOJPKOVwruCeoxmhapsb0C1+KMt9dLYWNrodxqzas+nrp8eu5l8tGYNMVEO4AbCcY+7znoD18viGzeOePT5ori9jjlYW7lkwydQ5CkxjOBkjvwDWRpXhHVNPl8OrcaxaXFto0TiRBp7I9zKyspl3eaQpw3Ta3Jbn5hi9L4StSdQnt7m4ivNQgkhuLhm3eYGBC7hwPkz8uMYGfU1MrqNlv8A1b/P5grc1+n/AAf6XyMDQPHniHX5Gjh8Oaaj+VbyIP7XkYESrvOT9mGNq5yeRuwo5PHXwa5ptzdfZoLpXnG/dEAd0ew4bcMfL264zkYqloPha00C8u7i2d3NwsUaK3/LNI41QD3JILE9+PQVJb+GrK21G4vkkuGubtWS6dpM/aFP3QwxjCg4XGMDPqaqW/u+f/AIjeyv/XcpWHibU9Tktrm00InSbxiLe6a5xIRtZhI8ezCxsF+U7i2WXKjPBVrw7o2paLaxWV3qsV5ZWsSw2qLaeVIEUADzH3kOwAAyqoOuQaKu8V0uTKMpO6dvu/UztP0fXNW1K+m8b2GkSWxiaGytrS7kmREcFXDq8S5Zgcb/AO6cADLFqPw20Hxd4VsX0XXf7Ll0eBm/s9oL2Wae3j3ZWFt0ShwAcBuCMYwe3dUVC0/r+v69WaPU4rxL4Hg1DxBJrEXh3w9rss8SRywazEFKlc4ZJfKkI4OCpXBwCCOc6/hPR/7HsZ4v+Ec0TQN8m7ydHk3pJx95v3MXPboeO9b1FC0B6nG6foniOX4mX2ua3baT/Zslj9htVgvJHljjDliWVogpLnbn5vl24+brU3gDQtb8LaRJoWp/YZdNs5ZBps8E7tKYC5KJIjIACqkDIY9PbJ6yihaK39dwerv/AFtYKKKKACiiigAooooA/9k=\" width=\"87\" height=\"29\"\u003e\u0026nbsp;needs to be divided into separate patches with size of P \u0026times; P[ViT]. An image can be obtained with 16 patches, which are further flattened into 1D format and then embedded into the D dimension by linear layers, which are further flattened into 1D format and then get embedded via a linear layer into D dimensions. For the patch embedding the position information of the image sequence is added using the standard learnable 1D position embedding. The resulting vector sequence is then fed into the Transformer encoder. More specifically, the encoder consists of alternating Multihead Self-Attention (MSA) and MLP block layers. LayerNorm (LN) is applied before each block and residual concatenation is applied after each block.\u003c/p\u003e\n\u003cp\u003e2.3 Self-Attention Overview\u003c/p\u003e\n\u003cp\u003eThe core of the Transformer structure is the SA mechanism, which is similar to the idea of attention. In the image classification task of this paper, SA can be understood as computing the correlation between each pixel point. The specific computation process is following: an input feature map \u003cimg src=\"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/4RD6RXhpZgAATU0AKgAAAAgABAE7AAIAAAAQAAAISodpAAQAAAABAAAIWpydAAEAAAAgAAAQ0uocAAcAAAgMAAAAPgAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFNhY2hpbiBNYWhhcm51cgAABZADAAIAAAAUAAAQqJAEAAIAAAAUAAAQvJKRAAIAAAADOTgAAJKSAAIAAAADOTgAAOocAAcAAAgMAAAInAAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADIwMjI6MTE6MDMgMjA6Mjc6MzAAMjAyMjoxMTowMyAyMDoyNzozMAAAAFMAYQBjAGgAaQBuACAATQBhAGgAYQByAG4AdQByAAAA/+ELImh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8APD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4NCjx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iPjxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+PHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9InV1aWQ6ZmFmNWJkZDUtYmEzZC0xMWRhLWFkMzEtZDMzZDc1MTgyZjFiIiB4bWxuczpkYz0iaHR0cDovL3B1cmwub3JnL2RjL2VsZW1lbnRzLzEuMS8iLz48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyI+PHhtcDpDcmVhdGVEYXRlPjIwMjItMTEtMDNUMjA6Mjc6MzAuOTgwPC94bXA6Q3JlYXRlRGF0ZT48L3JkZjpEZXNjcmlwdGlvbj48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyI+PGRjOmNyZWF0b3I+PHJkZjpTZXEgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj48cmRmOmxpPlNhY2hpbiBNYWhhcm51cjwvcmRmOmxpPjwvcmRmOlNlcT4NCgkJCTwvZGM6Y3JlYXRvcj48L3JkZjpEZXNjcmlwdGlvbj48L3JkZjpSREY+PC94OnhtcG1ldGE+DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgIDw/eHBhY2tldCBlbmQ9J3cnPz7/2wBDAAcFBQYFBAcGBQYIBwcIChELCgkJChUPEAwRGBUaGRgVGBcbHichGx0lHRcYIi4iJSgpKywrGiAvMy8qMicqKyr/2wBDAQcICAoJChQLCxQqHBgcKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKir/wAARCAAdAFcDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6RoryHxpbX/w08WWXjWHVNZ1Lw3vMOoaXcarNILdnGFljV3wwz1Vsheo9V7LwV4auNNSXV9U1i9v73UCZjEdSnntLZHIYRxK7EEDj5yMnnG0HbRHVX/q/9ag9HY6yisTUta1Eak+neH9Mhv7mGNZJ3uro28MQbO1d4R2LHBOAuAOpGRm5pM2rzQOddsbKzlDYRbO9e5Vlx1JaKPBz2wfrQBforzf4n31rB4j8MW13e61awNJcXF6dJnuwfs0cfO9Lf5sb2j+bHAzyM0/wENavJtcvdIvL5PDl5FE+hya3I92+4qd0oDOJPKOVwruCeoxmhapsb0C1+KMt9dLYWNrodxqzas+nrp8eu5l8tGYNMVEO4AbCcY+7znoD18viGzeOePT5ori9jjlYW7lkwydQ5CkxjOBkjvwDWRpXhHVNPl8OrcaxaXFto0TiRBp7I9zKyspl3eaQpw3Ta3Jbn5hi9L4StSdQnt7m4ivNQgkhuLhm3eYGBC7hwPkz8uMYGfU1MrqNlv8A1b/P5grc1+n/AAf6XyMDQPHniHX5Gjh8Oaaj+VbyIP7XkYESrvOT9mGNq5yeRuwo5PHXwa5ptzdfZoLpXnG/dEAd0ew4bcMfL264zkYqloPha00C8u7i2d3NwsUaK3/LNI41QD3JILE9+PQVJb+GrK21G4vkkuGubtWS6dpM/aFP3QwxjCg4XGMDPqaqW/u+f/AIjeyv/XcpWHibU9Tktrm00InSbxiLe6a5xIRtZhI8ezCxsF+U7i2WXKjPBVrw7o2paLaxWV3qsV5ZWsSw2qLaeVIEUADzH3kOwAAyqoOuQaKu8V0uTKMpO6dvu/UztP0fXNW1K+m8b2GkSWxiaGytrS7kmREcFXDq8S5Zgcb/AO6cADLFqPw20Hxd4VsX0XXf7Ll0eBm/s9oL2Wae3j3ZWFt0ShwAcBuCMYwe3dUVC0/r+v69WaPU4rxL4Hg1DxBJrEXh3w9rss8SRywazEFKlc4ZJfKkI4OCpXBwCCOc6/hPR/7HsZ4v+Ec0TQN8m7ydHk3pJx95v3MXPboeO9b1FC0B6nG6foniOX4mX2ua3baT/Zslj9htVgvJHljjDliWVogpLnbn5vl24+brU3gDQtb8LaRJoWp/YZdNs5ZBps8E7tKYC5KJIjIACqkDIY9PbJ6yihaK39dwerv/AFtYKKKKACiiigAooooA/9k=\" width=\"87\" height=\"29\"\u003e\u0026nbsp;with height H, weight W and channels C. The output of a self-attention layer is computed using the following equation:\u003c/p\u003e\n\u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cimg src=\"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/4RD6RXhpZgAATU0AKgAAAAgABAE7AAIAAAAQAAAISodpAAQAAAABAAAIWpydAAEAAAAgAAAQ0uocAAcAAAgMAAAAPgAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFNhY2hpbiBNYWhhcm51cgAABZADAAIAAAAUAAAQqJAEAAIAAAAUAAAQvJKRAAIAAAADODIAAJKSAAIAAAADODIAAOocAAcAAAgMAAAInAAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADIwMjI6MTE6MDMgMjA6Mjk6MTIAMjAyMjoxMTowMyAyMDoyOToxMgAAAFMAYQBjAGgAaQBuACAATQBhAGgAYQByAG4AdQByAAAA/+ELImh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8APD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4NCjx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iPjxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+PHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9InV1aWQ6ZmFmNWJkZDUtYmEzZC0xMWRhLWFkMzEtZDMzZDc1MTgyZjFiIiB4bWxuczpkYz0iaHR0cDovL3B1cmwub3JnL2RjL2VsZW1lbnRzLzEuMS8iLz48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyI+PHhtcDpDcmVhdGVEYXRlPjIwMjItMTEtMDNUMjA6Mjk6MTIuODIwPC94bXA6Q3JlYXRlRGF0ZT48L3JkZjpEZXNjcmlwdGlvbj48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyI+PGRjOmNyZWF0b3I+PHJkZjpTZXEgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj48cmRmOmxpPlNhY2hpbiBNYWhhcm51cjwvcmRmOmxpPjwvcmRmOlNlcT4NCgkJCTwvZGM6Y3JlYXRvcj48L3JkZjpEZXNjcmlwdGlvbj48L3JkZjpSREY+PC94OnhtcG1ldGE+DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgIDw/eHBhY2tldCBlbmQ9J3cnPz7/2wBDAAcFBQYFBAcGBQYIBwcIChELCgkJChUPEAwRGBUaGRgVGBcbHichGx0lHRcYIi4iJSgpKywrGiAvMy8qMicqKyr/2wBDAQcICAoJChQLCxQqHBgcKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKir/wAARCACdAjoDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6RooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKRmVFLOQqqMkk4AFYFn458PX+o29na3zs90SttM1tKsFyQM4imKiOQ4BOFYng+lHWwdLnQUViJ4v0eTxJdaCkl0dRtIfPmi+wThVj5w2/ZsIOCBgnJBAyRWdqfiufUfAtzr/g2WILBG8/manp86LIixlvlRvLZs/KA2cck84xRvsPrY6yiqumC/XTohq8tvNd8+ZJaxNHG3JwQrMxHGONx5zzVqglO6uFFFFAwooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACikZlRSzkKqjJJOABXnlj4jfxH8XrGGB72Kys9Ouphbz2ksCud8UaSgyIu/IMmCpIAx0JNC1ly/1tcHornolFeay+NPFv9ga3qtvZaMynUBp+jJ9rkIkfzfJ3MPKBbL5PBAIHYDc16/8AiBfaWn2OPTLfUdQsTDBf4vtitO4BEMB8rM0uCGK7UABHI6AWv9f15fegen9f1/SO8ori5PHOoIdTCaJDL9g1JLIPHfHZKGKAYJjB8zMmNgBGVbLAYJq3/ivxEmqeKJYLDTbjRtGiECD+0JIZJpiquw3CFsHawAAPDHGTnKm6uv6/q6+8OtjvqK4rUvGM+iWN/HpOkm/tvD9sG1Ke41Bh5eI95jR2V2mlCYJ3bQdwy2TT77xzd215rMUGkQzxadYx3iTG8KAqysxEo8s+WQF6LvJ3Lx12jsH9f1952VFeYJ4rmHgHWPHxt92qSyy2WnW875W2xP5CIB0G6RQ7kYLcAnCqBD448L2fh/wCt9Gv23xg09vHaatIM3Ut28igbX6qnLfIPlC5GMUdbPy/HoGltPP8D1Wiuc8e3cdj8PdYluraG8JtWjS2lQOk0r/KibTwcuyjHvXLeAPCXhrWvA+lQXel2stvpT7Bps9sM21ygKyNMCMtKc5Ofl5UjOAxFq35f1/XqHRP+v6/yZ6ZRXFfFmxs9R+HdzZXtrDc/abi3t4FlQHZJJMkYZfQjceRz1q1pvhvRdT1Kw1hNKs0ttLjMOjqkCqI0OMyKMcA4G0DoOf4uEB1dFcpoEy6R401HwrAWNlHZw6hZxk5Furu6PEp7ICgKjtuIGAABdvPEOp2t5LDB4P1q8jRsLcQTWQSQeoD3Ctj6gGmHVo3qKyruS41PwzMn77Q7y8geGL7Q8fmW8jAqpyjMpOcEYY15/4fu9D8O+fLd6U2j61oSCC80uyzs1CacqkM6cgSs5DKrN8w3sGIxR1f9f10+8Ox6rRXHN4z1O3XWrfUdIs7O/02wTUVRtQZ4TCxcfPIIsow8tsgKw6YJ6jkNATxKNZ8L3lzpmlv4j1WOa+1G6kvZRLHaFkPlAeWQoG9FCnIBVsEElw0ru39df8AJ39Aeiv/AF/Wq+89gorzvUPihfWF5rAPhSZrXR9SgsZ5DfRiSUTbAjRpyCcyA7WZQAQd2cqtuT4gX0NjqXn6PZQX2naillMk2qBYAHjWRG8zy95J3BQixsxY4GRzSWqv/XT/ADQ7NO39df8AI7mivOI/ijqE3hzT9bj8Np9juriWylLXcgaC5V3jjQr5G7Y7qq7mClS/zKMV0Hi3Wr/S/CELssdnqV/cW1iphfzVt5J5FjLKxVd20MSCVGSBxRZ/p8xGd8U5PM0PSdMuJGh07VdXt7PUJQcAQMSSrHsHKqh9mx3qPxiU1zxN4d8LaSV8yyv4dTvjF0tIIclAcfdLttUDuNx7VZ8TN4R07w6/hXVPEWmaGlxASUvbiBpZFZjmQifdvLMGJZgxY7jnPNYXhzVPBmkaQ+k6j428D3WmspV7WzhtbVJwRg+avmsrZ/2VQe2OKFZfJ3/K33NX+YPX7rf18n+BQsdYEXi2/wDFPiPY3hTWr77JbXO75IHhbyommHQxOyuVbO1WYEjlSPVNS0+11rSZ7C9DyWl1GUkEUzRllPUBkIIz7GuKOo/B1tMXTmvPA5sVlMy2pls/KEmMbwmcbscZxnFatl45+HunWcVnp/ijwzaW0K7Y4INQt0RB6BQ2AKNOXl/r5hd81zqYo1hhSNNxVFCjcxY4HqTyT7mnVz8et+E/GcU+k2OuaZqrNH5jw2N+jyIFYYkGxtylW2kMMEHBBBxVbwr4ljk8LXE+u38UbaZfT6dcXlw6xrI0UpjVyTgAsNvoNxwB2o3YbI6mivPPHFlolz8R/C02q6bbXctnDdXRZ4A7kLsSNBnqfMmUqP72CMGun0Hw3aaZfXusSWVtHrGqMHvJokGeAAqA9SAAOe5ye+ALUGblFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQBk69oT66lvH/a9/YQxSB5IrTyttxgg7ZN8bErx0BGQTnNVT4TB8ZSeIzrWpec9sbUWv7jyUjPOB+73/AHvmzuzn24roKKVkBy8HgKwt9H0bTUv9QMOjzCaFjIm6RhGyDfhMdGJyoU7vmznmki8AaXb+LrvxDbT3MFxesr3EcQiUSMoXGZAnnAfKMqJNp5yOTXU0VV3e4GB/wh9gLWzt457qNLXUH1BijqDPK5cnzPl5GZCeMEEKQeKhXwPZLpN1Yf2hqBjur8X8j+Ym7f5/nbfuY27uM43bcDPAx0tFJaf1/XZB/X5/5s5a68Aadeahdyz3uoGxvrhbq60sSqLeeVQo3N8u8g7Fyu/accqat3nhCxvLTUoTPcwnU7uO6uJYiodjHs2pypGzEargg5BPrW9RR/X9fcgOKn8IuNJ1jw09gt9ouoGW5tm+0eW0ErvvZGblh+8JdXUMRyCMqC1Hwvo+tvqVpceLNL1W81KwBWC71DULWS1iyNpaIQqjFyP4niDckbgOvodFNNoHqchdeB7jVtJ0621TxFqcT2d2b0rbPG6tJ5pkjUmaN2ZY8gLnHCgkZAws/gUxeLdQ8TaZrWpRX17CsctkJIo7WYKuFD7Yt+R/f3bx0BAwK66ikBz954QtdW8M6Xo+u3VzfLYtBI8pkIa5eIdXPJIJ5Izz610GMDAHA6AUUUPW/mBg6HpdydZv9f1WFbe8vY44IrcOHNvbx7iqsRwXLO7NgkDIALbdxLzwL4S1G8lu9Q8LaLdXMzbpJp9Pid3PqWK5JreooAx7rwvps+lWmnWsbaba2UqzW8en4gETLnGAoxgE5xjB7gjIOY/w/sri1u/tup6ld6hdSwTNqcjxCeNoW3RbQsYjAUknGzB3HOc11dFHmHkefXHg65vZNS0Sa0u3ttQuIJNR1y8uo3e/hQKTEETBTkFNoVUCliOTiurXw9br4qk14XFz572yW3k7lESqpYjAA3clzkZwcKcZANa1FH9foG5xOo/DG21KXVnk8Q6zEuq3sN9OkX2bCyRY8sLmEkAbE6k52jJOTlJPhhbvrM2rL4k1uO/kvRfLOptv3UgiMPCmHaQYztwwPQHrknt6KFpt/X9WQ27nDwfCywisbmwn1rVbyxnFyRa3ItnjieckvKv7nJfLNgsTjcRjBIrYvfCxu/BqaJJqdzcz2/lvb394EeUSxOHidtoUNhlXPAJAOTk5roKKP6+4XW/9alXTZry40+N9TtFtLrlZYlkEi5BxlWHVTjIyAcEZAOQI9b1WDQdA1DV7xJHt9PtZLqVYgC7KiliFBIGcDjJFXq5LxZqVjr3gqGLSby31G01q9t7DNrKsq3UDzqlyiFSc4hE+4ryoVmyNuQAJp/hnxIukWzax471n7eIEN19kt7ExebtG/YDa7tuc4zzjFM0nTbvXNLi1DTfH3iR7abOxpLKyibgkHKvaBhyD1FZ+teKdQuJNYuLDUTp1tpVx9htYIo0kk1C7wvDbwdsYZ1X5cN947gMV1mjJaaVZ2OgwSO72tkhVvKba6Lhd27G0knnGc85xzQtVf+u/5ag9P6+X5jxHf6VoXlwyXGuXqcI900UTSlm4LlEVVVc8lVztXgM3BZ4a0X+wdES0eQTXDySXFzKowJJpHLyMB2G5jgdhgVq0UAZM/huwufFdt4hnEj3trbNbQgv+7RWbJbb/AHu2fStaiijZWDrcKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigCO5toLy1ltbyGOe3mQxyxSoGSRSMFWB4IIOCDXGyaDfQ/EqwSCPbogmm1tXVWIhuvJa2khwAFRZPtImByWLrOSDuyvbUUAZ0nh7RZbm5uJdIsHnuyhuZWtkLTbCCm84y20gEZ6YGKzbrw5Pe+OLXWbkWk1taqVijlXc8TBTiROOGy8gPOMY74x0dFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUVT0rVrLW9PF7pk3n25kkiD7GX5o3aNxggHhlYfhxxQBcooooAKKKKACiiigAooqG5vbWyERvLmG3E0qwxmWQLvkY4VBnqxPQdTQBNRRRQAVBe31pptnJd6jdQ2ltGMvNPIERB6ljwKnrhrFV8S/FnWG1HEtt4aSCGytX5VZ5U8x5yp/i2lVU9gGx1NHWwdLnX6bqun6zZLeaRfW1/asSFntZllQkdQGUkVarhvBB+3+OPGWs2WF0q4uobaAqMLPNChWaUevzEJnv5fXiqdx8UPIuZYv7Q8Bjy3K4k8X7GGDjlfsxwfbPFF1p5hrqdvqut6VoVstzrmp2emwO+xZby4SFWbGcAsQM4B49qop428KyyyxxeJtHeSGIzSIt/ESkYGS5G7hQOc9MVg/Ea9SbwTpLSiGRL3VNP3CB/NRx5yOQjYG/IUgcDORxzis7w/Dcal8R/FtvcNuklFnHf4O5YoxEX8lTjv5m36BzwTQr6/d+C/zDTT7/AOvuO3k8T6DFo0ery63pqaZIcJetdxiFjnHD52nkEdavWl5bahZxXdhcRXVtMu6OaFw6OPUMOCK4jwUlvq2veIPEd/5f/EvvpdLsI2ACWFvAArbR0Qucsx9No6ACj4cXMNp4Y1vXryaKw0a/1W5vrMzMIo4rYkAPk4ChyrP/AMDz3oVnr5X/ACt+Dv8AJg7/AI2/P/Kx3tFUrHWtL1OVotN1KzvJEjSVkt51kKo4yjEA9GHIPQ9qu0AFFU9S1ay0eGCXUZvJS4uYrWM7GbdLI4RF4BxliBnoO9XKACiiigAqG8vLbT7OW7v7mG1toV3SzTyBERfUseAKmrnfHupzaV4H1KWyx9tmjFraL/enlIjjH/fTilJtLQcbN6lg+MvDC6dDqDeI9IFlPIYork30XlyOOqq27BPsOa2utcL8OIrGbRYLG8gEGq6EFtZdNkAxp7KpClB0IZSSJerBjyPuja8W+Jv+EatreXz9Dh85yudZ1b7ApwM/K3lvuPtgVUrIlXZ0FYU3jnwlbzXEVx4o0WKS1bbOj6hEpiOcYYFvlOeOe9UvCfjL/hJL6e3+1eGpvKj340fXft0g5A+ZfJTaOeuTz2rzrxPcfadC+IhZxDbTazFb3Eq9SNtvEsfTnkuxHpjs1Lrbyv8Ail+o9LfO36nqjeNfCybN/iXR18yD7Qmb+Ibouf3g+bleD83Tg0p8ZeGF06HUG8R6QLKeQxRXJvovLkcdVVt2CfYc1ieKJ59M8E6prAhVNYvYRZ2Sso3RmVhHDF/30wZvcnsBUXw4isZtFgsbyAQaroQW1l02QDGnsqkKUHQhlJIl6sGPI+6Gt35f1+Gn3hsl/X9dfuO660Vlv4m0GK6ltpNb05LiGZIJYmu4w8cj/cRhnIZsHAPJxxWpSAKKKKACiio7i4jtbWW4uG2RQoXdsE4UDJOB7UASUVXsL631TTba/sZPNtrqFJoZNpXcjAMpweRkEcGrFABRRRQBnav4h0XQEifXdXsNMWYkRte3KQhyOoG4jPWnLrukvqqaYmqWTag8XnLaC4QytH/fCZyV98YrmviLdQRy+GLa5AZZdZjmI2biRCjy/KO7FkUADklgBya6HStPZJpdTv41GoXShW6HyYxkrED6DJJPdiT0wALX+vL+vuB6P5fqadRXNzBZWstzeTx29vCheSWVwqIoGSSTwAPWuS8S+O/7A1p7H7X4Ti2orbdV8R/Y5uR3j8h8D0Oeavad4lGq+Db7Vml0mYQxyktpWo/bYflXOPM2Jz6jHHFTKVouS6FJXkk+pKnj3wfJ5Xl+K9Dbzm2RbdRhPmNwML83J5HA9aJ/HnhC2kkS58VaJC8UhikWTUYVKOOqnLcEYPFeZaE88Y+GMcSgyJp8z2cA+7I/2dVaVuOm6Vjn+6uR94113jWKz0qLwxYuiv5+tx3M7+WC8zxRvKXwBy7MigY7sAOwq5LldvO342/AhO6uu1/wudeuu6S+qppiapZNqDxectoLhDK0f98JnJX3xir9YKS2OiQza/4mu7TT57gKkk91MkaQJn5IQ5IHBJzzyxJ6YA1bTUrHUGnWwvLe6a2kMU4hlVzE45Ktg8HkcHmkMs0UUUAFFFFABRRVPVNWstGto7jUpvJiknit0bYzZkkcIi8A9WYDPQZ5oAuUVT0/VrLVJLxLGbzWsblrW4Gxl2ShVYryBnhl5GRzVygAooooAKKhlvbWC6gtZrmGO4ud3kQvIA8u0ZbaOpwOTjpU1ABRRUD3trFexWclzCl1MjPFA0gDyKuNxVepA3DJHTI9aAJ6KKKACiiigArm/GOr3WgLpGqJcrDp0eoRw6krqu0wy5jViSMrtkaM5BHGc8V0lQXlla6jZy2moW0N1bTLtkhnjDo49Cp4IoA850Txbr+p32naVNdbL2+1BNQDLCgMelvEZgrAjg7h5GfvdDkHms3wlcazo/hrRdRttXc2dz4kubKTTTBH5RikvZ1LbtvmeYGO4EMFwANp5J9VTTLCPUFv47G2W8SD7MtwsKiRYs58sNjO3Izt6ZpE0nTo7aO3j0+1WCKbz44lhUKku4vvAxgNuJbPXJJoA830nxJ411rVjqGn2epS28etS2c1p/oC2UVvHM0L5Jf7T5gVd/oW4C7SMtm8T+JbHwb4p8Svq0l09lqt1p9nZfZoFihjF55SyMSFLMiknl1UhRuGcsfQJfCvh641oaxPoWmS6oGVxfPZxmcMoAU+YRuyAAAc8YFXI9Osobae3is7dILhneaJYlCys5JcsMYJYkkk9cnNAHM+EJvFP9sXsGuW2qHTPJR7e41c2QnEuSHTFq20pjaQSAc7uTxjU8aazc+HvA2taxYRLNc2NjLcRI4JXcqEgkDqBjJ9hVjSPDWheH2lOg6Lp+mGbAlNlapD5mM43bQM4yevqa0mVXUq4DKwwQRkEUAecf2nrukeLLbR/wDhK21rzvD13qBWSC3EglVoljkHloo8s5faCOu7LNwFlsvGV9Ja+B3+2JdTanpEl5eQIE3XLLbq+RgfL85P3cDJx7VvWXgbStJ8UWWq6HaWelwW1rcQNZ2dokSytK0J8wlcDIEIHQ5yORitGw8MaDpVy1xpeiadZTs5dpbe0jjYsQQTlQDkgkZ96AOD8Ga/401f+x9WuINRn03UraSe8kuRYLa226MvGbcROZiAwCYk3HBycEGqthqOtXXw68H65rerf2pd6tqWlu6TWVuIoN0gBMahMhiCMsSSCMpsBxXolr4U8O2WqSanZ6DplvfyljJdxWcaSuW+8S4GTnJzzzVpdI01LK2s10+1W1tGRreAQrshKcoUXGFK9sdO1AHnGneJPGus6zNe6ZZ6lLDb63JZSWn+grZLbxzGJyxZ/tPmBQZM8DdgBSvXd0bUtZtPHUuneKLvUFe8e5bT4o0tjYyxIyldpVfPV1Rl3eYQpbdjIxXQz+FfD11rK6vc6Fpk2pqyut7JZxtMGXG0hyN2RgYOeMU/T/Deh6TfT3ulaNp9ld3GfOntrVI5JcnJ3MoBOTzz3oA0q4Xxb4L1DVfEkWsaXBp8uYfs91byX13ZPdR5yN8sDEMBk/K0bDBPPPHdUUdbgUNFs3sNJgtWtbOzWJQsdrZA+VCuOFU4Gceu1fpV+iim3d3Ar3VhbXsttJdRCRrSbz4ck/I+1l3Y78Mev160Wtha2Ut1LawrG93L507AkmR9oXJ/4Cqj8KsUUgPPNa8B31z4rutSttO0m/sL8KbvT5r+7tFkdeN8ipvimyByGiHQcnHPXXRvItPt7Ead9qNwDDK9uEWG2UjGWVnVioHGFyTjoueNWik1ePL0Drc53wJb31l4OsrPVbCeyu7ZPJcXEkbtLtGBISjuOQB1OR0xjFblte2t6JTZ3MNwIZWhkMUgbZIpwyHHRgeo6ipqz9E0W10HTvsln5jhpZJ5ZZSC80sjF3diAMksT2AHAAAAFU227sNjl/ivNNb+GdLmtbZruePXtOeO3R1UzMLlCEBYgAk8ZJxzzXPHxjr97Z+HYrK9vrm51yW6ku002C0jnsXhVc2cYuSFBUk7i+9j5bkBQRt9TubO2vURLy3iuFjkWVBKgYK6ncrDPQggEHqCKpXnhrQtRiuItQ0XTrqO6lE1wk9ojiaQDAdgR8zAAAE84pAcQPFfiPwtb6XqvjnzLTS913a3fnrB5gxmS2ncxEqHZUZGVTt3OMAcYjuNU8X6VpWjav4mvtRh002izajJpcNqXt5nlyVmSVCTEiOqDygX+RiQSQa6rWfBOm6l4btdAsoLXTdKivIbmW0t7VVjkSOUSmMKMBdzKMkds+ua0bzw1oWoatBql/oun3WoW+0w3c9qjyxbTuXa5GRg8jB4NAGnWNr3hTSvEstpJq6XTtZSCWDyL6eAI46NiN1BYdickZOOprZoo8wOfPgjQj4uPifyLoauVCG4GoXABQdE2b9m3vtxjPOM10FFFHSwdbhWd/YOmG1+zG0Qwm7+2lCSczeZ5m888nfz6cD0FaNFHmBja94U0rxLLaSaul07WUglg8i+ngCOOjYjdQWHYnJGTjqarTeCtETxRJ4qS0vJNZEe3zI9QnG9VHEewyCPb/skbcnJ9a6KijbYN9zkNNh1RvH76nc6JdW9pf2EYZpJISLSVGfAcLK2WZW5KAgDaM9TXX0UUdLB1uQy3trBdW9tPcwxz3JYQRPIA0pUZbaDy2BycdBU1Z9xo1rda9Z6tcb3nsopI7dCfkj8zG58f3iFAyegzjGTnQoA8303xbrEmpW3h69vguo6Xe3b6xdGFfms4VDRsVAwpkE1ueAOA+PbItfF2sXmoR209xqVzput6BeX0bX8NpGjBVQo8CwsZEQiQ/LMSwG3kkNXq0enWUOoTX8VnbpeTqqTXKxKJJFX7oZsZIHYHpVGy8J+HdOkeTT9A0u0eQMHaCyjQsGwGBIHOcDPrigDj/A2uT2z6Jp13eRw6XH4Nsr0LJsUK4ysjljzgLszzgcetZGn+MfFGs6f4TtIpNUknvtCXUrqfSY7FZ55MqpGLkiMKN2SFUnLL0Gc+l3XhrQr1bFb3RdOuF04AWQltEcWoGMeXkfJ91emPuj0pL3wxoGo6db2GoaHpt3ZWuPItp7SN44uMfKpGF444oA4jUtX8Y6doWlav4na+0qytbN21dtJW0eWOQOP3kiyhx5ewFsREsCSMNxj0sHKgjv6jFYz+D/DMstnLJ4d0l5LFVS0drGMm3VTlQh2/KAeQBjBrZoApXmjafqGoWN9e2qT3Onsz2sj5PlMw2kgdM44z2q7RRQAVDe2cGoWM9neJ5lvcRtFKmSNysMEZHI4PapqKAKo0yzW+hvFt0FxBA1vE4/gjYqSoHQAlF/IUy80bT9Q1Cxvr21Se509me1kfJ8pmG0kDpnHGe1XaKAOd8Yw32p6HfaPY6dcym7s5cXcbRBY3AO1QGkVt5OMHG0HknjFaehvO+hWjXli+n3BiHm20jKzRv3yVZgcnnOT155q/RQtACvLdD8a6pe+OPD/AJV3qN1o+vfahG93BaR28qxoXWS3WNjOo+XGJc8Pzg4r1Ksq08L+H9P1B7+w0PTba8eQyvcw2kaSM5DAsWAySQ7DP+0fU0AebWPifxXF8L/DWtXGrXmpah4juLW3cQW1pH9lVlckwhwq+Y20LmRmXc2QoHyV3HgubxG8N/F4ltryOOOcfYZr8232iWMqCfMFuxjyG3AEYyMZGck7LaLpb6N/ZDabZtpmwR/YjAvk7B0XZjbj2xSaToek6DbPb6HpdnpsDvveKzt0hVmwBuIUAE4AGfagCl4v1caJ4ZnuvtMlvIzxwRPDCssheR1RVRWIXeS2FLHaCQWyAQfMb7XdW1HT9S0vWXvHOl+JNGWI6gLYXKiSaFirm2PlHnJGMHDDNexXlla6jZy2moW0N1bTLtkhnjDo49Cp4IqlbeGNBsrX7NZ6Jptvbh0k8qK0jVNyNuRsAYyrcg9jyKAMPwD/AMf3jD/sYpv/AETDUnxL1y98O+Cn1DTL6HT51vLSM3NxGrxxo9xGjlgSBt2sc8g46Eda6a3s7a0aZrW3igM8hlmMaBfMcgAs2OpwAMnngVmeKvDw8TaKmnm4+zBbu2uS5j358qZJduMjrsxntnPPSgDhfEPiLxF4a/4SXTrTXn1KSxs7G7gvLu2hMlu81w0bROI0RGBVNw4DfMeehG1M2tHxbZ+FW8U3sGNPe+e/FvbfaLpvM2eWoMRjCKDk4Qt8y89c9LaeGNAsNNl06x0PTbaxmcSS2sNpGkTsMYYqBgn5V5PoPSpNW0DR9fhji13SbHU44m3RpeWyTBD0yAwODQB59pPibUNRPhW+v47e+uEfVYvPS3QG6EG5EkQ4JTeEBO0gHPpgUzwn4k8bX9tpmuS2epX2n31nLc3f2gWKW0RMRkjFuIXMxG8CPEm44bJwQa7Z/DbyeKtM1EXFvFp+lW8kVpYw22wq7hVJL7sbQowFCDGepxirNp4V8PWGrPqljoWmW2oSMzPdw2caTMW+8S4GSTnnnmgDzrV31TVvgvqmqXnjCS4bVfDNzfNZRwW6qp8oMywnZu8objG27e3zLh0Yc6VzdXmg6/4XH2h9YkGi6lcebc28HnSFFt2VFeONSo+YjCgZ4zuIFdtZ+G9D0+e7msNG0+1lvs/apIbVEa4ySTvIHzZJPXPU0th4e0XSxbjTNIsLMWwcQC3tkj8oOQXC4A27iq5x1wM9KAOH0rXNbgn8GahL4iOsx+JWK3Fj5EKxQq0Dzb4CihwEKBTvZ+G55waq+F9f8TPpngnWdT1575dfujaXVm1rDHEg8iZ1dCqh9+YRnLFTuOFHGPQNP8NaFpOoTX+laLp1leXAImuLa0SOSQE5O5lAJyRnnvU8ekabDb2kEOn2scNk++1jWFQsDYK5QYwpwzDIxwx9aAOV8K6lrEXiq40rxXd6j/aU0U1zBAUtjZPCsoG6Bo1Eo2h4wRMc/N/F1rtqzNK8N6HoMk0mh6Lp+mvPjzWs7VITJjONxUDPU9fWtOgAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAorlh4g8R6hqepR6Boel3VjZXX2VLu61WSHz2VELlVW3cYV2aM/Nw0bg4IIqX7d43/wChe8P/APg+n/8AkOgDpKKxNPu/FUt/Gmq6No9taHPmS22ryzSLwcYRrZAecD7wwOeeh26ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoorK17xBb6BBbGWC4u7m8nFvaWdqFMtxIQWwu5lUYVWYlmAAU89AQDVormZfGqo9paw6Dq8+qXKySHTESFZoURtrO7PIse3dgAhzuz8uQCRX/4WPpsttpL2Onape3GqtPHDZwwoJY5YDiWKTe6qjKcjJbb8pw2CCQDrqK4/TviRYai9ky6Rq9va3d2bD7XcQIscN0CwMLDeWzuUrvVWTdxu64aPiVp5lndtJ1ZdPttRfTbjUmhjEEM6zGLB+feVLbfmVSo3DJBDBQDsqK52Txrp0Wk6rqDQ3Ri0u/+wTKEXc0m5FyvzcrmQdcHrxWbZePJ117xRDrmkz2GlaE4Jv2MRVIxCshLhZWYlskrtT7pAOGyKAO0orlY/iBYQrdHXtP1DQTb2D6iF1BI90tun32URO/K5XKnDfMOKzLHxNe6v8TtFgk07V9Hgk0e7na0vigWb95b7HxG7qWGXGCdy5OQAwyAd7RXCfE3Vb60/wCEe06zs9Unh1LVUhuG026S3kdBHI/lK5kRlLFAcggbVYFhkA34PHemQtqscllqUNhoJeK+1G4CtFEUQNjJcySMQR91WOTzyRkA6yiuVh8ewCaSDVtE1bR5/sMl/DFepCTcRRgGTYY5HAZdy5Vip+Ye+Ktv8S7e7msobbw3rzzalbG605DBEv2uMbS7AmUCPaGUnzSmcjGSQCAdpRWfoOt2viLQ7bVLFZUhuFP7uZNrxspKsjDsysCpwSMjgkc155r/AI21u/8ABvxCj/su90ttHW4itb+KWJPL228TgEpKz+ZmQsCoC7dvIbIoA9TorlrDx1YtJc2+rWl/pJtdPOo+bfooWa2XhpRtZmXHBKuFcbh8vXE+j+MYNV1SHT7nStR0me6tmurMX6Rr9qiUqGK7HYqV3oSrhW+YccHAB0VFYGtanLZ+LfDdml7LBHfTXCPbpbo63G2FnAZyQUxjIKg5PBwOazdE+JOn63JpJj0rVbS01hmjsby6ijWOWRULmPAcuDhH5KhTtOGORkA7GiuOsfiTp96bKX+ydWg0+9vTYR6jNFGIRPvZAhAcvguu0MFK5IGQc4u/8Jxpv9jzal5F15MOrf2Qy7F3Gb7SLfcPmxs3nOc5x2zxQB0lFcp4U8V6lr2va/Y32h3FpBpt+1vFcloduBFE4V9srMXPmFshdu3aCQ2RVzxF4z0nwpdQJr7yWdtPDLIl46jyS6AExZznzCuSq4+bawBzxQBv0VTh1EPoi6lcWl3ajyPPe2eLfPGNuSpSPcS3bauSTwM1zlx8RbLTrXV5Na0jVNMn0myW/mtZ1heSSAllDoY5GQ8qRgsD7UAdfRXLJ4/06KTUV1iy1DR/sFgdSY30S/vbYEgyKEZiMY5Rgr/MPlrLsfE17q/xO0WCTTtX0eCTR7udrS+KBZv3lvsfEbupYZcYJ3Lk5ADDIB3tFFef+MdU1TwdDqk0GpXEx1iyuF0wzbXaHU1R2hhjTHzeYmFVMAZtxnfJMSwBY8L69baV4D0e7kjmubzxA8upWun2yhpWa6ke6Ma5IBEYlwXYqvy5O3IFafhN5b7UNc1V/t0UVxdrDFbXU7OIvKQK+1dzKv7zep2cHZnnqYz4L+yaxpNzoV8un2unaadMW3aEzMsGUK+W7N8jfIoJYPkAdxmuc8V+V4V8GaJ4cfWZP7Qlukcva3JtZpo4yZZ3AEgY5UNkbjln65Iwl3fXT8f8tf8AhhtX26f5f5/1rp6ZRWV4cbTZtGS60TUZNSs7pmmW5e7a43knnDMTgAjG0YA6ACtWqehKdwork9b8TxaD4tnOoX86afa6FPqE1qlqrLiORQZPMzv3YOAmMHOc5Fbei6pNq9l9pn0m80wEjYl28LGRSAQwMUjjHPcg+1IZo0Vyg8V6l/ws2fw4NDuXsY7GK4F4rQ8F3cFzmXOz5QoATduDcYwawfhx44aXwt4Ys9ZttVabUkaGHVLoK0dzMoZymS5kztRsMyhTt4J4yAek0Vy1n4+sLy+tEGn6hFYX1w1rZ6pKkYt7mUbvlUBzIAdjYZkCnHBOVzreINetfDml/bbyOabdKkEMFugaSeV2CoigkDJJHUgDqSBQBp0V5beeLdRGo+LLl5dV0I2i6SiwTwRzvAZJ3V9qbzEQ6lQWVjgH+8u0dBqfxK07S73VYZNJ1ea30a4ig1G8ggRorfzFRlf74ZxiTkIrMNpJUAqWAOyorl4/HunxrqX9rWOoaQ+nWyXckV5EpaSFyyq6CNnzllK7Ths8FRmoLj4i2Wn2Gpz61pOqaXNp1l9ve0uVhaWSDJXenlyMpwRggsCOMjkZAOvork3+Ien2q6mdX07U9LOn2B1IrdxJuntwSC6BHYg5ABV9rDcMgc4bqXiS8a30SZ7fUdB+1axDbGKaK2uGuUZGO3KSsEQn+IHcNvTBzQB11Fck3xDsFm83+zdSOkfbRY/20Ej+ymUv5fTf5m3zP3e/Ztz32/NTJviRp8F1dq2laqbOxvxp93fiKPyYJS6oucvuZSXXlVbGecUAdhRVbUdQtNJ0y51DUZ1t7S1iaaaVuiIoyT+Vc7F48DzJbz+HdYs7q4tXubGC5WBTeKgBZVIlIRgGB2yFDz7HAB1dFcT8Jbi61H4fWOq6iupfbL+NZ5pb+784TMyhi8S+YwjjJJAQBMY+6OM1tB8Vm28Y+JdLuHvNRu5NaEdpZxN5jQwi2ty74ZgI4lL5JyBlsDLMAQDv6K5S2+IFjdXVsE03Ulsb2dray1J0jEF1KA3yr8+8btjBWZVU44PIzZ07xtpuqQ+H3s4bpjryPJBGUAaAIuZPNGfl2nCHGfmIHvQB0VFZuo67baXqulWN2koOqTPBBMAPLWRY2kCsc5BZUbHBztx6Vjt4+spZlt9L03UdUupLi4hit7VYg0iwMElmDPIqCMOQmWYEtwAaAOqorlV+IGn3Vvp39kWGoape6gsrrp9ssazQrE2yUyGR1RdjkIfm5Y/LuGSK1p44vrrx22knQL2LTf7NgvftMwiiaLez5aRWlDBRt242bgytkYwSAdnRXKaf8QLK/udO3aXqdrYas5j07UriOMQXZwWXaA5kXeqllLouQPUgG74X8WReK7d7qx0vULezBZUurpI0SVlYqwUByxwQeSoU9iaAN6iuF1nxOug/E6Rb24uZLZtEQwWEJLtcXBnYBY484aQgYz2AJJABI7eGRpbeOR4nhZ1DNFIQWQkfdO0kZHTgke9AD6K4Tx7rl7p3inw5YQarqmm2d7HdtcNpWnrdzOYxHs+UwykD5myQvfk07wb42e60Pw/DrbyXOo6pd3VmkiRorKYfNceegI8uQxxjcoHDHGF6AA7miuI1/wCIFxZNbromi3d+RrY0q5A8oYO0Mdm6VeW3DaTxwc44z1000h0mScrJayGAvtYKzRHbnBAJUkexI460AWaK4TRvH4bR9BtkstX8QanfaPFqO+C1hhaVDgM7AyCONsnO3djnAJrrND1m08Q6FZ6vprM1reRLLGXXawB7EdiOlAF+ivKtf8b63qHgHxs7aXf6RJpcksVveRyxqV2+Xhcxys/mfMTkDbjvniuuj8eadHNqMesWl9ox0+yGoOb2Jf3ltlh5iiNmPVcbGAfkfLzQB09FYGi+LI9V1T+zbvSdQ0e+a2F3Fb6gIt0sW4KWHlyOBtJUEEgjcOOa36ACsTxL4fl1r+zrnT71bDUtLuvtNpPJD50YJRo3V4wyllZHYcMpBIOeMHbooA5Kbwtrp1O01m18Q2q6xHBJbXEk2ml7eaJn3hREsqsu0jg+YTjOc9m6T4CTS77Rbsai002nyXlxcu8QBu5ro7pH4PyANnA54wM8ZPX0UAcnD4G8nRrWw/tDP2fXH1fzPI+9uuHm8vG7j7+3d7Zx2rnNA8Ha1qlnq1nqWoNZ6PP4ju7uSyksCs8qreNImybcAI32oeUYkFsMARt9PooA4bUvh9qF42rWtrr8drpWqX6ahJb/AGAPKsgMZKiTeBsJjBxs3c/eq3qHgY6nfeI47nUF/snxFAI7u2Fv++jcRCIPHLv2gYUHBRuR1xxXXUUAef2PwstV0nU9O1GDw4iahYPZPc6NoC2FwAwALF/NcHoDt2gZA9MVq6b4W1lfFdlr2v6/BqE1nYzWSw22n/Z4yJGibzOZHO/MfPO0jbhVwS3V0UAY+vaB/bl3ok/2nyP7K1Fb7b5e7zcRSR7Oox/rc556dOazW8DQT6J4m0y7vZGi1+7kuS8SbGt96IoAOTkgxg5/SuqooA41vBeqanctdeJ9ct765i06extDaaebdIvOAEkrqZX3sdigYKgDdxzkX7Hwl9j1Dw3c/bd/9h6ZLp+3ysefvEI35z8uPI6c/e68c9HRQBk+GdD/AOEc0JNN+0fads88vmbNmfMmeTGMnpvx15xn2rA1PwFeX9r4ssItYhi0/wASq7vG1kXlt5mhji3B/MAZcRg7doOT96u1ooA5rWPBdtrmoSS31y32abRp9IlhRMMyyshLhs8EBOBg9c9qpeEPASeF78XBt/DZKQeSs+naALO5Y8cvKJWByAcgKMkg8YxXZUUAY+q6B/afiLQ9V+0+V/ZMs0nleXnzfMiaPGc/LjdnoemKydO8Cf2foHhTTf7R8z/hHbsXPmeRj7R+6lj243fL/rs5yfu+/HXUUAeY+DfB2s3nhrS4Nav2ttOtdWlv/wCzpLEx3G9LqR4wZSw/d7tj42ZP97BrTufh5qEzXFrDr8UOky6ymsC2Gn5lEgnSZozL5mChZWIwgYEj5iAQ3d0UAYOkeH7zR/EmsXsWoQyafqtx9re1a1PmxzeXHHkS78FcRD5dmcn71VvF3gxPGTQwajqE0OnwxOUt7fKP9pIxHOXDf8s+Sq4+8QxJwBXT0UAchcaH4m1ePUNL1TWngspNIWzF3axJE8t02d9wqglkAG0bd3JLYAwCcFfg6n2DXYFvdLsW1jSxpzLpWjC1ijwxYSFBISzc4OW5wOlem0UAcx4j8E2/ia9vZL26dILzR5tLeJE+ZRIwbzAxPUY4GP8ACo9N8Layviuy17X9fg1CazsZrJYbbT/s8ZEjRN5nMjnfmPnnaRtwq4Jbq6KACq1/p9rqdskF9F5sSTRTqu4rh4pFkQ8Hs6KcdDjByKs0UAFFFFABRRRQBgX/AIduJ/FX9vWV9DDcLpklhHHPbGVAXkR97AOpIGzG3IznqKh8G+ER4UhvwJbTN7Mspt9OszaWsOEC/u4d77ScZY55NdLRQBgz6Bd/8JvH4gsdQhiR7NbO6tprYyGRFdnUo4ddjZds5DAjHArP07wJ/Z+geFNN/tHzP+Eduxc+Z5GPtH7qWPbjd8v+uznJ+778ddRQBwHh34XWnhvVreW0t/DstrbStJFLLoK/bwCSQPtIkAypOA3l5wBnJ5rpvE+gt4h0uKCC6+x3VtcxXdrcGLzFjljYMNyZG5TyCMg4PBHWtmigDhbn4e3+pRa0+ra+k91q5si7xWPlxw/ZpjIAibycMDj5mJByckEKLl94E+2WXiq3/tHZ/wAJFcJPu8jP2fbFHHj73zf6rOeOvtXXUUAcp4j8CW3ia81WS+umWHUtLi08xLECYykryCTLZB5cfKRj5fesj/hVyyeGdb0kjw3YPqlmbUXWj+HxaOgPUt++beOhx8vSvQqKAMLUPD9xceIm1myvYIZ/7NexSO4tjNGC0ivvYB1LD5cbcjr1rB074aCxhgH261gMesQ6obewsTb2qeXHs2Rw+Y3l7vvM2Tk5OK7uigDhl+H179lGiya5G/hpb/7aLL7ERc4E3niLz/Mxs8zH/LPdt+Xd/FVybwL5ug63pv8AaOP7W1T+0PM8j/VfPG2zG75v9XjOR16cV1tFAFDXdHtfEPh+/wBHv/MFtfW728pjbDBWGCQfXmsKx8J6tLqtjqHifXbfU5tOt5IbX7LYG2G6RQryyZkfc2BgY2gZbg5GOsooAyfCuhf8Iz4R0vQ/tH2r+z7VLfztmzzNoxnbk4z6ZNc+PhyLbxVfeJdN1JbTWLq/FwLgWwObfy443tZBuHmIfL3A5BVsEdDu7aigDgPDvwutPDerW8tpb+HZbW2laSKWXQV+3gEkgfaRIBlScBvLzgDOTzT/AAX4aksvHnifVXjnhsVuDbaZDNFsCK+2a5dPVXmPXp8ld5RQBjeKdAbxFootYLr7FdwzxXVrd+V5nkTRuHVtuRuHGCMjIJFczqPwqsLnT9Ajtjp00+jWrWg/tjTRfQ3CPtLO0ZdSJN6Bg4b+JwQd3Hf0UAcbH4Iu9PbSrzw/eaTpOoWFvNautvpJWylildXYC3WUFGDIpBD/AN7Oc8W28K3x8RwaodWjmEmnJp+oR3Fpk3KKzNuRkdRGxLtnhhjGAMV09FAHE6X4E1G1TQ7HVdejvtJ0CRZLGBLHypmKKUi86XzCH2q38KJkgE9wd7wroP8AwjPhq20n7T9q8hpD5vl7N2+Rn6ZPTdjr2rYooA4/xL8PbbxJ4l/tuS8a2vIbFbeymiiBltJllMgmViffBXGGGQeDit3T7LV4NRkn1LV47uB7aKMW0doIlSZc+ZIDuLYfI+Uk7cdTmtOigDm/EXhzUtT8QaRrGjana2NzpiXEYW7smuUkEoQHhZYyCNnqetZv/Cv7mG2s7my1mNNbttUl1Rryaz3wySyxtHIvkhwQmx8AB8jaCSTnPbUUAcQPAN8ujyxjXkfU21ldYW7kscxiUBQUMSuCU4IHzAgEZJIJPYNBLNpxguJUaZ4tkkkaFVLEYJCknAz2yfrU9FAHJ+GfA/8Awjl1pE39ofaf7N0NNIx5GzzNrK3mfeOPu/d569a1/DGif8I34ZstI+0fafsqFPN2bN3JPTJx19a1aKAOJ1HwDeX+n+J9MXWYY9O16Rp1Q2Raa3lYRhjv8wB1+Q4XaD833jjm/wCIvBFt4lvr+S/uXW3vtJbTXijQblBff5gY5GR6EEfyrp6KAOS8I+CV8M3sly9t4aSRofKE2k6CLCUjIJ3OJXyDgHAA5APautoooA//2Q==\" width=\"570\" height=\"157\"\u003e\u003c/span\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eactual process of getting such affinities, the calculation is very expensive. Inspired by axial attention[Axial-DeepLab], self-attention is decomposed into two self-attention modules. The first module performs self-attention on the feature map height axis and the second one operates on the width axis. Axial attention can effectively capture non-local contextual information and is computationally more efficient. It can also effectively embed location information and capture long-range dependencies between feature mappings. After that, the updated self-attention mechanism along with width axis can be written as:\u003c/p\u003e\n\u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cimg src=\"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/4RD6RXhpZgAATU0AKgAAAAgABAE7AAIAAAAQAAAISodpAAQAAAABAAAIWpydAAEAAAAgAAAQ0uocAAcAAAgMAAAAPgAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFNhY2hpbiBNYWhhcm51cgAABZADAAIAAAAUAAAQqJAEAAIAAAAUAAAQvJKRAAIAAAADODcAAJKSAAIAAAADODcAAOocAAcAAAgMAAAInAAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADIwMjI6MTE6MDMgMjA6MzA6MDEAMjAyMjoxMTowMyAyMDozMDowMQAAAFMAYQBjAGgAaQBuACAATQBhAGgAYQByAG4AdQByAAAA/+ELImh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8APD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4NCjx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iPjxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+PHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9InV1aWQ6ZmFmNWJkZDUtYmEzZC0xMWRhLWFkMzEtZDMzZDc1MTgyZjFiIiB4bWxuczpkYz0iaHR0cDovL3B1cmwub3JnL2RjL2VsZW1lbnRzLzEuMS8iLz48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyI+PHhtcDpDcmVhdGVEYXRlPjIwMjItMTEtMDNUMjA6MzA6MDEuODY5PC94bXA6Q3JlYXRlRGF0ZT48L3JkZjpEZXNjcmlwdGlvbj48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyI+PGRjOmNyZWF0b3I+PHJkZjpTZXEgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj48cmRmOmxpPlNhY2hpbiBNYWhhcm51cjwvcmRmOmxpPjwvcmRmOlNlcT4NCgkJCTwvZGM6Y3JlYXRvcj48L3JkZjpEZXNjcmlwdGlvbj48L3JkZjpSREY+PC94OnhtcG1ldGE+DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgIDw/eHBhY2tldCBlbmQ9J3cnPz7/2wBDAAcFBQYFBAcGBQYIBwcIChELCgkJChUPEAwRGBUaGRgVGBcbHichGx0lHRcYIi4iJSgpKywrGiAvMy8qMicqKyr/2wBDAQcICAoJChQLCxQqHBgcKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKir/wAARCABTAh0DASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6RooooAKKKKACiiigAooooAKKKKACiiigAoorE16/v3jfTvDqW9xqRUNIJrkwrBGT94uEcqx5C/KecnoDSbsBqWt9bX3nfZJllEErQyFeiuv3lz7dD75Hap68d0tpl+Cdla2N3Lo95ruqzokljIuYlkuZHkIdl+6sSudwAOFB4q9qN1Lomv6jqg1rVLyx8H6SFWy/cBbucpuYFViHIQxZZcEeZgYBIL0S1/rS7/rzDrp/WtkeqUV5NJr3iqTR9RvRrl5FNJZQiCAWMMTpdzShI/KhkQyiHOVJmBLclSMZrT1vXNYsYvEaHWZXlt7CGCCG2jiDR3srYjVCUJLYaLcWyP3gIVelDuv6/r+mgWv9en+f4M9AmvbW3uLeC4uYYprlikEbyBWlYAsQoPLEAE8dgamrzRdQn8C2RX+3r7WbbSNBe9uIZvIIuHZ/3b71jDclZcsWPABOTkl02u6zpOrXNvqHiKS+EeiT3+pi0t4VXTnAXyvJ+QnLfvMLIXztz04oen4/hf8AyBJv+vT/ADR6QzKilnIVVGSScACs1vEmkJoY1mS/iTTmOEuXyqyfNtUpkfMGP3SuQ2QVyCK8w8Tal4lTT9A0vVtUaZrxLSPWbfyokHly3EUZL4UEM/70YXChQwwcbq6i/wBmofG7TLLUDtt9N0l72xhfhZLhpPLZx6siDHt5hNOzvbza+5XYrq1/T8dEdJo/ifStdubi2sJplubcK0tvdWsttKqtna2yVVYqcHDAY4PNM1rxXpPh++sLPVJLlJ9Qk8q1WKymmEj9du6NGAOATgkcAnoDXM6bfW2s/FS88RR3cUekWFmujW9w0gCXdy8od1QnhtpCoMdWLDnFUPHF1ql548t5tGie7tPDFr9p1KCBtszifKlYj/z1WNGOOCQ+AQSDS/lff+vys/mOzu12/r89Pkd3ZeItO1DWr3SrRrl7qxO243WcyxocA480qEJwwOAxODUMvizSYfFUfhySS6/tSWPzUiWxnZCnd/MCbNoJAJ3YBIHXinaJrWgXukadLod7ZtZ3q/6EsThfMwNxCr1yACSOowc1zFxeT33xh1Cy09yJ7bR4IN4/5YLLI7ySemcJGAO5I7A4NVJJ/wBaMLpxbX9ao6K18Y6JdrI8dzNHHFefYXkuLSaFBPkjZudAOo25zjcQuckA7lUZNJsToUulyQI1i8LRPG/IZSDu3Z6k5JJPUnNc34V1fUp/g9pWp3MrJftp8bGZ7OW8ZjgAOYoyHkJGDgHPOaOj8rf1+A99jsqztY1/T9CjhOoyS752KQw29vJPLIQMnbHGrMcDkkDjvXJ6T4m1u61i1gudS8yKSQK6f8IRqdruHp5skhRP95gQKseLnS+u7i60LU4rLxB4ZhNwPPZTC8cq5aOUZzsYIPm4IIBGcEUm7K72BJt2R1Om6tYaxatcaXdR3UKuYzJGcgMOq/UdCOxBB5BFS3t9a6bZS3l/PHb28K7pJJGwFFedQeK31SyZ7e9PhDTrfSotZv5dkbzB7ku4VTIrKACrZJUklgBtOaq3Kat4w1HwPZatqV5p9ytous6paQJEsamLHlswdCQxlZflJwNjcZAIpqzs9P6d/wAn/WpKel/66W/NHpGkaxZa5YfbNNkd4t7RnzIXidWU4ZWRwGUgjoQKvV5PBc61a/DXxb4x0zxFdPe3STXdoL2KKZbeFMmLaoVcFo8MASVG8HacndX1XxB4wsl14L4mP+g+HotaQrYQgiY+ZmJcg4iOzkNuf0cd1ddf60b/AEZVn/Xql+qPYKK8k13xnrv9qpf6JqsrWVvqdhZ3VuYoI4E84x7owCryyPiTJYNGoBGNxVhW1omoaynxEl0jUvEl1dWCtLLZStBCq3hAUPbkrCBmI5OVbLbsH7jAOz/rys/yf9aE3Vr/ANdf8v61O3vdStNOa3W8mCPcyiGCMAs8rnsqjJOBknA4AJOACarav4i0zQ3ij1Cd/PmBMVtbwSXE0gHUrFGrOQMjJAwM81jWUstz8XNWS6X5LHSbYWmewlklMrD6mKMH/cFZPg/UbGG/8a+Jdeuo7e5g1OS1ne4YL9ltoQBEvPRSCX9y/el/wX+Nv1K2f3L71c6SbxroMHhWTxG147aXFnzZo7aV2iKnDB41UuhUg7gQNuOcU4eMNIa40yBWvTLqsZltUGnXGSgIBZ/3f7sDcOX29Qa8zns9XvvBM1jbBbTVfHGuyXlvbXanEFsAH/eL1AMcK7h6y49q7/wNrNjqljcw7Gt9btZBFq1rO4aaOYDqTxuQjlGAClSMAdA0m9/LT5a/c3a5L/r79PvSH+GrzxJdaxqkWu3GlSW9nKIQlnayRuHMccg+ZpGDDEhH3V+6D3wOmqlY6TaaddXtzarKJb6YTXBkneQM+0LkBiQvAAwuBwK5vx3ceIY7JU07S9Mns/tlmVmn1KSKQv8AaI8DYIGAGcDO7pzjtS7L0H1OxrA1Pxx4f0e5mgvb2TdbY+0vBayzR2uRn986KVi4OfnI456Ump6nrtl4H1bULnT7aHVLa0mlgt7K5a5ViqEry0aEkkdNv51ySXlh4f8AgfY2ejyJqGoa1Y+XZoG3SX93OuWc85PzOWY9gDnpSd9bdLfO97fkNW0v1v8Ahv8AmdNdX/iI+NLK0sbrSf7JuYWuAr20jzNGhiDYkEgUE+aSPkI+Udc8dPWBoPhS10ix0TznmmvdJ05bFJvtEgVl2qGygO05KA5IzwPSs+P+3tW8T67Dp+uSWthaz20aFreKTy3ULJNGnygkMjKCWLFSePQW0k+Vef5/5WJV2rsdc6/rXnanqVp9hGkaXO0Etq8DtcT7AN7rIHCpjJwpRt23qN3y9bXL6z4It9Q1yHWNPupLK7WVZJ4WZ3tLwjABmgDqruoA2vwQQpO4KBWvq82sQwxnQrCxvHJPmLeXr2wUdiCsUmT7YFRe0f6/r/IfU0axrrxXpNn4ntfD1xJcjUrtDJDEtlMyOo6t5gQoAMjJLcZGcZFZfw8n1uTwppaapp+n21othF5MttfvNI/yjG5GhQLx6M39a5LXdTn/AOFgan4lukNx4U0lV0jUPKJ3x8CSSTAGWiVnRZFHPydwpUv7SX9f1ewdH/X9aXOr1bxfLe+GdVv/AAfLH52leaZ5NS064ER8sPuVM+XvO5NuVYgdeeAel0wX66dENXlt5rvnzJLWJo425OCFZmI4xxuPOeahuLPTfEHh5rTcs+mXkGzNrOyK8ZH8LxkEAj0PSsvxTNf2cOiWehXcsF1PqEUSqdriWJVZ5A5dWO3YjZYENnHPYmzt6f15Bvr6kXjfxLdeFotMvImh+yS3Riuke2aRivlO4KsHURj5MF2yqg7jgA1qRahJpWl2Z8TXtobmeUQme3geKAuxO1fmZtueFBZvmYgDBYLUCeGvtQtZPEF9PqdxZ3ctxbSKzW6oGJCoyRsFcKh2/MDkZ9TUlp4Y0jTfDc2iW8Eh0ySNo2t7i4kmVUK7di+YzbUA4CjAHYUm+WLbHu0Xr3U7PTmt1vJgj3MohgjALPK57KoyTgZJwOACTgAmsa08e6Be2urXFvNemLR9wvmfTblPJKjLLhowWYDkhcnBB6GuS+F2p6jrmqW9zrgcz23hyx+zvISS4leXzJOe7eVHk/7Iq54CJ8SWepTH5tPn1m7uJ2zxMyylI4vdQiIzdjkD+8KppqXL6/nb/g/gLbfy/FX/AOAd5p9/BqenwXtp5nkToHj82F4mIPTKOAw/ECrFVNTl1KKzLaNaWt3c7hiO7umgTHc71jc59tv41zHge58RPFOmoaVpcFmb+8LzQanJLIr+fJkBDAoI3cZ3DjnHalu9A2V2dlXLw6trJ8QXelTXOnieO9jeFBaPlrJlJyf3p+fKOu/AGV+7zUEf9vat4n12HT9cktbC1nto0LW8UnluoWSaNPlBIZGUEsWKk8eg6U6bZnVl1M26fbVgNuJ/4hGWDFfpkA0Lo/67/wBeQPZr+v6/ULLUrTUGuFs5hI9rMYZ0IKtG47Mp5GRgg9CCCMgg1ark9QeS1+LOiC0Ti/027S8x3WJojGx+jSOB/vn1rrKOif8AXYOoUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVlL4W8PpJfSJoWmq+ogresLOMG6BOSJDj58k85zWrRQBlWvhXw9Y28MFloWmW8ME32iGOGzjRY5cY8xQBw2OMjnFWl0nTkku5EsLVXvv+PphCoNxxt+c4+bjjnPFW6KAM+x8P6NplolrpukWNnbpIJkht7ZI0Vx0YKBgN79aWfQtIuhcC50qymF1Iks4kt0bznTG1myPmIwME9MCr9FAFZtMsGuJp2srczXEQgmkMS7pIxnCMcZKjJ4PHJ9ar2fh7RtO059P0/SLC1snOXtoLZEjY+pUDBrRooAz9Q0HS9US6F7Ywu91B9nmmCBZGjByBvHzDBORzweRzXOeIvAVx4k0mwttQ1a2up9PlLxSX2kwXMcw5A81HHJ24BMbR5OTgcAdnRQO5h2/h6SXT7W21m6hnW2ljlWCytRbW+YzlAELOwAYK2N/VR2yDJH4Q8NQy3ksXh7SkkvlZLt1sowbhWOWDnb8wJ5IOc1sUUbiMOy8G6Bpl5bz6VpVpp627tKkNnbpDGZCpTzGCgZYKzKM9mPtjXjtoIriWeKCNJpsebIqANJgYG49TgcDNS0UAZ+tabLq+ntYpePaQzHbcNEv7x4iDlFb+AngFsEgZxgkMt2CCK2t44LeNYoYlCRxoMKqgYAA7ACn0UAFZ2oeHtF1dkbVdIsL5o5PNQ3NskhV8AbhuBwcADPsK0aKAMnWPDmn6vi5ksrL+04YmWzv5rOOaS1Yjhl3DsecZGaZo+gtZT3N9qlxHqGp3kaRXFwsPlIUTO1ETLbV+Zjgkkljz0A2aKAM9tA0Z9F/sd9JsW0vAX7CbZDBgHIHl429Rnp1qgfAfhBt2fCmiHfGImzp0PzIMYU/L0G1cD2HpW/RQBgyeBPCMzFpvCuiSExrES2nRHKKAFX7vQBQAOgwPSpovCPhuBbNYPD2lRixcyWgSyjH2dyQSyYX5SSAcjHStiigDL1DRvtWq2mqWdwbS9tv3bOE3LNCSC0TrxkcZU5BVueQWVuf1bwDNeeMU8SWd5pf2sIqFdR0SK5CbehjdTHKre5dh7dK7SihaWsG5kN4X0e5vrfUtU0ywv8AVLcDZfz2cZmUgkja23KgE8YPH15p1n4V8Padqj6np+g6Za38hYvdwWcaSsW5bLgZOe/PNatFABTJoIriPZPEkqblba6hhlSCDz3BAI9xT6KAGuGaMhGCsRwSM4/CuK8P/D+48NaxdXGm6hpogupGeSUaHDHeYJyV86MqhGf70Zrt6KFo7oOlgqOG3htw4t4Y4hI5kfYoXcx6scdSfWpKKACiiigCOO2ghtVtoYY44ETYsSoAoXGMAdMY7VjxeCfCsFhcWMHhnR47S5KtPbpYRCOUqcqWULhsHpnpW5RQBU0zSdO0WzFno2n2un2wYsILWFYkBPU7VAGane3hknjnkhjaWIERyMoLJnrg9s4GakooAKz9a02XV9PaxS8e0hmO24aJf3jxEHKK38BPALYJAzjBIZdCigadtjIufD8RvbC80uQafcWKiBfLjBjkt8jMLLxxgZUjBU8jILK2lbW0FnbrBaQRwQp92OJAqrzngDjrUtFAgpkUEVupWCJIlZmchFABZjknjuSSSe5NPooAjht4bcOLeGOISOZH2KF3MerHHUn1qSiigDMstG8jWLrVby4N1eTr5UZ2BVghDEiNByeeCzEksR2AVV06KKA63CiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACioY7y2mupraG4ikuLfaZolcFo9wyu4dRkA4z1qagAooooAKKK5mbxNqp8Q6lpen6HHetZG2KyJe7NySn5y25AFZF+baC24dweKOtgJ7nxdb2+pz2406+mtLWQRXWpRrH9nt3IB2tlw5xuXJVGAzyRhsb9cTrOga9bX80elPFe+H9TuDJqNiIlF1GWADeS7uqbGIywYbgC20kkAdsc7Tt5PbJo+ymHUZPI0NvJIkTzMilhGhAZyB0GSBk+5A96xPD3iuLxIsUlnpt7FA9uszzTNCViZsERNtkY79pDYAIAIyQTiqmg+JNZ8RWUEsWjw2kM6XCvefavOjikR9se1SqNIrDLZ+XGCPeovCPg7UPCMdta2mrWr6csKrcWa2BRWlAIMsR80+Xu+UsuGBYFhtLMS15/1v/X3dAe39f1/T8jrqoa5qo0PQ7vU3tLi8S1jMjw22zzCo6kb2VeBk8kcD8Kl1O6ex0m7u41jd4IXkCyyFEO0E8sFYgcdQD9DWX4f1nUNet4Zr3RBZWVxYxTiRrkPukfO+LbtBwox8x4YN0qdXe39b/5D21f9f1c0bPUTeXdxD9jnhWFY2EzlCkm9c4UqxOR3yB1GMjmrtZfh3RB4f0WLTxcyXZjJ/fS/eYdFB9dqhVz321qVTtfQlBRRUL3ltFeQ2ktxElzOrPFCzgPIq43FV6kDcucdMj1pDJqKKKACiiigAooooAKKKKACiioZLy2huobaa4ijuLjcYYmcBpNoy20dTgEZx0oAmoqE3lsL4WRuIhdNGZRBvG8oCAW29cZIGemSKmoAKKKKACiqa6xpj6w+kpqNo2pRx+a9kJ1Mypx8xTO4Dkc4xyKuUAFFFQzXltbTQQ3FxDFLcuY4EkcK0rBSxVQepwCcDsCaAJqKKKACiobW8tr6EzWVxDcxB2jLwuHUMrFWXI7gggjsQRU1ABRRUMl5bQ3UNtNcRR3FxuMMTOA0m0ZbaOpwCM46UATUUUUAFFFFABRRRQAUUUUAFFFQ295bXbTLa3EU5gkMUwjcN5bgAlWx0OCDg88igCaiiigAooooAKKKKACiiigAooooAKKKijurea5mt4p4nmg2+bErgtHuGRuHUZHIzQBLRUN1eW1jCJr24htoi6xh5nCKWZgqrk9ySAB3JAqagAooooA4zxg8Wh+MfDPiWU+XAJJdLvJAPuxTLuQn2EsUY/4HXnenXOpab4d13SZJ5Vv/ABrDb6naAjLRSX0xhmUenlq0R9h9K9m1/wAP6Z4o0SfSNdtftVjcbTJFvZMlWDA7lIIwQDwaivPC+jX+raVqV1Yq93o+8WMgdl8kMArDAODwB1BxjI5oA8/lk1jw94n8bz+GBp0VvpOnWcphu4nfzVit3IiXayhMhcbzux/dNXNZ+IurHVJrXQrXy/sunw3bI2jXl+biSVSyxBoMLEABje27JbhflOe4k8PaXNLqkklrl9XiWG9PmN+9QKUA6/L8rEcY61TvPBeiXs8Mxiu7aaGBbcS2OoXFq7Rr91XaJ1Lgc43Zxk46mgDFsPFWv6v4ueyht7TTbC20uz1K5ju7eR7pTMZN0OA6hWAT7xztIxtbPy5vhP4ia14g1DRp30+SXTtXDMYotFvIvsClC8bvdSfuphgBTtC8uCCQOe8ttFsLPVZ9SghYXlxBFbyzNK7F44yxQHJPI3tz1OeSao6Z4P0fRr5bnS1vbYIWKWyajcfZk3ZyFtzJ5QHJwAuB2xQBuVT0/S4NOmvZYWkeS+uDcTPIQSW2qoAwBwFRQPYVcooAKyYbLVX1XVjdX7iwuERLREdC0J24Zh+6UqcnozSZxn5fu1rUUrX3AradYQaVpdrp9mu23tYUhiU9lUAD9BVmiiqbbd2GxgLod9qWk6vp3iG9kmtr6d1jCSIWS3OPkyIkwCMgghiAT87cEb4GBgcCiikAV5dYa7qWleH7p9B0+C0tv+Ej1KK8uo7G4vxAqTS4kMCSeYxdgMlTtXJO0Dp6jWDL4M0aW1e3Rb62ikuZrpxZ6nc2++WVi0hYxyKTliTg8DPAFAGb4x1O8m+Der6roerW/n/2NLcx39tGwVwISxeLD5Qnnadx2nB+bHODqKeI28ceB0W/02XU5NN1EtdvaSCJF/0U58rzSzt0H+sXrnttPoP9jad/YJ0UWcQ0w2xtfsoXCeVt27Mem3iqlj4U0nTp7CeCK4ebTo5YrWW4vJp2jSXbvXMjkkHy1xnOMcYoA4p/iTqraXotv9nig1O+uL6C4uIdNub6OP7JKYmZYITvO9tuAWAUE5LYGew8H6zf674fF1q1jLZ3STSQsJLaS3EwViFlWOQb0VhhgrcjOMnGSk3grQZrOO2+ySRLDczXcUtvdSwzRSyszSMsqMHXcXbIBAwcYxxWpp2nw6XYpaWz3DxpnDXNzJcSHJzy8jMx69zx0oA4Dw5428T3mkeE9b1pNINj4iljtza2kUiy27PGzK4kZyHBKHKbQVDY3NtyZbHxvr82kaB4lnTTTo+uXsFulhHC4uLeOdisbmbeVdgShZfLUDLYb5cm54D+G2n+FtD0UX0Kzarp1uELR3Uz28cpXa8kUTEIjMM5YIGO5s/eOdez8C+HrDUI7u1spFMM73EMDXcz28ErFizxwM5jRvnblVBG5sdTQBx+seKNQvPC3i+08TpZxSxaTfyx6JNp9xCzxxhgp+0+btnQrt3GIKRvHKnitDR5tfl+JqwRanax6UuhWk/2E20jbVLyAhT52A+V+/tOV2jHGT0H/CC6Cy3ouILu7+3WstnM15qNxcMIZP8AWIhkkYxhsDOzHQegxbfwzpT6vZ6mIZo7yzhEEUkN1LGGjByEdVYCRQSSA4OCT60AcH4I1zUrrw34Y0HwtBpmks+km9le4hluIooxJ5axonmq5JJyWZ+AOhzxbtPHPiLVpNC0+xh0u2v7y41CzvZpo5JYkktH2GSNQyllYgkKW43AbjtyelfwF4dax0+0itJ7ZNMRo7SS1vZ4JYkb7y+ajhypwMgkg4HoKuWnhbRrE6YbOxWH+ykkSz2u37sSDDk8/MW6ktkk5OckmgDlV8cak3hxDczWNpqo1K509mjsLi880wOyl4raI+YwIXJ+f5M8k9Dm6L4jn8V+IfAuqXkKxXBbVoJVSN41LRERlgj/ADJnZna3K5wc4zXZ3XgjQbuNFe1miaO7lvEmtryaCVJZc+YRJG4cBtxyoO3pxwMSad4O0LSWsTp9h5J0953tsSufLac7pTy3OSe+cdsUAc3raatJ8ZrFNCnsrac6BPvmvIHmVF+0RdEV0LEnA+8McnnpVHTPFB1jxD4T1TUrCxhvWsNXjnn85wkTW80Ub7DnAjZlLZZWIAHPXPZ6x4R0bXdSh1DUIJxewRGGK5trya3kjQnJAaN1Iz39enSli8IaDClgkemQiPTrWW0to+SixS7fMUrnDbti5LAnr6nIBx2lfEHW77W7jTra3g1WWfR59R09U06509WkjKBYvMuDtlVvMXEq7QMZIwwx0ngnX7jXbG4/tC+gnvYHUTW66XPp8ltuUMFeKZ2Y5zw3APOM4NFj8PvDumyLJaQXyypavZxzNql08kULbcojtIWQfIuNpGOcYyc6mjeH9P0FZxp6Tl7hw801zdS3MshAwMySszEADgZwOcdTQB5tomr69ocviiXz9Nu7rUPE40+DNo8SpO8cKiRz5pJRUH3BySPvDPGprPjjxF4bj1yxvY9L1DUrCOxubaeGGSCGWO4uPJKOhdyjAqxDBjkMDt4IPXXHhHQ7qw1CynsQ8GpXP2u5HmuC03y4kVs5RhsXBUjBGRg1BH4H0BNPu7NrSaeO9kiluZLm8mmmlaNg0e6V3LkKVBC7sDnjk5ALEsetx+FLhbq/sTqohci5hs3SFTg4/dmUtwP9sevHSvM7eTxFdeG/hXcXOo2d5qN1NHJDcTQSAIradIcy5kZpXGSxIKbzx8mcj2NlV1KuAysMEEZBFYWn+CdC0uLT47S2n8vTZzPZJLeTSi3YxmPCB3OF2MQEHyjOQM0Acpd/EPV9P0s2l3Bbvq4119I+1W1jPNDhYvP84W8ZaQny8DYGPzZO4DOOl8F69qWuWF5/a9nLDLaXRhS5bT5rJLtNoYSLDN86Y3FSCTypIODgWrrwhol5b3UM9m2Lq8F9I8c8iSLcBVUSI6sGjbaoGUI4z6nN3StKt9HtDbWkl3IhcuTd3s10+SAPvyszY46Zx+dAHmPhDWdd0PRLW5X+z5NIufE91YvbmJ/tB829lQSiTcFXDsPk2NkAncCcLeg+I+s3utNJY6dLcWCas2nmyi0S8eQxrKYWm+1j9yMMC5TB+VdpYN07aPwro0WnRWMdni2hvf7QRPNfifzTLvznP3yTjp2xjiol8H6PFrD6nbre208k/wBokjttRuIYJJOMs0KSCNicfNlfm75zQBwdr4wXw9pYt7STR9BS+8T6natfX5ka3i8uSVi7BpVJdymPvquW4HRTFbat4l8Q6p4G1KSTTI9QuzqZhm+ySpAIAFVJRGZCzhlUOPnUEOOeOel8R+B5JHsX8OW8QjivLm7uLdtTuLF5JZ+WdbiINIo3FsxjCtuHTaBWh4Z8Ky2NrZT+IJWvNRsprl7Rjdy3H2WOY/6rzZMNLgYG5xn0AAFAHJXfiDxVrFz4XFtqNjp1yviC8026C2srw3DwxXADlRMpMZEefLJJDbTu+XB6vX/Fs1npGrf2NaXc+p6bA05W40u5jgmEZBdUkZVRiVzt2uecHkA1duvBmhXdn9mktZUQXsl+rwXUsUiTuWLusiMGUne3AIGCRjFbflr5XlsNybdpDfNke+etAHn2ofEi4t59be0t7aa1js86O/Obq5CxZRsHkF7mFRjByH69oX1TVdM1rxfLpGmW02pxzacLqeGCabcrRKJJfJEmX2LnCIVYgdz16e18A+GrLTdKsLXTAltpFx9pso/PkPlSbt2SS2WGeQGyBheOBi1deFtLu5r6ZluoJr943uZbS+nt3cxrtX5o3UgAcYGAe+aAOPvfiJdpFollp17b3l3fpcyz38Gg3kyxCF1Ux/Y42MqvudVO9xt2nPJC11XhbXbzV/CcepaxZNp90nmLMtxE9urbGK+aFkG9EYAOA3IDYOcZLZPA+gSWFraC1miFpJJLDPBeTRXCvISZGM6uJCXJy2WO44JzgVq2umW1ppY05RLPbBWQi7ne4Z1OchnkLM3UjknjjpQBwOkeO9b1fxENItJbGd9Q0ye8sLhtJu7aGJ0KBQWlI+0xnzB88e3gZwNwFZN346k8a/D3VdSm0yy/s+0urC3NpLKXZ7n7RCzvvikH7sblKf3sbj8pwe2g+G/hi3j2R2l2QLSSyQvqVy5jgcKGjQtISi/IuAuMc4xk5t3Hgnw9dNMZNOVRcQwQSpFK8aOkD74gVVgMqehxnHGccUAcz4o+IN7omtuLOS1ubS2vraznto9NupT+9eNSWu1xDC480Hy2DHAXnLgDU8A/8f3jD/sYpv8A0TDV298A+HNQv57u6spi9xOlzNGl5MkLzJt2yGJXCF/kX5tuTjk1r6fpNlpcl49jD5TX1y11cHezb5SqqW5Jxwq8DA4oA4X/AIWBqL+MNP0+2ns7mz1K8msomi0u6CQOiSMHF0xEU/MRDIgUjJAJ2knAW98Z/wDCrbS8bxJamWTX4oFl+xzeYF/tIxlWf7R8yE7fl4+QFMnO4ehWvgHw5Z6lb3tvZTLLa3L3VuhvJmit5H37jHEX2ID5jZCqBz04GH/8IN4f/s++sfss/wBkvphcSwfbZ9kcgkMu+Ib8QneS2Y9vOD2FAHLa7468QafrN9pVjHHPdaVaRyTGLQb26W+mdSwRDCWFuvAG5zIct0+Xma68XeKb7UdZh0WDTrBNM0i21Ix6jbSySl5UlbyWCugX/V4Lfw/3Wzx0dz4I0W5mjmYahFPHCsBuLfVbqGWVFztEkiSBpcZOC5YjJ9TV5fD+mLdX9yLcmbUbeO2unaVyZY0DhQcnjHmPyOTnknigDj9I1XX9Z+JFrcwajbwaVdaDbX32CS3kcqrscgMJQu/Ofn2dMDbxk0PBmu6jcaB4b0DwtBpulSS2E97LJcRS3EUUSTeWESPzQ7MWbJZnwAOhyMdwvhLR473TruGCaGfTbdba3aG7lj/dLjCOFYCRRjo+7v6mq0ngPw89jYWiWk1ummhxaSWt7PBLEHOXUSo4faT1BbBwPQUAc3beOPEWpS6NptnDpcGo3V/f6deTyxySQq9rkGSNQwJDbc7C2RnG7jJ6fwfrV7rWlXR1WOBbyyvriyla3UrHKY3Kh1UklQRg4JOPU1YtPCui2A0sWlisX9kiT7Htdv3ZkBDsefnZsklmySSSTkk1csNMs9LW4Wxh8oXNw9zL8xbdI5yzck4yew4oA87sfGvjK907QL9V0JYtdv5dOjiMM263ZRKRMW3/ADjELfu8Lk4G8ZyL0XjXXGWPR2TTjrkmuSaQt55Ti2ASH7R5vlb92fL+XZv5b+ICuqt/CujWtnptrBZ7YdLuWurNfNc+VKwcFsk5PEr8HI+b2GI7zwdoV/a3UFzZMVur1b93jnkjkW4UKBIjqwaNsIBlCOMjucgHLXHjbxBZXN1oUsenT61HqlrYw3iQyJbFZ0MnmNFvLAqqONvmcnbyM1myeItV8J6l4wvdQS1vdSa5020he1t5fKZpR5aOYhvfjdkopYtjAOWGO3Hgfw//AGPLprWcjwzXC3Ukr3UrXDTKQVk88t5u9dq4bdkAADgYpIfAvhyHT9Qshp2+LUtn2xpZ5JJJmX7rmRmLbwcEPncCAc5AwAcD4g8R6rrHhu7s9WtZWS11bSXgv/7IudOjnD3keUEU5LblK8kMQQ69K9erCXwbo/2B7KcX93A88VwVvdTubgh4nDoQZJGK4YA4BAOOc1fsNFsNMvtQvLGDyp9SmE90+9j5jhQgOCcD5VAwMDigC9RRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB//9k=\" width=\"541\" height=\"83\"\u003e\u003c/span\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eEq.\u0026nbsp;2 describes the axial attention applied along the height axis of the tensor, and a similar formulation is also used to apply axial attention along the width axis after. Figure\u0026nbsp;2 illustrates the calculation process of axial attention along two directions.\u003c/p\u003e\n\u003cp\u003eDataset Details\u003c/p\u003e\n\u003cp\u003eThe main manifestations of RVO fundus are tortuous dilatation of the affected veins, flame-like haemorrhage along the retinal veins[\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e]. Each eye of the patient is treated as an independent classification sample. Multicolor(MC) image is one of the new technologies based on confocal scanning laser ophthalmoscope (cSLO), which can obtain multiple modal images with different wavelengths. We collected the results of the MC image examination from 29 patients as an internal dataset with a total of 220 images (some cases with missing images). The number was increased to 1320 by image enhancement strategies, including random cropping, rotation, horizontal flipping and color jitting. The ophthalmologist manually labelled them as RVO or not. These labels are treated as the ground truth for validating our algorithm. As shown in the figure below:\u003c/p\u003e\n\u003cp\u003eExperimental Setting\u003c/p\u003e\n\u003cp\u003eWe implement the proposed framework and perform experiments using the PyTorch library. All experiments are performed on a GPU cluster with four NVIDIA GeForce RTX 3090 GPUs and each with 24 GB of memory. The optimization process is run for 50 epochs. We set the learning rate as 0.0001, and the batch size is set as 12. All methods are optimized by the Adam optimizer. The cross-entropy function is selected as the classification loss function.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eOne of the research difficulties is that Transformer mechanism is a relatively new image processing technology introduced from the field of natural language processing, which is not as mature as the application of convolutional neural network in the field of computer vision. Moreover, the reason why Transformer has strong ability to learn long-term semantic information is that it does not assume to start from local information, but can get global information from the beginning, so it will be more difficult to learn. In view of these difficulties, the improved network structure in this paper requires a lot of experiments to constantly adjust the parameters to optimize the classification results.\u003c/p\u003e \u003cp\u003eLess pathological sample data is a major difficulty in medical image analysis task, which is also inevitable in this study. Although the number of data sets can be expanded through data enhancement, in order to avoid over fitting of classification model training, it is also necessary to balance data expansion and diversity. In addition, in order to improve the generalization ability of the model, we need to continue to find large public datasets for comparative experiments.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eVaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need[J].Advances in neural information processing systems,30\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarion N, Massa F, Synnaeve G et al (2020) End-to-end object detection with transformers[C]//European conference on computer vision. Springer, Cham, : 213\u0026ndash;229\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDosovitskiy A, Beyer L, Kolesnikov A et al An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv:2010.11929, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTouvron H, Cord M, Douze M et al (2021) Training data-efficient image transformers \u0026amp; distillation through attention[C]//International Conference on Machine Learning. PMLR, : 10347\u0026ndash;10357\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZheng S, Lu J, Zhao H et al (2021) Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. : 6881\u0026ndash;6890\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaghu M, Unterthiner T, Kornblith S et al (2021) Do vision transformers see like convolutional neural networks?[J]. Adv Neural Inf Process Syst 34:12116\u0026ndash;12128\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUsman M, Fraz MM, Barman SA (2017) Computer vision techniques applied for diagnostic analysis of retinal OCT images: a review[J]. Arch Comput Methods Eng 24(3):449\u0026ndash;465\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSingh A, Dutta MK, ParthaSarathi M et al (2016) Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image[J]. Comput Methods Programs Biomed 124:108\u0026ndash;120\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFu H, Cheng J, Xu Y et al (2018) Disc-aware ensemble network for glaucoma screening from fundus image[J]. IEEE Trans Med Imaging 37(11):2493\u0026ndash;2501\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSingh A, Dutta MK, ParthaSarathi M et al (2016) Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image[J]. Comput Methods Programs Biomed 124:108\u0026ndash;120\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi Z, He Y, Keel S et al (2018) Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs[J]. Ophthalmology 125(8):1199\u0026ndash;1206\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi A, Cheng J, Wong DWK et al (2016) Integrating holistic and local deep features for glaucoma classification[C]// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, :1328\u0026ndash;1331\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSzegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. : 2818\u0026ndash;2826\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKarri SPK, Chakraborty D, Chatterjee J (2017) Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration[J]. Biomedical Opt express 8(2):579\u0026ndash;592\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang J, Wang Z, Li F et al (2019) Joint retina segmentation and classification for early glaucoma diagnosis[J]. Biomedical Opt express 10(5):2639\u0026ndash;2656\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAcharya UR, Chua CK, Ng EYK et al (2008) Application of Higher Order Spectra for the Identification of Diabetes Retinopathy Stages[J]. J Med Syst 32(6):481\u0026ndash;488\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDu N, Li Y (2013) Automated identification of diabetic retinopathy stages using support vector machine[C]// Control Conference (CCC), 2013 32nd Chinese. IEEE, : 3882\u0026ndash;3886\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-2225149/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2225149/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe classification method based on convolutional neural network can improve the performance of the pattern recognition system by automatically extracting and self-learning features. At present, the conventional image classification network, including the network method designed in this paper, directly uses the common depth convolution network to directly extract and classify features. Therefore, the core of optimizing the experimental results is how to better extract the features in the image, that is, to extract more advanced and richer features. The advantage of Transformer is to capture the global context information in the way of attention, so as to establish a long-distance dependence on the target, so as to extract more powerful features.\u003c/p\u003e","manuscriptTitle":"ResTr: A Joint Framework for Retinal Vein Occlusion Image Classification","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-11-03 19:28:06","doi":"10.21203/rs.3.rs-2225149/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"9d5e78f9-f1f6-4a95-b395-3e21ee0dc6e4","owner":[],"postedDate":"November 3rd, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2022-11-08T00:14:18+00:00","versionOfRecord":[],"versionCreatedAt":"2022-11-03 19:28:06","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-2225149","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2225149","identity":"rs-2225149","version":["v1"]},"buildId":"ehx78VzkSd0WSzXnipQa-","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}