Feature Compensated Adaptive Fusion Shuffle Network for Image Super-Resolution Reconstruction

Feature Compensated Adaptive Fusion Shuffle Network for Image Super-Resolution Reconstruction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Feature Compensated Adaptive Fusion Shuffle Network for Image Super-Resolution Reconstruction Xiaoqiang Zhao, Xiyao Li This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2404952/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Recently, deep networks with multiple branches or different filtering methods have achieved good performance in image super-resolution, however, they ignore the fact that some high-frequency components are lost during multiple captures of contextual information by a single branch. To solve this problem, we propose a feature-compensated adaptive fusion shuffle network (CAFN) for image super-resolution. Specifically, we propose a feature-compensated shuffle model (FCSM). It uses branch compensation to compensate for the high frequency information lost by single channel feature transfer. Then, we propose an attention adaptive compensation fusion module (ACFM), which adaptively learns the weight information about the channel structure, and improves the branch compensation method to better adapt to the learned weight information. ACFM can further capture contextual dimension information and extract more high-frequency features. Experimental results show that our proposed CAFN has an excellent performance in terms of both visual results and evaluation metrics. Deep convolutional networks Attention mechanism Channel shuffle Deep separable convolution Feature compensation Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 1 Introduction In the process of image acquisition, affected by the image processing technology and transmission environment, the resolution of the obtained original image is usually low. However, the original low-resolution images lack key information, and it is usually difficult to meet the technical requirements of application scenarios. To solve this problem, super-resolution (SR) reconstruction, as a low-level computer vision task, has become a research hotspot. Single image super-resolution reconstruction (SISR) aims at the technique of recovering a high-resolution image (HR) from a low-resolution image (LR). The super-resolution reconstruction process is a typical ill-conditioned inverse problem [ 1 , 2 ], which means that an input LR image map has multiple output HR images. Although indicating that SISR is a difficult research area, the wide range of applications of SR in computer vision[ 3 ] makes it very valuable for research, such as urban surveillance[ 4 , 5 ], medical imaging[ 6 ], remote sensing images[ 7 , 8 ], etc. In recent years, the rapid development of deep learning techniques has shown the advantages for SISR. Dong et al [ 9 ] first applied convolutional neural network (CNN) to super-resolution reconstruction and proposed SRCNN, which constructed a 3-layer neural network for image reconstruction. Then, Dong et al [ 10 ] further proposed an accelerating super-resolution convolutional neural network (FSRCNN), which used deconvolution instead of bicubic for image feature up-sampling, and constructed a neural network with 8 convolutional layers. Shi et al [ 11 ] proposed an efficient sub-pixel convolutional neural network (ESPCN), it relied on sub-pixel convolution operation instead of deconvolution operation, which greatly improved the computational speed. Recently, many CNN-based SR methods have been proposed, mainly by constructing new connection methods and increasing the network depth of CNN. Although these methods can provide a larger receptive field and richer contextual information, they come with a large number of parameters that are difficult to train, as well as the problems such as gradient explosion and gradient disappearance. He et al [ 12 ] proposed residual network to make deep neural network training easy. Their use of skip connections in adjacent feature layers ensures there is no less feature information in the forward propagation process. Residual network solves the problem of gradient explosion in the training process and makes the network much deeper. Kim et al [ 13 ] obtained the residual information between LR and HR in the form of residual learning and proposed a very deep convolutional network super-resolution (VDSR), which constructed a 20-layer network and enableed feature information to be passed between shallow features and deep features. The emergence of residual network has greatly improved the depth of the network. Recently, many proposed deep convolutional neural networks use residuals, such as DRCN [ 14 ], DRRN [ 15 ], EDSR[ 16 ], RCAN [ 17 ], etc. SR networks vary in depth from 3 layers of SRCNN to over 400 layers of RCAN. Although deep networks exhibit state-of-the-art performance, these methods suffer from some limitations, for example, (1) Most methods tend to gain performance through deeper networks. However, with the increase of network depth, the number of parameters, training time, and required hardware memory would also increase, which would limit the application in real-world scenarios. (2) Most CNN-based SR models rarely optimize for convolutional filtering and pay little attention to the loss of information in a single stream during feature delivery. Various previous works have pointed out that network branching methods can increase the richness of feature extraction. They use different filtering methods for different branches to obtain information of different contexts, which improve the representativeness of the networks by fusing the different contextual information captured. Ren et al [ 18 ] proposed an image super-resolution based on fusing multiple convolution neural networks (CNF), which employed SRCNN in each branch. Li et al [ 19 ] proposed a multi-scale residual network (MSRN) for super-resolution, which used different convolution filters to extract features, combined local multi-scale features with global features, and fused residual features in layers to maximize the use of LR image features. Although, these methods extend the size of the receptive field by using different scale filtering methods for multiple branches. However, different branches ignore the loss of information during feature image filtering, which would lead to insufficient high-frequency reconstructed texture details. To address above problems, we propose a feature compensated adaptive fusion shuffle network (CAFN) for image super resolution reconstruction. CAFN adaptively compensates for the different background information captured by LR images passing through different filters. This compensation method can obtain richer high-frequency features, resulting in better reconstruction performance. On the one hand, in contrast to a purely branch network, the feature-compensated adaptive fusion shuffle network extracts richer contextual information by constructing two compensated branches to capture the missing feature information of a single path. On the other hand, compared to the traditional branching equivalent processing with branching feature extraction of different filters, CAFN uses attention adaptive fusion branching information to give different attention to the compensation branches of different filters, and saves fusion computation while obtaining better feature extraction results. The main contributions of this paper are as follows: (1) We propose a feature-compensated shuffle model (FCSM), which compensates for the part of the high-frequency information lost in the single-branch feature transfer process through the branch network. FCSM captures different compensation information from different dimensions of the input image and fuses the information of each branch to improve the ability of the model to learn complex mappings. (2) We propose an adaptive compensated fusion of channel structured attention model (ACFM), which adaptively learns the weight information of different channel structures. The compensation branch adaptively fuses the captured features and further focuses on the feature context dimension information. (3) We propose a Feature Compensated Adaptive Fusion Shuffle Network (CAFN), which adaptively adjusts the different contextual information captured by different two-way compensation branches to obtain better high frequency reconstruction performance. The overall structure of this paper is as follows. In Section 2 , we briefly review the works related to the proposed methods. In Section 3 , we describe the architecture of our proposed model in detail. In Section 4 , we present experimental details and datasets, and discuss the comparison of state-of-the-art models with our proposed model, as well as ablation studies and experimental results. Finally, in Section 5 we conclude the work of this paper. 2 Related Works 2.1 Depthwise separable convolution Depthwise separable convolution was proposed in Xception [ 20 ] and MobileNets [ 21 ]. Standard convolution can map all input feature images, and the number of feature channels is transformed with the number of convolution kernels during the convolution operation. The depthwise separable convolution divides standard convolution into two steps, namely deep convolution and point convolution. Compared with the standard convolution, for depthwise convolution, one input channel is convolved with one convolution kernel, and the number of convolution kernels remains the same as the number of channels in the previous layer. After one convolution operation, the size of the feature map and the number of channels do not change. Since the convolution operation is kept independent for each channel of the input layer, the feature information from the same space of different channels is not used, the whole convolution operation is done by post-convolution with Pointwise Convolution. Pointwise Convolution is similar to standard convolution in that the convolution size is 1x1 convolution, the convolution channel size is the output channel size of the previous layer, the convolution operations is weighted and expanded in depth, and a new feature map is output, and the output channel size remains the same as the convolution kernel channel size. Deeply separable convolution performs spatial convolution on each channel of the input independently of standard convolution, before performing point-by-point convolution, namely 1x1 convolution. By using depthwise separable convolution, the number of parameters and the amount of computation can be significantly reduced while achieving the same results as standard convolution. 2.2 ShuffleNet network ShuffleNet network is a highly flexible and high-performance deep network model, which reduces the high computational effort and computational complexity of convolutional operations by introducing pixel-level group convolution [ 22 ]. It uses the channel shuffle operation to complete the exchange of information between different channels, which helps to enhance the generalization ability and recognition accuracy of the model, and to solve the problem of missing feature representations. Compared to other networks, ShuffleNet network not only reduces the computational cost, but also maintains the accuracy. ShuffleNet consists of group convolution and channel-mixing operations. Group convolution was first used in AlexNet [ 23 ]. And Xception [ 20 ] proposed depthwise separable convolution based on group convolution. Although the computational effort is moderated compared to normal convolution, there is still the problem that the point-by-point convolution process consumes the computational effort of the model. Therefore, group convolution at the pixel level is proposed in the ShuffleNet network instead of 1x1 point-by-point convolution to reduce the computational complexity of the model. However, while reducing the complexity of the model by limiting the convolution operation to within-group convolution, multiple group convolutions only perform convolution stacking within this group, which can cause the output feature information to come from only a small part of the input features. The output features within a group are only related to the input features within that group, and there is no flow of feature information between different groups, limiting the exchange of features between group channels. For this purpose, a channel shuffle operation is proposed to allow group convolution to obtain input data from different groups, increasing the flow of feature information between groups. In particularly, channel blending takes each group of the previous feature map, divides it into several subgroups, and then reorganizes the subgroups into new groups for fusion, so that the output feature information includes the features from different groups. The process is shown in Fig. 1 : If an input convolution has groups, the output features have \(g \times n\) channels. The channel mixing operation converts the output channel number to \((g,n)\) , which is converted to \((n,g)\) by transposition. The channels are shuffled to flatten the transposed features as the input to the next layer. The channel shuffling operation allows for more flexible variations in the construction of multiple group convolutions, making it possible to build more complex networks. 2.3 Dynamic network Most deep learning networks learn in a static manner, in which network parameters remain consistent before and after training, which can limit their representational power and efficiency. In contrast to static network, dynamic network can be selectively activated for model components by adaptively adjusting its structure or parameters according to the input during the inference process. Therefore, it has more efficient reasoning and learning ability. Dynamic network can improve model capacity by applying feature conditional attention weights on the aggregation of the convolution kernels [ 24 ], [ 25 ] to achieve an optimal balance between accuracy and efficiency. The core idea of dynamic network is to adaptively adjust the convolution parameters based on the input image. It dynamically learns the attention vector based on the input and obtains a new convolution kernel by weighted aggregation of multiple parallel convolution kernels from the attention learning weights. This non-linear convolutional aggregation can significantly improve the expressive power of the network with the introduction of a very small amount of calculation. This is expressed as follows: $$y=h(\mathop {{W^T}}\limits^{\sim } (x)x+\mathop b\limits^{\sim } (x))$$ 1 $$\mathop W\limits^{\sim } (x)=\sum\limits_{{k=1}}^{N} {{\pi _k}} (x){\mathop W\limits^{\sim } _k},\mathop b\limits^{\sim } (x)=\sum\limits_{{k=1}}^{N} {{\pi _k}} (x){\mathop b\limits^{\sim } _k}$$ 2 $$s.t.{\text{ 0}} \leqslant {\pi _k}(x) \leqslant 1,\sum\limits_{{k=1}}^{N} {{\pi _k}} (x)=1$$ 3 where \({\pi _k}\) denotes the attention weight of the k-th filter, which changes by depending on each input , \({\mathop W\limits^{\sim } _k}\) is the convolutional aggregation weight, \({\mathop b\limits^{\sim } _k}\) represents the bias, and \(\mathop W\limits^{\sim } (x)\) and \(\mathop b\limits^{\sim } (x)\) indicate the kernel weights and bias containing the attention weights. In recent years, dynamic networks have more and more attracted the attention of researchers. Jia et al [ 26 ] first introduced dynamic learning to deep learning by proposing a network that dynamically generates convolutions based on inputs. Yang et al [ 24 ] proposed a conditional parameterized convolutions for efficient inference (Condconv), which learned a specific convolution kernel parameter for each feature, generated kernel weights based on the input samples, and dynamically selected convolution kernels to improve model capacity. Chen et al [ 25 ] proposed a dynamic convolution, which enhanced model representation by fusing multiple parallel convolution kernels. Zhang et al [ 27 ] proposed an accelerated convolutional neural network with dynamic convolution (DyNet), which adaptively generated convolutional kernels based on the image content and reduced the amount of redundant information that existed between convolutional kernels. 3. The Proposed Method Although the image super-resolution reconstruction based on deep learning has been well developed, there are still problems, such as increasing hardware requirements due to larger network volumes and difficulties in training due to the deepening complexity of the network structure. Deep learning training would increase the number of parameters and computational complexity, they not only affect the memory consumption of hardware facilities, but also reduce the training effect. In order to maintain a dynamic balance between the number of parameter and the training effect of the network, we propose a feature compensated adaptive fusion shuffle network for image super-resolution reconstruction. First, we propose a feature compensated shuffle model (FCSM), which uses depthwise separable convolution to reduce the number of parameters for the main convolution. FCSM has improved the problem of single-stream networks that could lead to partial feature loss. Secondly, we propose an adaptive compensation fusion of channel structured attention model (ACFM), which consists of an adaptive weight learning block focusing on the channel structure, and an improved feature compensated shuffle model. The weight learning block adaptively learns the image weight information, and the improved feature compensation adjusts FCSM to reduce the redundant information generated during extraction. Finally, the feature information from the two modules is fused and stacked with the above operations. We reconstruct the captured deep feature information to generate super-resolution images. 3.1 Feature compensated shuffle network In recent years, many scholars have proposed many neural networks for image super-resolution reconstruction, such as convolutional neural networks [ 28 ], residual networks[ 12 ] and dense networks [ 29 ]. The image super-resolution reconstruction based on these networks can achieve better reconstruction results, but there are still the problems of large network models and more parameters. Moreover, the network performance is positively correlated with the model size, so it is difficult to obtain better performance by shrinking the model. During the feature extraction process of the model, continuous filtering would gradually lose some high-frequency information. If this loss can be compensated for by branching compensation features, more image information can be captured under the condition of limited expansion of the model. Also, depthwise separable convolution can be used instead of standard convolution to minimize the number of model parameters. Based on the above purpose, we propose feature-compensated shuffle network. As shown in Fig. 2 , the network mainly consists of a split-channel compensation branch and a multi-channel compensation branch. Both the split-channel compensation and multi-channel compensation are composed of a channel group extraction module and a channel shuffling module. The channel grouping extraction module consists of multiple grouping convolutions and depth-separable convolutions. The split-channel compensation branch compresses part of the feature information, reducing the number of parameters in the feature capture process, and the multi-channel compensation branch extends the channels to increase their feature complexity. For the doubled parameter amount of the multi-channel compensation branch, the split-channel compensation branch compression extraction can effectively reduce the total parameters and computational effort of compensation fusion. The detailed operation of this network is shown as follows: (1) Split-channel compensation The network feeds the input features into a split-channel compensation branch for split-channel feature extraction. In the split-channel compensation branch, we perform a channel split on the input features, the dimensional input features are divided into two \(m/2\) dimensional ones for two-way feature extraction. The left branch maintains \(m/2\) dimensions to achieve the retention of the initial features. The right branch is used as a compensating branch to fully extract the feature information. First, the \(m/2\) dimensional features are extracted by group convolution, and then the depthwise separable convolution is used to extract features, and finally the group convolution is used to extract features to obtain richer feature information. In order to reduce the computational effort, the size of the group convolution is all 1x1. Afterwards, the features of the two branches are fused to compensate for the feature information and return the number of channels to the dimension before splitting, and a channel shuffle is performed. The specific operation is described below: $${F_s}=H_{{m/2}}^{1}(X)=H_{{m/2}}^{2}(X)$$ 4 where \(H_{{m/2}}^{1}( \cdot )\) and \(H_{{m/2}}^{2}( \cdot )\) indicate channel splitting operations, \({F_s}\) indicates the feature map for channel splitting, $${E_k}=(\sigma (C_{N}^{1}({F_s})))$$ 5 where \(C_{N}^{1}( \cdot )\) is \(1 \times 1\) standard convolution and \(\sigma ( \cdot )\) represents the Relu activation function, \({E_k}\) is for retained branch feature map, $${E_t}=\sigma (C_{{G2}}^{1}(\sigma (D_{{SC}}^{1}(\sigma (C_{{G1}}^{1}({F_s}))))))$$ 6 where \(C_{{G1}}^{1}( \cdot )\) and \(C_{{G2}}^{1}( \cdot )\) are the two previous group convolution operations, and \(D_{{SC}}^{1}( \cdot )\) represents the depth-separable convolution. Afterwards, the obtained feature channel correlation information \({E_t}\) is used to compensate for the reserved branch features and the channel dimension, which would be recovered. $${F_{Of}}={H_{m - cat}}({E_k},{E_t})$$ 7 where the \({H_{m - cat}}( \cdot )\) feature dimension is recovered as a dimensional merge operation, \({F_{Of}}\) represents the feature map for channel dimension recovery, $${F_{OH}}={}_{h}H_{{CS}}^{n}({F_{Of}})$$ 8 where \({}_{h}H_{{CS}}^{n}( \cdot )\) is the channel wash, is the wash group factor, \({F_{OH}}\) is the split channel compensation output feature. (2) Multi-channel compensation Since the split-channel compensation branch obtains the features through branch processing and fusion extraction in the case of splitting part of the channels, the features have insufficient information. If these features were used directly to reconstruct SR images, the image reconstruction would not be sufficiently detailed, so we construct multi-channel compensation branch to extend and complement these features, and different depths of extraction are used for the features in different branches. Similar to the split-channel compensation branch, the multi-channel compensation branch also uses two channels for feature extraction, which keeps the number of channels at for both the left and right branches for a dimensional feature input. In order to keep the image size constant, a convolution with a stride size of 1 is always used to extract the features. In contrast to the left branch of the split-channel compensation branch, which maintains the feature information, the left branch of our multi-channel branch uses two layers of feature extraction with constant dimensionality of the feature information. The non-linearity of the features is performed by end-to-end group convolution with depthwise separable convolution extraction by using a depthwise separable convolution of size \(3 \times 3\) . We use a stride size of 1 to ensure that the size of its feature map does not change. The right branch is used as a compensation branch to ensure that the number of input channels is the same as the number of branch channels. The right branch is used as a compensation branch, identical to the compensation branch for sub-channel compensation, to enrich the features. Similarly, the left and right branches are fused to obtain a feature map of channel dimension \(2m\) . In order to recover the number of channels, we use a channel compression convolution operation to recover the channels to dimensions. This is followed by a channel shuffle operation, as follows: $${F_d}=N_{m}^{1}(X)=N_{m}^{2}(X)$$ 9 where \(N_{m}^{1}( \cdot )\) and \(N_{m}^{2}( \cdot )\) denote that the channel is held and features are extracted once. \({F_d}\) means the extracted initial feature map, $${E_l}=\sigma (C_{{G1}}^{2}(\sigma (D_{{SC}}^{2}({F_d}))))$$ 10 where \(D_{{SC}}^{2}( \cdot )\) denotes the depthwise separable convolution, \(\sigma ( \cdot )\) denotes the Relu activation, and \(C_{{G1}}^{2}( \cdot )\) denotes the left-branch group convolution, \({E_l}\) means the feature map extracted from the left branch, $${E_r}=\sigma (C_{{G2}}^{3}(\sigma (D_{{SC}}^{3}(\sigma (C_{{G1}}^{3}({F_d}))))))$$ 11 where \(C_{{G1}}^{3}( \cdot )\) and \(C_{{G2}}^{3}( \cdot )\) are the two group convolution operations on the right branch, \(D_{{SC}}^{3}( \cdot )\) represents the depth-separable convolution. \({E_r}\) means the feature map extracted from the right branch. We propose multi-channel compensation that has a left-right branch structure. This structure has more non-linear mappings, which can better fit the complex relationship between channels. Subsequently, we fuse the feature information from both the left and right branches to complement the information. We use the \(concat\) merge operation to retain all extracted features and double the channel dimension. Afterwards, we access the \(3 \times 3\) compression convolution to place the channel dimension as the input dimension. We use channel shuffle to increase the interaction and fusion between different groups of channels, to correct for high frequency information: $${F_{Ob}}={H_{2m - cat}}({E_l},{E_r})$$ 12 where the \({H_{2m - cat}}( \cdot )\) feature dimension is multiplied to the \(2m\) dimension \(concat\) merge operation. \({F_{Ob}}\) means dimension doubled feature map, $${F_{OD}}={}_{d}H_{{CS}}^{n}(C_{{G3}}^{3}({F_{Ob}}))$$ 13 where \(C_{{G3}}^{3}( \cdot )\) is the channel compression convolution, \({}_{d}H_{{CS}}^{n}( \cdot )\) is the channel blending, is the blending group factor, and \({F_{OD}}\) is the split-channel compensated output feature. Finally, a superposition operation \({H_{Add}}( \cdot )\) is used to fuse the split-channel and multi-channel compensation features to obtain an output \({F_{OUT}}\) , which completes a feature-compensated shuffle operation. $${F_{OUT}}={H_{Add}}({F_{OD}},{F_{OH}})$$ 14 3.2 Feature Compensated Adaptive Fusion Shuffle Network In the feature compensated shuffle network, we compress and multiply the extracted model by split-channel compensation and multi-channel compensation to supplement the missing feature information and improve the accuracy. However, due to the branching path processing feature, it increases the fragmentation of the network, introduces extra overhead during kernel startup and synchronicity, and results in excessive pressure on parallel computation. It is difficult for us to maintain computational speed at deeper network layers, and even over-fitting can occur, resulting in inability to compute. At the same time, FCSM completely overlays and fuses the feature information extracted by split-channel compensation and multi-channel compensation, without selective extraction according to its own structural characteristics, which would cause redundant feature information to occupy excessive computing space and make it difficult accurately recover useful feature information. The influence of this situation would increase with the deepening of the number of training layers. In order to solve these problems, we propose a feature-compensated adaptive fusion shuffle network model, whose structure is shown in Fig. 3 . The network is structured to control that both compensation branches maintain the same number of channels, and the learning of the target features is reinforced by the neural network with a soft attention module. Adaptive attention learning is used to calculate the weights and dynamically adjust the weight ratio of the two compensated branches to improve the network representation. The attention module is a weighted attention module that extracts useful information and suppresses useless information by focusing on the regions of interest features. SEnet [ 30 ] focuses on the different weights of the channels and SAnet [ 31 ] focuses on the target spatial location to obtain the weights. However, these attention modules only focus on the information changes before and after the modules in the network, and are unable to focus on the structural information differences between modules. In a branching network structure, the different extraction methods of the two branches would produce different representation information. The depth of the network, the filtering method and the number of branching channels would result in different perceptual fields of information being extracted from the branches. Aggregation of these different information results in more complex feature information. If the extracted image features were aggregated at the same level of attention without regard to structural differences, extensive computational resources would be spent on the redundant feature extraction. In this paper, in order to optimize the accuracy of the fused features of the branched information, we hope that different branch structures have different information attention, so a novel adaptive compensated fusion of attention of channel structured Attention Model (ACFM) is proposed, which consists of an adaptive weight learning module and an attention network module with feature-compensated shuffle. The adaptive weight learning module performs the initial processing of the learned features through the residual blocks, which are then connected end-to-end through a pooling layer, a fully connected layer and a softmax function. The connections are shown in Fig. 3 , a feature-compensated shuffle module with attention, where image features are extracted from the residual blocks, after which the features are passed into two separate compensation branches. In order to adapt to the fusion of learning weights, the two compensation branches are further improved to increase their representation capabilities. The adaptive weight learning module assigns adaptive learning weights to the two compensated branch extracted feature images. Finally, the two features are fused to obtain mixed feature information focusing on the differences in the structure of the different modules. The ACFM process can be divided into three steps: Step1: The input features are extracted by the residual block to obtain the image feature information \({F_u}\) , the residual block primary processes the input features and enriches the extraction of details, and its mathematical process can be described as follows: $${F_u}={R_o}(X)$$ 15 where \({R_o}( \cdot )\) stands for the residual block processing. Step2: The adaptive weight learning module compresses the dimension of the image features \({F_u}\) by global averaging pooling, which compresses the spatial dimensional information into the channel and aggregates the input feature map into a vector of \(1 \times 1 \times {C_i}\) , this vector is then fed into the end-connected activation layer of the fully connected layer for operations in the end-connected fully connected layer, and finally the features obtained from these operations are fed into the Softmax activation function. The Softmax activation function is to map the input between [0, 1] and ensure that the normalized sum is 1, that is, the adaptive weight information \({\lambda _n}(x)\) of the attention of the structure is obtained. In the adaptive weight learning module, adaptive learning is performed according to the input image features, and the relevant weight coefficients are calculated. The weights would be adaptively adjusted according to the network structure. In order to prevent the model from being too computationally intensive, the kernel space is compressed through aggregation constraints, as shown in formula (16): $${\lambda _n}(x)=\delta (\mu (\sigma (\mu (\varphi ({F_u}))))),{\text{ }}0 \leqslant {\lambda _n}(x) \leqslant 1,{\text{ }}1=\sum\limits_{{n=1}}^{z} {{\lambda _n}} (x)$$ 16 where \(\varphi ( \cdot )\) represents the global average pooling, \(\mu\) represents the fully connected operation, \(\sigma\) represents the Relu activation function, and \(\delta\) represents the Softmax activation function. The attention network module with feature-compensated shuffle performs a convolution of the initial feature information \({F_u}\) , which is then fed into two compensation branches to obtain the channel compensation information. This is followed by a convolution activation mapping, where the activation function uses LeakyReLu to activate more neurons to extend the subsequent module to capture more feature rich information. The channel weights are then selected by Senet attention to obtain the split-channel compensation feature \({V_1}\) and the multi-channel compensation feature \({V_2}\) , which contain the channel weight shares. The mathematical model can be expressed as: $${V_1}=S_{E}^{1}(\nu (C_{{co}}^{2}({S_{CC}}(C_{{co}}^{1}({F_u})))))$$ 17 $${V_2}=S_{E}^{2}(\nu (C_{{co}}^{2}({M_{CC}}(C_{{co}}^{1}({F_u})))))$$ 18 where \(C_{{co}}^{1}( \cdot )\) is a feature mapping operation, \({S_{CC}}( \cdot )\) and \({M_{CC}}( \cdot )\) represent the split-channel compensation and multi-channel compensation respectively, \(C_{{co}}^{2}( \cdot )\) is the convolution used to extract global features, \(\nu\) denotes the LeakyReLu activation function, \(S_{E}^{1}( \cdot )\) and \(S_{E}^{2}( \cdot )\) denotes the SEnet attention modules. Step3: The information of \({V_1}\) and \({V_2}\) is aggregated, and fusion uses the adaptive weight learning model learning weights and the attention network model with feature compensation shuffle function of compensation branch by branch multiplication, in the multiplication of the resulting features containing weight accounted for aggregation. The features containing concerns about the adaptive structure of the network are obtained. $$U={V_1}\cdot {\lambda _1}(x)+{V_2}\cdot {\lambda _2}(x)$$ 19 $$1={\lambda _1}(x)+{\lambda _2}(x)$$ 20 where \({\lambda _1}(x)\) and \({\lambda _2}(x)\) represent the weight components learned by the adaptive weight learning model, and \(\cdot\) represents the corresponding multiplication between the compensated branch structure and the adaptive weights. 3.3 Network Framework We propose a feature compensated adaptive fusion shuffle network for image super resolution reconstruction (CAFN), its model framework is shown in Fig. 4 . The network framework consists of four parts, namely, shallow feature extraction, deep feature extraction, upsampling and image reconstruction. If \({I_{LR}}\) and \({I_{HR}}\) are the input low-resolution image and the output reconstructed image, respectively, the network model first extracts the shallow features in the low-resolution image by using the primary convolution block. $${F_0}={H_I}\left( {{I_{LR}}} \right)$$ 21 where \({H_I}\) represents the mapping function of the initial convolutional layer, and \({F_0}\) represents the extracted shallow feature. Shallow features capture the deep features of the image through a shuffling network of feature compensation adaptive fusion, namely: $${F_f}={F_0}+{f_f}({C_M}( \cdot \cdot \cdot ({C_m}({C_{m - 1}}( \cdot \cdot \cdot {C_1}({F_0}) \cdot \cdot \cdot ))) \cdot \cdot \cdot ))$$ 22 where, \({F_0}\) performs feature update through ACFMs and a convolutional layer, and then combines the updated features and shallow features \({F_0}\) through global residual learning. Specifically, \({C_m}( \cdot )\) represents the feature extraction operation of the m-th ACFM, \({f_f}( \cdot )\) represents the convolution operation of end-to-end convolution, and \({F_f}\) represents the extracted deep features. At this point, \({F_f}\) is input to the up-sampling module, and the up-sampling module amplifies the combined feature output to the desired SR image size, namely: $${F_{UP}}={H_{UP}}({F_f})$$ 23 After the reconstruction of \({F_{UP}}\) by the reconstruction module, the corresponding SR image is obtained, $${I_{SR}}={H_{RE}}({F_{UP}})$$ 24 where \({H_{RE}}( \cdot )\) denotes the reconstruction mapping function. 4 Experimental Results And Analysis 4.1 Experimental setup For the training phase, this paper uses the DIV2K [ 32 ] dataset as our training dataset and augments the training dataset with 90°, 180°, 270° rotations and random horizontal flips. The LR image is cropped to a block of size \(48 \times 48\) pixels and the HR image is cropped to a block of size \(48s \times 48s\) pixels, where denotes the magnification factor. In the testing phase, this paper adopts four commonly used standard datasets: Set5[ 33 ], Set14[ 34 ], BSD100[ 35 ], Urban100[ 36 ]. During the test, the image color space is converted from RGB to YCbCr, and PSNR and SSIM are calculated in the Y channel. In this paper, the network is optimized by the Adam algorithm. When the network weight is updated, its exponential decay rate is set to \({\beta _1}=0.9\) , \({\beta _2}=0.999\) , and the numerically stable small constant \(\varepsilon ={10^{ - 8}}\) . We use L 1 as the loss function, the training iterations are 1000 (epoch), the batch size is 16, and the learning rate is \({10^{ - 4}}\) . The network framework construction, training and testing of this experiment are carried out on Ubuntu18.04 system, the programming platform is Pytorch1.2, the experimental GPU is NVIDIA GeForce RTX2080Ti, the CPU is Inter(R) Core (TM) i9-9900K, and the memory size is 64G. 4.2 Experiment evaluation index In this paper, Peak Signal to Noise Ratio (PSNR) and Structural Simi-Larity (SSIM) are used as evaluation indicators. The size of PSNR is related to the mean square error of the current image and the reference image and the maximum pixel value. The calculation form is simple and the physical meaning is clear; SSIM is a full-reference image quality indicator that evaluates image similarity from three aspects: brightness, contrast, and structure. It simulates the visual function of the abstract object structure of the human visual system as a whole, and conforms to visual perception. 4.3 Experimental results and analysis To verify the effectiveness of our algorithm, we use the DIV2K dataset to train CAFN, and compare the proposed algorithm model with Bicubic, SRCNN[ 9 ], FSRCNN[ 10 ], VDSR[ 13 ], DRCN[ 14 ], LapSRN[ 37 ], DRRN[ 15 ], MSRN [ 19 ], CARN [ 38 ], OISR [ 39 ], SeaNet [ 40 ]. Table 1 shows the evaluation index values of different scale factors of different algorithms on the four test data sets of Set5, Set14, BSD100 and Urban100, where the best and second best performing algorithms are bolded and underlined, respectively. As it can be seen from Table 1 , our proposed CAFN achieves optimal and sub-optimal PSNR and SSIM values for quantitative evaluation at 2、3 and 4 times magnification compared to the other algorithms mentioned above. For example, on the Set5 test set at x3 magnification, PSNR and SSIM of CAFN achieve 0.07dB and 0.007 higher r than that of the suboptimal OISR; When the magnification is 4, PSNR of CAFN achieves the highest values on all four test datasets, and SSIM values are the highest on the Set5, B100 and Urban100 datasets. In particular, PSNR and SSIM of our proposed method are 0. 12dB and 0.0014 higher on Set5 than that of the suboptimal SeaNet, while on Urban100 our proposed CAFN has a PSNR and SSIM 0.13dB and 0.0017 higher than the suboptimal results obtained by OISR. The reason is that CAFN compensates for part of the contextual information lost in the single-flow network pass, especially the high-frequency components, and we can obtain better reconstruction results in image data that focus on capturing feature integrity. Table 1 Quantitative evaluation of 12 SR methods tested on four benchmark sets at different magnifications Method Scale Parameters Set5 Set14 BSD100 Urban100 PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM Bicubic 2 33.66/0.9299 30.24/0.8688 29.56/0.8431 26.88/0.8403 SRCNN 57K 36.66/0.9542 32.42/0.9063 31.36/0.8879 29.50/0.8946 FSRCNN 12k 37.00/0.9559 32.75/0.9098 31.51/0.8939 29.87/0.9065 VDSR 665K 37.53/0.9587 33.03/0.9124 31.90/0.8960 30.77/0.9140 DRCN 1774K 37.63/0.9588 33.08/0.9118 31.08/0.8942 30.41/0.9133 LapSRN 812K 37.52/0.9581 33.08/0.9109 31.80/0.8949 30.41/0.9112 DRRN 297K 37.74/0.9591 33.23/0.9136 32.05/0.8973 31.23/0.9188 MSRN 5890K 38.08/0.9605 33.74/0.9170 32.23/0.9013 32.22/0.9326 CARN 1592K 37.76/0.9590 33.52/0.9166 32.09/0.8978 31.92/0.9256 OISR 4970K 38.12/0.9609 33.78/0.9196 32.26/0.9007 32.52/0.9320 SeaNet 7102K 38.08/0.9606 33.75/0.9190 32.27/0.9008 32.50/0.9318 CAFN 4755K 38.14/0.9610 33.76/0.9192 32.24/0.9003 32.47/0.9313 Bicubic 3 30.39/0.8682 27.55/0.7742 27.21/0.7385 24.46/0.7349 SRCNN 57K 32.75/0.9090 29.28/0.8209 28.41/0.7863 26.24/0.7989 FSRCNN 12K 33.02/0.9135 29.49/0.8271 28.50/0.7937 26.41/0.8161 VDSR 665K 33.66/0.9213 29.78/0.8314 28.82/0.7976 27.14/0.8279 DRCN 1774K 33.82/0.9226 29.79/0.8311 28.82/0.7963 27.07/0.8276 LapSRN 812K 33.82/0.9207 29.89/0.8304 28.82/0.7950 27.07/0.8298 DRRN 297K 34.03/0.9244 29.96/0.8349 28.95/0.8004 27.53/0.8378 MSRN 6080K 34.38/0.9262 30.34/0.8395 29.08/0.8041 28.08/0.8554 CARN 1592K 34.29/0.9255 30.29/0.8407 29.06/0.8034 28.06/0.8493 OISR 5640K 34.56/0.9284 30.46/0.8450 29.20/0.8077 28.56/0.8606 SeaNet 7471K 34.55/0.9282 30.42/0.8444 29.17/0.8071 28.50/0.8594 CAFN 4939K 34.63/0.9289 30.67/0.8469 29.19/0.8071 28.54/0.8596 Bicubic 4 28.42/0.8104 26.00/0.7027 25.96/0.6675 23.14/0.6577 SRCNN 57K 30.48/0.8628 27.49/0.7503 26.90/0.7101 24.52/0.7221 FSRCNN 12K 30.66/0.8646 27.71/0.7562 26.98/0.7124 24.60/0.7221 VDSR 665K 31.35/0.8838 28.02/0.7674 27.29/0.7251 25.18/0.7524 DRCN 1774K 31.35/0.8854 28.19/0.7670 27.23/0.7233 25.14/0.7510 LapSRN 812K 31.54/0.8811 28.19/0.7635 27.32/0.7162 25.21/0.7564 DRRN 297K 31.68/0.8888 28.21/0.7720 27.38/0.7284 25.44/0.7638 MSRN 6330K 32.07/0.8903 28.60/0.7751 27.52/0.7273 26.04/0.7896 CARN 1592K 32.13/0.8937 28.60/0.7806 27.58/0.7349 26.07/0.7837 OISR 5500K 32.33/0.8968 28.73 / 0.7845 27.66/0.7389 26.37/0.7953 SeaNet 7397K 32.33/0.8970 28.72/ 0.7855 27.65/0.7388 26.32/0.7942 CAFN 4902K 32.45/0.8984 28.77 / 0.7852 27.67 / 0.7389 26.45/0.7959 In order to further evaluate the visual quality of CAFN reconstruction, this paper selects Image011,Image024,and Image030 in the Urban100 dataset under magnification to compare the visual effects of other advanced SR methods, as shown in Fig. 5 . From Fig. 5 , we can see that the SR images reconstructed by other advanced methods suffer from blur, artifacts and distortion. Especially for Image024 images, other advanced methods appear distorted, and CAFN is closer to the HR images in the details of reconstructed SR images; in Image011 and Image030, the reconstructed details have serious line blur problems, and our proposed CAFN is clearer, the lines and details of the LR image are reconstructed without blurring and distortion problems. We also evaluate our proposed CAFN in comparison with other state-of-the-art SR methods in terms of parameter quantities. Figure 6 shows the comparative test analysis results between the parameter amount and PSNR value in the Urban100 test set under \(\times 4\) magnification. From Fig. 6 , we can analyze that our proposed CAFN has the highest PSNR value on Urban100, while we propose CAFN with a lower number of parameters compared to OISR, SeaNet and MSRN methods. Compared with SRCNN, VDSR, DRCN, DRRN and CARN, CAFN has more parameters, but it has achieved great improvement in visual quality and quantitative evaluation. Specifically, our proposed CAFN is better than SRCNN, VDSR, DRCN, DRRN and CARN are improved by 1.93dB, 1.27dB, 1.31dB, 1.01dB and 0.38dB, respectively. 4.4 Ablation Study To verify the effect of ACFM, we conduct ablation experiments of ACFM to verify the effect of each block, we gradually modify our model, and compare their differences. As shown in Table 2 , in order to show the effect of our proposed feature compensation module and adaptive weight learning module, we denote the network with the adaptive weight learning module removed by ACFM-C, the network with the feature compensation attention module removed by ACFM-A, and the network with the adaptive weight learning module and feature compensation attention module removed by ACFM-N. ACFM is trained with the above networks under the same training details and convolutional activation layers. In particular, we set the same supervised training model at 595 epochs, and in Table 2 , we find that our model also performs better at low training counts. Table 2 Average PSNR and SSIM of \(\times 4\) magnification factor for Attention Adaptive Compensation Fusion (ACFM) of Channel Structure Model Adaptive Weight Learning Module Feature Compensation Attention Module Set5 PSNR/SSIM Set14 PSNR/SSIM BSD100 PSNR/SSIM Urban100 PSNR/SSIM ACFM-N × × 32.27/0.8954 28.64/0.7815 27.57/0.7357 26.13/0.7850 ACFM-C × √ 32.43/0.8980 28.75/0.7843 27.62/0.7388 26.43/0.7955 ACFM-A √ × 32.42/0.8976 28.73/0.7848 27.62/0.7383 26.39/0.7954 ACFM √ √ 32.45/0.8981 28.75/0.7847 27.62/0.7392 26.40/0.7959 From the iterative results of the loss function during training in Fig. 7 , it can be seen that our proposed ACFM has a faster convergence rate and a lower loss value after convergence than the comparative experiments such as ACFM-N, ACFM-C, and ACFM-A. According to the analysis of objective evaluation indicators in Fig. 8 , it can be concluded that on the Set5 data set, for \(\times 4\) magnification, ACFM obtains a better PSNR value than the comparison methods, which verifies that ACFM has better reconstruction and magnification performance. 5 Conclusion In this paper, we propose a feature compensated adaptive fusion shuffle network (CAFN) for image super-resolution reconstruction. Specifically, CAFN captures contextual feature information by using branch-way feature-compensated shuffled attention model. In addition, CAFN uses an adaptive attention model to attention to the channel structure. It is adaptive in learning the different weight components obtained from different channel structures and enables more efficient attention to feature information of interest. CAFN to compensate for the high frequency components of the information captured by the two branches of the shuffled attention model to obtain more accurate high frequency features. Compared to other state-of-the-art methods, our proposed CAFN achieves competitive performance in terms of image visual quality assessment and quantitative evaluation. Declarations Funding and/or conflicts of interests/Competing interests The authors declare that they have no conflicts of interest to this work. The people involved in the experiment have been informed and formally accepted. Acknowledgments This research work has been awarded by the National Natural Science Foundation of China (62263021), The Science and Technology Project of Gansu Province (21JR7RA206, 21YF5GA072), the Open Fund project of Gansu Provincial Key Laboratory of Advanced Control for Industrial Process (2022KX07), the Industrial Support Project of Education Department of Gansu Province (2021CYZC-02). References J. Yu, X. Gao, D. Tao, X. Li, and K. Zhang, "A unified learning framework for single image super-resolution," IEEE Transactions on Neural networks and Learning systems, vol. 25, no. 4, pp. 780-792, 2013. M. Zhang, J. Xin, J. Zhang, D. Tao, and X. Gao, "Curvature Consistent Network for Microscope Chip Image Super-Resolution," IEEE Transactions on Neural Networks and Learning Systems, 2022. K. Li, S. Yang, R. Dong, X. Wang, and J. Huang, "Survey of single image super‐resolution reconstruction," IET Image Processing, vol. 14, no. 11, pp. 2273-2290, 2020. S. Jung and J. Kim, "Adaptive and stabilized real-time super-resolution control for UAV-assisted smart harbor surveillance platforms," Journal of Real-Time Image Processing, vol. 18, no. 5, pp. 1815-1825, 2021. Y. Lee, J. Yun, Y. Hong, J. Lee, and M. Jeon, "Accurate license plate recognition and super-resolution using a generative adversarial networks on traffic surveillance video," in 2018 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia) , 2018, pp. 1-4: IEEE. Y. Li, B. Sixou, and F. Peyrin, "A review of the deep learning methods for medical images super resolution problems," Irbm, vol. 42, no. 2, pp. 120-133, 2021. S. Zhang, Q. Yuan, J. Li, J. Sun, and X. Zhang, "Scene-adaptive remote sensing image super-resolution using a multiscale attention network," IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 7, pp. 4764-4779, 2020. D. Zhang, J. Shao, X. Li, and H. T. Shen, "Remote sensing image super-resolution via mixed high-order attention network," IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 6, pp. 5183-5196, 2020. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295-307, 2015. C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in European conference on computer vision , 2016, pp. 391-407: Springer. W. Shi et al. , "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 1874-1883. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770-778. J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 1646-1654. J. Kim, J. K. Lee, and K. M. Lee, "Deeply-recursive convolutional network for image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 1637-1645. Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual network," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 3147-3155. B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, "Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops , 2017, pp. 136-144. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 286-301. H. Ren, M. El-Khamy, and J. Lee, "Image super resolution based on fusing multiple convolution neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops , 2017, pp. 54-61. J. Li, F. Fang, K. Mei, and G. Zhang, "Multi-scale residual network for image super-resolution," in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 517-532. F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 1251-1258. A. G. Howard et al. , "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017. X. Zhang, X. Zhou, M. Lin, and J. Sun, "Shufflenet: An extremely efficient convolutional neural network for mobile devices," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 6848-6856. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017. B. Yang, G. Bender, Q. V. Le, and J. Ngiam, "Condconv: Conditionally parameterized convolutions for efficient inference," Advances in Neural Information Processing Systems, vol. 32, 2019. Y. Chen, X. Dai, M. Liu, D. Chen, L. Yuan, and Z. Liu, "Dynamic convolution: Attention over convolution kernels," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 11030-11039. X. Jia, B. De Brabandere, T. Tuytelaars, and L. Van Gool, "Dynamic filter networks for predicting unobserved views," in Proceedings ECCV 2016 workshops , 2016, pp. 1-2. Y. Zhang, J. Zhang, Q. Wang, and Z. Zhong, "Dynet: Dynamic convolution for accelerating convolutional neural networks," arXiv preprint arXiv:2004.10694, 2020. J. Gu et al. , "Recent advances in convolutional neural networks," Pattern recognition, vol. 77, pp. 354-377, 2018. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 2472-2481. J. Hu, L. Shen, and G. Sun, "Squeeze-and-excitation networks," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 7132-7141. S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, "Cbam: Convolutional block attention module," in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3-19. E. Agustsson and R. Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops , 2017, pp. 126-135. M. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. A. Morel, "Neighbor embedding based single-image super-resolution using semi-nonnegative matrix factorization," in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2012, pp. 1289-1292: IEEE. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in International conference on curves and surfaces , 2010, pp. 711-730: Springer. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation," IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 5, pp. 898-916, 2010. J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 5197-5206. W.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, "Deep laplacian pyramid networks for fast and accurate super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 624-632. N. Ahn, B. Kang, and K.-A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 252-268. X. He, Z. Mo, P. Wang, Y. Liu, M. Yang, and J. Cheng, "Ode-inspired network design for single image super-resolution," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 1732-1741. F. Fang, J. Li, and T. Zeng, "Soft-edge assisted network for single image super-resolution," IEEE Transactions on Image Processing, vol. 29, pp. 4656-4668, 2020. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2404952","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":162616057,"identity":"8fdeca7d-dbc5-4c06-84e3-27023e407524","order_by":0,"name":"Xiaoqiang Zhao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAvklEQVRIiWNgGAWjYHACNiC2kQEzeUjQksYDpkjRcpgELQbH0589+FFxnsfgfgPjg7dtDPLmhLRI9jxIN+w5c5vH4BgDs+HcNgbDnQ0EtPBLJByT4G0Da2GT5m1jSDA4QMgjEoltkn/bzoG0sP8mSgu/RDLI8ANgW5iJ0iLZ84xNWuZMMo/kscRmyTnnJAw3ENICCjHJNxV2cnyHDx/88KbMRp6gLQwMCTAGYwOQkCCoHlnLKBgFo2AUjAIcAADW0TkcybWQygAAAABJRU5ErkJggg==","orcid":"","institution":"Lanzhou University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Xiaoqiang","middleName":"","lastName":"Zhao","suffix":""},{"id":162616058,"identity":"e66aa3ce-9412-4dad-b7d9-9b00f7ca980f","order_by":1,"name":"Xiyao Li","email":"","orcid":"","institution":"Lanzhou University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Xiyao","middleName":"","lastName":"Li","suffix":""}],"badges":[],"createdAt":"2022-12-22 11:44:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2404952/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2404952/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":31035400,"identity":"f39bdb29-19b0-435a-9d75-2a61543f7ce4","added_by":"auto","created_at":"2023-01-03 15:37:02","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":39168,"visible":true,"origin":"","legend":"\u003cp\u003etwo-layer group convolution channel mash. GConv indicates group convolution. a) two group convolution layers of the same number of groups, with the output related to the group input only; b) two-layer group convolution taking data from different groups and with the number of input channels equal to the number of output channels; c) completed channel mash.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/e26ec8fcdd178a6e12483570.png"},{"id":31034121,"identity":"950c7a76-1716-4d66-bafa-166044794580","added_by":"auto","created_at":"2023-01-03 15:29:03","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":122074,"visible":true,"origin":"","legend":"\u003cp\u003eStructure of the feature-compensated shuffle network\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/96e8b04bcb74cd25b318465d.png"},{"id":31032869,"identity":"78a96c4a-e2c9-494a-a08b-d88ef6b5b990","added_by":"auto","created_at":"2023-01-03 15:21:03","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":149257,"visible":true,"origin":"","legend":"\u003cp\u003eAdaptive compensated fusion of Channel Structured Attention Model Structural diagram\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/d8b6b9f792876d76677f67b3.png"},{"id":31036871,"identity":"f5d00e39-7d5f-471d-abd8-9511fe4494ac","added_by":"auto","created_at":"2023-01-03 15:45:03","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":130001,"visible":true,"origin":"","legend":"\u003cp\u003eImage super-resolution framework of feature Compensated Adaptive Fusion Shuffle Network\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/f6e4a70c5b11f4b5e062211c.png"},{"id":31032870,"identity":"69f9c007-4e82-4e4e-906b-66111d0f8b8d","added_by":"auto","created_at":"2023-01-03 15:21:03","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":333146,"visible":true,"origin":"","legend":"\u003cp\u003eVisual quality of the Urban100 test set 4x magnification of CAFN compared to advanced methods\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/27e5f44521031e16e1d55570.png"},{"id":31032863,"identity":"99ebad15-9460-43d0-b999-d58b35a8dbe8","added_by":"auto","created_at":"2023-01-03 15:21:02","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":54854,"visible":true,"origin":"","legend":"\u003cp\u003eEvaluation results of the average PSNR value and parameter amount compared to other advanced methods on the Urban100 test set at 4x magnification\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/d9541ba930b25e6227294a58.png"},{"id":31032867,"identity":"d920307f-8def-4b90-aa59-69068971953c","added_by":"auto","created_at":"2023-01-03 15:21:03","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":46787,"visible":true,"origin":"","legend":"\u003cp\u003eL1 loss performance analysis of different networks under the Set5 test set\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/d3697c60a686fe71c351c33a.png"},{"id":31032868,"identity":"b6f0e227-6afe-4176-8c71-66776b22fcd1","added_by":"auto","created_at":"2023-01-03 15:21:03","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":14947,"visible":true,"origin":"","legend":"\u003cp\u003ePSNR performance analysis of different networks under the Set5 test set\u003c/p\u003e","description":"","filename":"8.png","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/a25aaf2e7261c80cd5303a45.png"},{"id":37003083,"identity":"191b30bd-c6c2-4846-bab5-44c887dac21c","added_by":"auto","created_at":"2023-05-14 12:59:45","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1716866,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2404952/v1/bad11160-7afc-4f96-96a2-7032088ef895.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Feature Compensated Adaptive Fusion Shuffle Network for Image Super-Resolution Reconstruction","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eIn the process of image acquisition, affected by the image processing technology and transmission environment, the resolution of the obtained original image is usually low. However, the original low-resolution images lack key information, and it is usually difficult to meet the technical requirements of application scenarios. To solve this problem, super-resolution (SR) reconstruction, as a low-level computer vision task, has become a research hotspot.\u003c/p\u003e \u003cp\u003eSingle image super-resolution reconstruction (SISR) aims at the technique of recovering a high-resolution image (HR) from a low-resolution image (LR). The super-resolution reconstruction process is a typical ill-conditioned inverse problem [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e], which means that an input LR image map has multiple output HR images. Although indicating that SISR is a difficult research area, the wide range of applications of SR in computer vision[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] makes it very valuable for research, such as urban surveillance[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e], medical imaging[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e], remote sensing images[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e], etc.\u003c/p\u003e \u003cp\u003eIn recent years, the rapid development of deep learning techniques has shown the advantages for SISR. Dong et al [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] first applied convolutional neural network (CNN) to super-resolution reconstruction and proposed SRCNN, which constructed a 3-layer neural network for image reconstruction. Then, Dong et al [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] further proposed an accelerating super-resolution convolutional neural network (FSRCNN), which used deconvolution instead of bicubic for image feature up-sampling, and constructed a neural network with 8 convolutional layers. Shi et al [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] proposed an efficient sub-pixel convolutional neural network (ESPCN), it relied on sub-pixel convolution operation instead of deconvolution operation, which greatly improved the computational speed. Recently, many CNN-based SR methods have been proposed, mainly by constructing new connection methods and increasing the network depth of CNN. Although these methods can provide a larger receptive field and richer contextual information, they come with a large number of parameters that are difficult to train, as well as the problems such as gradient explosion and gradient disappearance. He et al [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] proposed residual network to make deep neural network training easy. Their use of skip connections in adjacent feature layers ensures there is no less feature information in the forward propagation process. Residual network solves the problem of gradient explosion in the training process and makes the network much deeper. Kim et al [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] obtained the residual information between LR and HR in the form of residual learning and proposed a very deep convolutional network super-resolution (VDSR), which constructed a 20-layer network and enableed feature information to be passed between shallow features and deep features. The emergence of residual network has greatly improved the depth of the network. Recently, many proposed deep convolutional neural networks use residuals, such as DRCN [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], DRRN [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], EDSR[\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e], RCAN [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e], etc. SR networks vary in depth from 3 layers of SRCNN to over 400 layers of RCAN. Although deep networks exhibit state-of-the-art performance, these methods suffer from some limitations, for example, (1) Most methods tend to gain performance through deeper networks. However, with the increase of network depth, the number of parameters, training time, and required hardware memory would also increase, which would limit the application in real-world scenarios. (2) Most CNN-based SR models rarely optimize for convolutional filtering and pay little attention to the loss of information in a single stream during feature delivery.\u003c/p\u003e \u003cp\u003eVarious previous works have pointed out that network branching methods can increase the richness of feature extraction. They use different filtering methods for different branches to obtain information of different contexts, which improve the representativeness of the networks by fusing the different contextual information captured. Ren et al [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] proposed an image super-resolution based on fusing multiple convolution neural networks (CNF), which employed SRCNN in each branch. Li et al [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] proposed a multi-scale residual network (MSRN) for super-resolution, which used different convolution filters to extract features, combined local multi-scale features with global features, and fused residual features in layers to maximize the use of LR image features. Although, these methods extend the size of the receptive field by using different scale filtering methods for multiple branches. However, different branches ignore the loss of information during feature image filtering, which would lead to insufficient high-frequency reconstructed texture details.\u003c/p\u003e \u003cp\u003eTo address above problems, we propose a feature compensated adaptive fusion shuffle network (CAFN) for image super resolution reconstruction. CAFN adaptively compensates for the different background information captured by LR images passing through different filters. This compensation method can obtain richer high-frequency features, resulting in better reconstruction performance. On the one hand, in contrast to a purely branch network, the feature-compensated adaptive fusion shuffle network extracts richer contextual information by constructing two compensated branches to capture the missing feature information of a single path. On the other hand, compared to the traditional branching equivalent processing with branching feature extraction of different filters, CAFN uses attention adaptive fusion branching information to give different attention to the compensation branches of different filters, and saves fusion computation while obtaining better feature extraction results.\u003c/p\u003e \u003cp\u003eThe main contributions of this paper are as follows:\u003c/p\u003e \u003cp\u003e(1) We propose a feature-compensated shuffle model (FCSM), which compensates for the part of the high-frequency information lost in the single-branch feature transfer process through the branch network. FCSM captures different compensation information from different dimensions of the input image and fuses the information of each branch to improve the ability of the model to learn complex mappings.\u003c/p\u003e \u003cp\u003e(2) We propose an adaptive compensated fusion of channel structured attention model (ACFM), which adaptively learns the weight information of different channel structures. The compensation branch adaptively fuses the captured features and further focuses on the feature context dimension information.\u003c/p\u003e \u003cp\u003e(3) We propose a Feature Compensated Adaptive Fusion Shuffle Network (CAFN), which adaptively adjusts the different contextual information captured by different two-way compensation branches to obtain better high frequency reconstruction performance.\u003c/p\u003e \u003cp\u003eThe overall structure of this paper is as follows. In Section \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, we briefly review the works related to the proposed methods. In Section \u003cspan refid=\"Sec6\" class=\"InternalRef\"\u003e3\u003c/span\u003e, we describe the architecture of our proposed model in detail. In Section \u003cspan refid=\"Sec10\" class=\"InternalRef\"\u003e4\u003c/span\u003e, we present experimental details and datasets, and discuss the comparison of state-of-the-art models with our proposed model, as well as ablation studies and experimental results. Finally, in Section \u003cspan refid=\"Sec15\" class=\"InternalRef\"\u003e5\u003c/span\u003e we conclude the work of this paper.\u003c/p\u003e"},{"header":"2 Related Works","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Depthwise separable convolution\u003c/h2\u003e \u003cp\u003eDepthwise separable convolution was proposed in Xception [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] and MobileNets [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. Standard convolution can map all input feature images, and the number of feature channels is transformed with the number of convolution kernels during the convolution operation. The depthwise separable convolution divides standard convolution into two steps, namely deep convolution and point convolution. Compared with the standard convolution, for depthwise convolution, one input channel is convolved with one convolution kernel, and the number of convolution kernels remains the same as the number of channels in the previous layer. After one convolution operation, the size of the feature map and the number of channels do not change. Since the convolution operation is kept independent for each channel of the input layer, the feature information from the same space of different channels is not used, the whole convolution operation is done by post-convolution with Pointwise Convolution. Pointwise Convolution is similar to standard convolution in that the convolution size is 1x1 convolution, the convolution channel size is the output channel size of the previous layer, the convolution operations is weighted and expanded in depth, and a new feature map is output, and the output channel size remains the same as the convolution kernel channel size. Deeply separable convolution performs spatial convolution on each channel of the input independently of standard convolution, before performing point-by-point convolution, namely 1x1 convolution. By using depthwise separable convolution, the number of parameters and the amount of computation can be significantly reduced while achieving the same results as standard convolution.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 ShuffleNet network\u003c/h2\u003e \u003cp\u003eShuffleNet network is a highly flexible and high-performance deep network model, which reduces the high computational effort and computational complexity of convolutional operations by introducing pixel-level group convolution [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. It uses the channel shuffle operation to complete the exchange of information between different channels, which helps to enhance the generalization ability and recognition accuracy of the model, and to solve the problem of missing feature representations. Compared to other networks, ShuffleNet network not only reduces the computational cost, but also maintains the accuracy.\u003c/p\u003e \u003cp\u003eShuffleNet consists of group convolution and channel-mixing operations. Group convolution was first used in AlexNet [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. And Xception [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] proposed depthwise separable convolution based on group convolution. Although the computational effort is moderated compared to normal convolution, there is still the problem that the point-by-point convolution process consumes the computational effort of the model. Therefore, group convolution at the pixel level is proposed in the ShuffleNet network instead of 1x1 point-by-point convolution to reduce the computational complexity of the model. However, while reducing the complexity of the model by limiting the convolution operation to within-group convolution, multiple group convolutions only perform convolution stacking within this group, which can cause the output feature information to come from only a small part of the input features. The output features within a group are only related to the input features within that group, and there is no flow of feature information between different groups, limiting the exchange of features between group channels. For this purpose, a channel shuffle operation is proposed to allow group convolution to obtain input data from different groups, increasing the flow of feature information between groups. In particularly, channel blending takes each group of the previous feature map, divides it into several subgroups, and then reorganizes the subgroups into new groups for fusion, so that the output feature information includes the features from different groups. The process is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e:\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIf an input convolution has\u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003egroups, the output features have \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(g \\times n\\)\u003c/span\u003e\u003c/span\u003e channels. The channel mixing operation converts the output channel number to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((g,n)\\)\u003c/span\u003e\u003c/span\u003e, which is converted to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((n,g)\\)\u003c/span\u003e\u003c/span\u003e by transposition. The channels are shuffled to flatten the transposed features as the input to the next layer. The channel shuffling operation allows for more flexible variations in the construction of multiple group convolutions, making it possible to build more complex networks.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Dynamic network\u003c/h2\u003e \u003cp\u003eMost deep learning networks learn in a static manner, in which network parameters remain consistent before and after training, which can limit their representational power and efficiency. In contrast to static network, dynamic network can be selectively activated for model components by adaptively adjusting its structure or parameters according to the input during the inference process. Therefore, it has more efficient reasoning and learning ability. Dynamic network can improve model capacity by applying feature conditional attention weights on the aggregation of the convolution kernels [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] to achieve an optimal balance between accuracy and efficiency.\u003c/p\u003e \u003cp\u003eThe core idea of dynamic network is to adaptively adjust the convolution parameters based on the input image. It dynamically learns the attention vector based on the input and obtains a new convolution kernel by weighted aggregation of multiple parallel convolution kernels from the attention learning weights. This non-linear convolutional aggregation can significantly improve the expressive power of the network with the introduction of a very small amount of calculation. This is expressed as follows:\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$y=h(\\mathop {{W^T}}\\limits^{\\sim } (x)x+\\mathop b\\limits^{\\sim } (x))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\mathop W\\limits^{\\sim } (x)=\\sum\\limits_{{k=1}}^{N} {{\\pi _k}} (x){\\mathop W\\limits^{\\sim } _k},\\mathop b\\limits^{\\sim } (x)=\\sum\\limits_{{k=1}}^{N} {{\\pi _k}} (x){\\mathop b\\limits^{\\sim } _k}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$s.t.{\\text{ 0}} \\leqslant {\\pi _k}(x) \\leqslant 1,\\sum\\limits_{{k=1}}^{N} {{\\pi _k}} (x)=1$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\pi _k}\\)\u003c/span\u003e\u003c/span\u003e denotes the attention weight of the k-th filter, which changes by depending on each input \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathop W\\limits^{\\sim } _k}\\)\u003c/span\u003e\u003c/span\u003e is the convolutional aggregation weight, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathop b\\limits^{\\sim } _k}\\)\u003c/span\u003e\u003c/span\u003e represents the bias, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\mathop W\\limits^{\\sim } (x)\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\mathop b\\limits^{\\sim } (x)\\)\u003c/span\u003e\u003c/span\u003e indicate the kernel weights and bias containing the attention weights.\u003c/p\u003e \u003cp\u003eIn recent years, dynamic networks have more and more attracted the attention of researchers. Jia et al [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] first introduced dynamic learning to deep learning by proposing a network that dynamically generates convolutions based on inputs. Yang et al [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e] proposed a conditional parameterized convolutions for efficient inference (Condconv), which learned a specific convolution kernel parameter for each feature, generated kernel weights based on the input samples, and dynamically selected convolution kernels to improve model capacity. Chen et al [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] proposed a dynamic convolution, which enhanced model representation by fusing multiple parallel convolution kernels. Zhang et al [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e] proposed an accelerated convolutional neural network with dynamic convolution (DyNet), which adaptively generated convolutional kernels based on the image content and reduced the amount of redundant information that existed between convolutional kernels.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. The Proposed Method","content":"\u003cp\u003eAlthough the image super-resolution reconstruction based on deep learning has been well developed, there are still problems, such as increasing hardware requirements due to larger network volumes and difficulties in training due to the deepening complexity of the network structure. Deep learning training would increase the number of parameters and computational complexity, they not only affect the memory consumption of hardware facilities, but also reduce the training effect. In order to maintain a dynamic balance between the number of parameter and the training effect of the network, we propose a feature compensated adaptive fusion shuffle network for image super-resolution reconstruction. First, we propose a feature compensated shuffle model (FCSM), which uses depthwise separable convolution to reduce the number of parameters for the main convolution. FCSM has improved the problem of single-stream networks that could lead to partial feature loss. Secondly, we propose an adaptive compensation fusion of channel structured attention model (ACFM), which consists of an adaptive weight learning block focusing on the channel structure, and an improved feature compensated shuffle model. The weight learning block adaptively learns the image weight information, and the improved feature compensation adjusts FCSM to reduce the redundant information generated during extraction. Finally, the feature information from the two modules is fused and stacked with the above operations. We reconstruct the captured deep feature information to generate super-resolution images.\u003c/p\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Feature compensated shuffle network\u003c/h2\u003e \u003cp\u003eIn recent years, many scholars have proposed many neural networks for image super-resolution reconstruction, such as convolutional neural networks [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e], residual networks[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] and dense networks [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. The image super-resolution reconstruction based on these networks can achieve better reconstruction results, but there are still the problems of large network models and more parameters. Moreover, the network performance is positively correlated with the model size, so it is difficult to obtain better performance by shrinking the model. During the feature extraction process of the model, continuous filtering would gradually lose some high-frequency information. If this loss can be compensated for by branching compensation features, more image information can be captured under the condition of limited expansion of the model. Also, depthwise separable convolution can be used instead of standard convolution to minimize the number of model parameters.\u003c/p\u003e \u003cp\u003eBased on the above purpose, we propose feature-compensated shuffle network. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, the network mainly consists of a split-channel compensation branch and a multi-channel compensation branch. Both the split-channel compensation and multi-channel compensation are composed of a channel group extraction module and a channel shuffling module. The channel grouping extraction module consists of multiple grouping convolutions and depth-separable convolutions. The split-channel compensation branch compresses part of the feature information, reducing the number of parameters in the feature capture process, and the multi-channel compensation branch extends the channels to increase their feature complexity. For the doubled parameter amount of the multi-channel compensation branch, the split-channel compensation branch compression extraction can effectively reduce the total parameters and computational effort of compensation fusion. The detailed operation of this network is shown as follows:\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e(1) Split-channel compensation\u003c/p\u003e \u003cp\u003eThe network feeds the input features into a split-channel compensation branch for split-channel feature extraction. In the split-channel compensation branch, we perform a channel split on the input features, the\u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003edimensional input features are divided into two \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(m/2\\)\u003c/span\u003e\u003c/span\u003e dimensional ones for two-way feature extraction. The left branch maintains \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(m/2\\)\u003c/span\u003e\u003c/span\u003e dimensions to achieve the retention of the initial features. The right branch is used as a compensating branch to fully extract the feature information. First, the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(m/2\\)\u003c/span\u003e\u003c/span\u003edimensional features are extracted by group convolution, and then the depthwise separable convolution is used to extract features, and finally the group convolution is used to extract features to obtain richer feature information. In order to reduce the computational effort, the size of the group convolution is all 1x1. Afterwards, the features of the two branches are fused to compensate for the feature information and return the number of channels to the dimension before splitting, and a channel shuffle is performed. The specific operation is described below:\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e\n$${F_s}=H_{{m/2}}^{1}(X)=H_{{m/2}}^{2}(X)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(H_{{m/2}}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(H_{{m/2}}^{2}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e indicate channel splitting operations, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_s}\\)\u003c/span\u003e\u003c/span\u003e indicates the feature map for channel splitting,\u003cdiv id=\"Equ5\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ5\" name=\"EquationSource\"\u003e\n$${E_k}=(\\sigma (C_{N}^{1}({F_s})))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e5\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{N}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(1 \\times 1\\)\u003c/span\u003e\u003c/span\u003e standard convolution and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\sigma ( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represents the Relu activation function, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({E_k}\\)\u003c/span\u003e\u003c/span\u003e is for retained branch feature map,\u003cdiv id=\"Equ6\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ6\" name=\"EquationSource\"\u003e\n$${E_t}=\\sigma (C_{{G2}}^{1}(\\sigma (D_{{SC}}^{1}(\\sigma (C_{{G1}}^{1}({F_s}))))))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e6\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{G1}}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{G2}}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e are the two previous group convolution operations, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(D_{{SC}}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represents the depth-separable convolution. Afterwards, the obtained feature channel correlation information \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({E_t}\\)\u003c/span\u003e\u003c/span\u003e is used to compensate for the reserved branch features and the channel dimension, which would be recovered.\u003cdiv id=\"Equ7\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ7\" name=\"EquationSource\"\u003e\n$${F_{Of}}={H_{m - cat}}({E_k},{E_t})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e7\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H_{m - cat}}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e feature dimension is recovered as a \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003edimensional merge operation, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_{Of}}\\)\u003c/span\u003e\u003c/span\u003e represents the feature map for channel dimension recovery,\u003cdiv id=\"Equ8\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ8\" name=\"EquationSource\"\u003e\n$${F_{OH}}={}_{h}H_{{CS}}^{n}({F_{Of}})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e8\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({}_{h}H_{{CS}}^{n}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is the channel wash, \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e is the wash group factor, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_{OH}}\\)\u003c/span\u003e\u003c/span\u003e is the split channel compensation output feature.\u003c/p\u003e \u003cp\u003e(2) Multi-channel compensation\u003c/p\u003e \u003cp\u003eSince the split-channel compensation branch obtains the features through branch processing and fusion extraction in the case of splitting part of the channels, the features have insufficient information. If these features were used directly to reconstruct SR images, the image reconstruction would not be sufficiently detailed, so we construct multi-channel compensation branch to extend and complement these features, and different depths of extraction are used for the features in different branches. Similar to the split-channel compensation branch, the multi-channel compensation branch also uses two channels for feature extraction, which keeps the number of channels at \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e for both the left and right branches for a \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003edimensional feature input. In order to keep the image size constant, a convolution with a stride size of 1 is always used to extract the features. In contrast to the left branch of the split-channel compensation branch, which maintains the feature information, the left branch of our multi-channel branch uses two layers of feature extraction with constant dimensionality of the feature information. The non-linearity of the features is performed by end-to-end group convolution with depthwise separable convolution extraction by using a depthwise separable convolution of size \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(3 \\times 3\\)\u003c/span\u003e\u003c/span\u003e. We use a stride size of 1 to ensure that the size of its feature map does not change. The right branch is used as a compensation branch to ensure that the number of input channels is the same as the number of branch channels. The right branch is used as a compensation branch, identical to the compensation branch for sub-channel compensation, to enrich the features. Similarly, the left and right branches are fused to obtain a feature map of channel dimension \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(2m\\)\u003c/span\u003e\u003c/span\u003e. In order to recover the number of channels, we use a channel compression convolution operation to recover the channels to \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e dimensions. This is followed by a channel shuffle operation, as follows:\u003cdiv id=\"Equ9\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ9\" name=\"EquationSource\"\u003e\n$${F_d}=N_{m}^{1}(X)=N_{m}^{2}(X)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e9\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N_{m}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N_{m}^{2}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e denote that the channel is held and features are extracted once. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_d}\\)\u003c/span\u003e\u003c/span\u003e means the extracted initial feature map,\u003cdiv id=\"Equ10\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ10\" name=\"EquationSource\"\u003e\n$${E_l}=\\sigma (C_{{G1}}^{2}(\\sigma (D_{{SC}}^{2}({F_d}))))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e10\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(D_{{SC}}^{2}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e denotes the depthwise separable convolution, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\sigma ( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e denotes the Relu activation, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{G1}}^{2}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e denotes the left-branch group convolution, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({E_l}\\)\u003c/span\u003e\u003c/span\u003e means the feature map extracted from the left branch,\u003cdiv id=\"Equ11\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ11\" name=\"EquationSource\"\u003e\n$${E_r}=\\sigma (C_{{G2}}^{3}(\\sigma (D_{{SC}}^{3}(\\sigma (C_{{G1}}^{3}({F_d}))))))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e11\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{G1}}^{3}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{G2}}^{3}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e are the two group convolution operations on the right branch, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(D_{{SC}}^{3}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represents the depth-separable convolution. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({E_r}\\)\u003c/span\u003e\u003c/span\u003e means the feature map extracted from the right branch. We propose multi-channel compensation that has a left-right branch structure. This structure has more non-linear mappings, which can better fit the complex relationship between channels. Subsequently, we fuse the feature information from both the left and right branches to complement the information. We use the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(concat\\)\u003c/span\u003e\u003c/span\u003e merge operation to retain all extracted features and double the channel dimension. Afterwards, we access the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(3 \\times 3\\)\u003c/span\u003e\u003c/span\u003ecompression convolution to place the channel dimension as the input dimension. We use channel shuffle to increase the interaction and fusion between different groups of channels, to correct for high frequency information:\u003cdiv id=\"Equ12\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ12\" name=\"EquationSource\"\u003e\n$${F_{Ob}}={H_{2m - cat}}({E_l},{E_r})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e12\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H_{2m - cat}}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e feature dimension is multiplied to the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(2m\\)\u003c/span\u003e\u003c/span\u003e dimension \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(concat\\)\u003c/span\u003e\u003c/span\u003e merge operation. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_{Ob}}\\)\u003c/span\u003e\u003c/span\u003e means dimension doubled feature map,\u003cdiv id=\"Equ13\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ13\" name=\"EquationSource\"\u003e\n$${F_{OD}}={}_{d}H_{{CS}}^{n}(C_{{G3}}^{3}({F_{Ob}}))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e13\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{G3}}^{3}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is the channel compression convolution, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({}_{d}H_{{CS}}^{n}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is the channel blending, \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e is the blending group factor, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_{OD}}\\)\u003c/span\u003e\u003c/span\u003e is the split-channel compensated output feature. Finally, a superposition operation \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H_{Add}}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is used to fuse the split-channel and multi-channel compensation features to obtain an output \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_{OUT}}\\)\u003c/span\u003e\u003c/span\u003e, which completes a feature-compensated shuffle operation.\u003cdiv id=\"Equ14\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ14\" name=\"EquationSource\"\u003e\n$${F_{OUT}}={H_{Add}}({F_{OD}},{F_{OH}})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e14\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Feature Compensated Adaptive Fusion Shuffle Network\u003c/h2\u003e \u003cp\u003eIn the feature compensated shuffle network, we compress and multiply the extracted model by split-channel compensation and multi-channel compensation to supplement the missing feature information and improve the accuracy. However, due to the branching path processing feature, it increases the fragmentation of the network, introduces extra overhead during kernel startup and synchronicity, and results in excessive pressure on parallel computation. It is difficult for us to maintain computational speed at deeper network layers, and even over-fitting can occur, resulting in inability to compute. At the same time, FCSM completely overlays and fuses the feature information extracted by split-channel compensation and multi-channel compensation, without selective extraction according to its own structural characteristics, which would cause redundant feature information to occupy excessive computing space and make it difficult accurately recover useful feature information. The influence of this situation would increase with the deepening of the number of training layers. In order to solve these problems, we propose a feature-compensated adaptive fusion shuffle network model, whose structure is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe network is structured to control that both compensation branches maintain the same number of channels, and the learning of the target features is reinforced by the neural network with a soft attention module. Adaptive attention learning is used to calculate the weights and dynamically adjust the weight ratio of the two compensated branches to improve the network representation. The attention module is a weighted attention module that extracts useful information and suppresses useless information by focusing on the regions of interest features. SEnet [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] focuses on the different weights of the channels and SAnet [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] focuses on the target spatial location to obtain the\u003c/p\u003e \u003cp\u003eweights. However, these attention modules only focus on the information changes before and after the modules in the network, and are unable to focus on the structural information differences between modules. In a branching network structure, the different extraction methods of the two branches would produce different representation information. The depth of the network, the filtering method and the number of branching channels would result in different perceptual fields of information being extracted from the branches. Aggregation of these different information results in more complex feature information. If the extracted image features were aggregated at the same level of attention without regard to structural differences, extensive computational resources would be spent on the redundant feature extraction.\u003c/p\u003e \u003cp\u003eIn this paper, in order to optimize the accuracy of the fused features of the branched information, we hope that different branch structures have different information attention, so a novel adaptive compensated fusion of attention of channel structured Attention Model (ACFM) is proposed, which consists of an adaptive weight learning module and an attention network module with feature-compensated shuffle. The adaptive weight learning module performs the initial processing of the learned features through the residual blocks, which are then connected end-to-end through a pooling layer, a fully connected layer and a softmax function. The connections are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e, a feature-compensated shuffle module with attention, where image features are extracted from the residual blocks, after which the features are passed into two separate compensation branches. In order to adapt to the fusion of learning weights, the two compensation branches are further improved to increase their representation capabilities. The adaptive weight learning module assigns adaptive learning weights to the two compensated branch extracted feature images.\u003c/p\u003e \u003cp\u003eFinally, the two features are fused to obtain mixed feature information focusing on the differences in the structure of the different modules. The ACFM process can be divided into three steps:\u003c/p\u003e \u003cp\u003eStep1: The input features \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e are extracted by the residual block to obtain the image feature information \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_u}\\)\u003c/span\u003e\u003c/span\u003e, the residual block primary processes the input features and enriches the extraction of details, and its mathematical process can be described as follows:\u003cdiv id=\"Equ15\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ15\" name=\"EquationSource\"\u003e\n$${F_u}={R_o}(X)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e15\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({R_o}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e stands for the residual block processing.\u003c/p\u003e \u003cp\u003eStep2: The adaptive weight learning module compresses the dimension of the image features \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_u}\\)\u003c/span\u003e\u003c/span\u003e by global averaging pooling, which compresses the spatial dimensional information into the channel and aggregates the input feature map into a vector of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(1 \\times 1 \\times {C_i}\\)\u003c/span\u003e\u003c/span\u003e, this vector is then fed into the end-connected activation layer of the fully connected layer for operations in the end-connected fully connected layer, and finally the features obtained from these operations are fed into the Softmax activation function. The Softmax activation function is to map the input between [0, 1] and ensure that the normalized sum is 1, that is, the adaptive weight information \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\lambda _n}(x)\\)\u003c/span\u003e\u003c/span\u003e of the attention of the structure is obtained. In the adaptive weight learning module, adaptive learning is performed according to the input image features, and the relevant weight coefficients are calculated. The weights would be adaptively adjusted according to the network structure. In order to prevent the model from being too computationally intensive, the kernel space is compressed through aggregation constraints, as shown in formula (16):\u003cdiv id=\"Equ16\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ16\" name=\"EquationSource\"\u003e\n$${\\lambda _n}(x)=\\delta (\\mu (\\sigma (\\mu (\\varphi ({F_u}))))),{\\text{ }}0 \\leqslant {\\lambda _n}(x) \\leqslant 1,{\\text{ }}1=\\sum\\limits_{{n=1}}^{z} {{\\lambda _n}} (x)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e16\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varphi ( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represents the global average pooling, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\mu\\)\u003c/span\u003e\u003c/span\u003e represents the fully connected operation, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\sigma\\)\u003c/span\u003e\u003c/span\u003e represents the Relu activation function, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\delta\\)\u003c/span\u003e\u003c/span\u003e represents the Softmax activation function.\u003c/p\u003e \u003cp\u003eThe attention network module with feature-compensated shuffle performs a convolution of the initial feature information \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_u}\\)\u003c/span\u003e\u003c/span\u003e, which is then fed into two compensation branches to obtain the channel compensation information. This is followed by a convolution activation mapping, where the activation function uses LeakyReLu to activate more neurons to extend the subsequent module to capture more feature rich information. The channel weights are then selected by Senet attention to obtain the split-channel compensation feature \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({V_1}\\)\u003c/span\u003e\u003c/span\u003e and the multi-channel compensation feature \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({V_2}\\)\u003c/span\u003e\u003c/span\u003e, which contain the channel weight shares. The mathematical model can be expressed as:\u003cdiv id=\"Equ17\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ17\" name=\"EquationSource\"\u003e\n$${V_1}=S_{E}^{1}(\\nu (C_{{co}}^{2}({S_{CC}}(C_{{co}}^{1}({F_u})))))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e17\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ18\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ18\" name=\"EquationSource\"\u003e\n$${V_2}=S_{E}^{2}(\\nu (C_{{co}}^{2}({M_{CC}}(C_{{co}}^{1}({F_u})))))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e18\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{co}}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is a feature mapping operation, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S_{CC}}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({M_{CC}}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represent the split-channel compensation and multi-channel compensation respectively, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(C_{{co}}^{2}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e is the convolution used to extract global features, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\nu\\)\u003c/span\u003e\u003c/span\u003e denotes the LeakyReLu activation function, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(S_{E}^{1}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(S_{E}^{2}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e denotes the SEnet attention modules.\u003c/p\u003e \u003cp\u003eStep3: The information of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({V_1}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({V_2}\\)\u003c/span\u003e\u003c/span\u003e is aggregated, and fusion uses the adaptive weight learning model learning weights and the attention network model with feature compensation shuffle function of compensation branch by branch multiplication, in the multiplication of the resulting features containing weight accounted for aggregation. The features containing concerns about the adaptive structure of the network are obtained.\u003cdiv id=\"Equ19\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ19\" name=\"EquationSource\"\u003e\n$$U={V_1}\\cdot {\\lambda _1}(x)+{V_2}\\cdot {\\lambda _2}(x)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e19\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ20\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ20\" name=\"EquationSource\"\u003e\n$$1={\\lambda _1}(x)+{\\lambda _2}(x)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e20\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\lambda _1}(x)\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\lambda _2}(x)\\)\u003c/span\u003e\u003c/span\u003e represent the weight components learned by the adaptive weight learning model, and\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\cdot\\)\u003c/span\u003e\u003c/span\u003erepresents the corresponding multiplication between the compensated branch structure and the adaptive weights.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Network Framework\u003c/h2\u003e \u003cp\u003eWe propose a feature compensated adaptive fusion shuffle network for image super resolution reconstruction (CAFN), its model framework is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e. The network framework consists of four parts, namely, shallow feature extraction, deep feature extraction, upsampling and image reconstruction. If \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({I_{LR}}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({I_{HR}}\\)\u003c/span\u003e\u003c/span\u003e are the input low-resolution image and the output reconstructed image, respectively, the network model first extracts the shallow features in the low-resolution image by using the primary convolution block.\u003cdiv id=\"Equ21\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ21\" name=\"EquationSource\"\u003e\n$${F_0}={H_I}\\left( {{I_{LR}}} \\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e21\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H_I}\\)\u003c/span\u003e\u003c/span\u003e represents the mapping function of the initial convolutional layer, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_0}\\)\u003c/span\u003e\u003c/span\u003e represents the extracted shallow feature. Shallow features capture the deep features of the image through a shuffling network of feature compensation adaptive fusion, namely:\u003cdiv id=\"Equ22\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ22\" name=\"EquationSource\"\u003e\n$${F_f}={F_0}+{f_f}({C_M}( \\cdot \\cdot \\cdot ({C_m}({C_{m - 1}}( \\cdot \\cdot \\cdot {C_1}({F_0}) \\cdot \\cdot \\cdot ))) \\cdot \\cdot \\cdot ))$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e22\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003ewhere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_0}\\)\u003c/span\u003e\u003c/span\u003e performs feature update through \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e ACFMs and a convolutional layer, and then combines the updated features and shallow features \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_0}\\)\u003c/span\u003e\u003c/span\u003e through global residual learning. Specifically, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({C_m}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represents the feature extraction operation of the m-th ACFM, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f_f}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e represents the convolution operation of end-to-end convolution, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_f}\\)\u003c/span\u003e\u003c/span\u003erepresents the extracted deep features. At this point, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_f}\\)\u003c/span\u003e\u003c/span\u003e is input to the up-sampling module, and the up-sampling module amplifies the combined feature output to the desired SR image size, namely:\u003cdiv id=\"Equ23\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ23\" name=\"EquationSource\"\u003e\n$${F_{UP}}={H_{UP}}({F_f})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e23\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAfter the reconstruction of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({F_{UP}}\\)\u003c/span\u003e\u003c/span\u003e by the reconstruction module, the corresponding SR image is obtained,\u003cdiv id=\"Equ24\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ24\" name=\"EquationSource\"\u003e\n$${I_{SR}}={H_{RE}}({F_{UP}})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e24\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H_{RE}}( \\cdot )\\)\u003c/span\u003e\u003c/span\u003e denotes the reconstruction mapping function.\u003c/p\u003e \u003c/div\u003e"},{"header":"4 Experimental Results And Analysis","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Experimental setup\u003c/h2\u003e \u003cp\u003eFor the training phase, this paper uses the DIV2K [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] dataset as our training dataset and augments the training dataset with 90\u0026deg;, 180\u0026deg;, 270\u0026deg; rotations and random horizontal flips. The LR image is cropped to a block of size \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(48 \\times 48\\)\u003c/span\u003e\u003c/span\u003e pixels and the HR image is cropped to a block of size \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(48s \\times 48s\\)\u003c/span\u003e\u003c/span\u003e pixels, where \u003cspan class=\"InlineEquation\"\u003e\u003c/span\u003e denotes the magnification factor. In the testing phase, this paper adopts four commonly used standard datasets: Set5[\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e], Set14[\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e], BSD100[\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e], Urban100[\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. During the test, the image color space is converted from RGB to YCbCr, and PSNR and SSIM are calculated in the Y channel. In this paper, the network is optimized by the Adam algorithm. When the network weight is updated, its exponential decay rate is set to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\beta _1}=0.9\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\beta _2}=0.999\\)\u003c/span\u003e\u003c/span\u003e, and the numerically stable small constant \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varepsilon ={10^{ - 8}}\\)\u003c/span\u003e\u003c/span\u003e. We use L\u003csub\u003e1\u003c/sub\u003e as the loss function, the training iterations are 1000 (epoch), the batch size is 16, and the learning rate is \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({10^{ - 4}}\\)\u003c/span\u003e\u003c/span\u003e. The network framework construction, training and testing of this experiment are carried out on Ubuntu18.04 system, the programming platform is Pytorch1.2, the experimental GPU is NVIDIA GeForce RTX2080Ti, the CPU is Inter(R) Core (TM) i9-9900K, and the memory size is 64G.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Experiment evaluation index\u003c/h2\u003e \u003cp\u003eIn this paper, Peak Signal to Noise Ratio (PSNR) and Structural Simi-Larity (SSIM) are used as evaluation indicators. The size of PSNR is related to the mean square error of the current image and the reference image and the maximum pixel value. The calculation form is simple and the physical meaning is clear; SSIM is a full-reference image quality indicator that evaluates image similarity from three aspects: brightness, contrast, and structure. It simulates the visual function of the abstract object structure of the human visual system as a whole, and conforms to visual perception.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Experimental results and analysis\u003c/h2\u003e \u003cp\u003eTo verify the effectiveness of our algorithm, we use the DIV2K dataset to train CAFN, and compare the proposed algorithm model with Bicubic, SRCNN[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e], FSRCNN[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], VDSR[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e], DRCN[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], LapSRN[\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e], DRRN[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], MSRN [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], CARN [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e], OISR [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e], SeaNet [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the evaluation index values of different scale factors of different algorithms on the four test data sets of Set5, Set14, BSD100 and Urban100, where the best and second best performing algorithms are bolded and underlined, respectively. As it can be seen from Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, our proposed CAFN achieves optimal and sub-optimal PSNR and SSIM values for quantitative evaluation at 2、3 and 4 times magnification compared to the other algorithms mentioned above. For example, on the Set5 test set at x3 magnification, PSNR and SSIM of CAFN achieve 0.07dB and 0.007 higher r than that of the suboptimal OISR; When the magnification is 4, PSNR of CAFN achieves the highest values on all four test datasets, and SSIM values are the highest on the Set5, B100 and Urban100 datasets. In particular, PSNR and SSIM of our proposed method are 0. 12dB and 0.0014 higher on Set5 than that of the suboptimal SeaNet, while on Urban100 our proposed CAFN has a PSNR and SSIM 0.13dB and 0.0017 higher than the suboptimal results obtained by OISR. The reason is that CAFN compensates for part of the contextual information lost in the single-flow network pass, especially the high-frequency components, and we can obtain better reconstruction results in image data that focus on capturing feature integrity.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eQuantitative evaluation of 12 SR methods tested on four benchmark sets at different magnifications\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eMethod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eScale\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eParameters\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eSet5\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSet14\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eBSD100\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eUrban100\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBicubic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\" morerows=\"11\" rowspan=\"12\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e33.66/0.9299\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e30.24/0.8688\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e29.56/0.8431\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.88/0.8403\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSRCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e57K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e36.66/0.9542\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e32.42/0.9063\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e31.36/0.8879\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e29.50/0.8946\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFSRCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.00/0.9559\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e32.75/0.9098\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e31.51/0.8939\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e29.87/0.9065\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVDSR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e665K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.53/0.9587\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.03/0.9124\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e31.90/0.8960\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e30.77/0.9140\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDRCN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1774K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.63/0.9588\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.08/0.9118\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e31.08/0.8942\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e30.41/0.9133\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLapSRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e812K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.52/0.9581\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.08/0.9109\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e31.80/0.8949\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e30.41/0.9112\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDRRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e297K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.74/0.9591\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.23/0.9136\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e32.05/0.8973\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e31.23/0.9188\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMSRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5890K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e38.08/0.9605\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.74/0.9170\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e32.23/0.9013\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e32.22/0.9326\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCARN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1592K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.76/0.9590\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.52/0.9166\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e32.09/0.8978\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e31.92/0.9256\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOISR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4970K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e38.12/0.9609\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e33.78/0.9196\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e32.26/0.9007\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e32.52/0.9320\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSeaNet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7102K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e38.08/0.9606\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e33.75/0.9190\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e32.27/0.9008\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e32.50/0.9318\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCAFN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4755K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e38.14/0.9610\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e33.76/0.9192\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e32.24/0.9003\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e32.47/0.9313\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBicubic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\" morerows=\"11\" rowspan=\"12\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e30.39/0.8682\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e27.55/0.7742\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.21/0.7385\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e24.46/0.7349\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSRCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e57K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.75/0.9090\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e29.28/0.8209\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e28.41/0.7863\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.24/0.7989\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFSRCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e33.02/0.9135\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e29.49/0.8271\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e28.50/0.7937\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.41/0.8161\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVDSR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e665K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e33.66/0.9213\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e29.78/0.8314\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e28.82/0.7976\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e27.14/0.8279\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDRCN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1774K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e33.82/0.9226\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e29.79/0.8311\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e28.82/0.7963\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e27.07/0.8276\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLapSRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e812K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e33.82/0.9207\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e29.89/0.8304\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e28.82/0.7950\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e27.07/0.8298\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDRRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e297K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e34.03/0.9244\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e29.96/0.8349\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e28.95/0.8004\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e27.53/0.8378\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMSRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6080K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e34.38/0.9262\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e30.34/0.8395\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e29.08/0.8041\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e28.08/0.8554\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCARN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1592K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e34.29/0.9255\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e30.29/0.8407\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e29.06/0.8034\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e28.06/0.8493\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOISR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5640K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e34.56/0.9284\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e30.46/0.8450\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e29.20/0.8077\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e28.56/0.8606\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSeaNet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7471K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e34.55/0.9282\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e30.42/0.8444\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e29.17/0.8071\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e28.50/0.8594\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCAFN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4939K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e34.63/0.9289\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e30.67/0.8469\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e29.19/0.8071\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e28.54/0.8596\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBicubic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\" morerows=\"10\" rowspan=\"11\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e28.42/0.8104\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e26.00/0.7027\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e25.96/0.6675\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e23.14/0.6577\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSRCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e57K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e30.48/0.8628\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e27.49/0.7503\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e26.90/0.7101\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e24.52/0.7221\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFSRCNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e30.66/0.8646\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e27.71/0.7562\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e26.98/0.7124\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e24.60/0.7221\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVDSR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e665K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e31.35/0.8838\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.02/0.7674\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.29/0.7251\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e25.18/0.7524\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDRCN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1774K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e31.35/0.8854\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.19/0.7670\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.23/0.7233\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e25.14/0.7510\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLapSRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e812K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e31.54/0.8811\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.19/0.7635\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.32/0.7162\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e25.21/0.7564\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDRRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e297K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e31.68/0.8888\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.21/0.7720\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.38/0.7284\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e25.44/0.7638\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMSRN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6330K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.07/0.8903\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.60/0.7751\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.52/0.7273\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.04/0.7896\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCARN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1592K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.13/0.8937\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.60/0.7806\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.58/0.7349\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.07/0.7837\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOISR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5500K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.33/0.8968\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e28.73\u003c/span\u003e\u003cb\u003e/\u003c/b\u003e0.7845\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e27.66/0.7389\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e26.37/0.7953\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSeaNet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7397K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e32.33/0.8970\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.72/\u003cb\u003e0.7855\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.65/0.7388\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.32/0.7942\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCAFN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4902K\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e32.45/0.8984\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e28.77\u003c/b\u003e/\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.7852\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e27.67\u003c/b\u003e/\u003cb\u003e0.7389\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e26.45/0.7959\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn order to further evaluate the visual quality of CAFN reconstruction, this paper selects Image011,Image024,and Image030 in the Urban100 dataset under magnification to compare the visual effects of other advanced SR methods, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e. From Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e, we can see that the SR images reconstructed by other advanced methods suffer from blur, artifacts and distortion. Especially for\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eImage024 images, other advanced methods appear distorted, and CAFN is closer to the HR images in the details of reconstructed SR images; in Image011 and Image030, the reconstructed details have serious line blur problems, and our proposed CAFN is clearer, the lines and details of the LR image are reconstructed without blurring and distortion problems. We also evaluate our proposed CAFN in comparison with other state-of-the-art SR methods in terms of parameter quantities. Figure\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e shows the comparative test analysis results between the parameter amount and PSNR value in the Urban100 test set under \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times 4\\)\u003c/span\u003e\u003c/span\u003e magnification. From Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e, we can analyze that our proposed CAFN has the highest PSNR value on Urban100, while we propose CAFN with a lower number of parameters compared to OISR, SeaNet and MSRN methods. Compared with SRCNN, VDSR, DRCN, DRRN and CARN, CAFN has more parameters, but it has achieved great improvement in visual quality and quantitative evaluation. Specifically, our proposed CAFN is better than SRCNN, VDSR, DRCN, DRRN and CARN are improved by 1.93dB, 1.27dB, 1.31dB, 1.01dB and 0.38dB, respectively.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Ablation Study\u003c/h2\u003e \u003cp\u003eTo verify the effect of ACFM, we conduct ablation experiments of ACFM to verify the effect of each block, we gradually modify our model, and compare their differences. As shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, in order to show the effect of our proposed feature compensation module and adaptive weight learning module, we denote the network with the adaptive weight learning module removed by ACFM-C, the network with the feature compensation attention module removed by ACFM-A, and the network with the adaptive weight learning module and feature compensation attention module removed by ACFM-N. ACFM is trained with the above networks under the same training details and convolutional activation layers. In particular, we set the same supervised training model at 595 epochs, and in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, we find that our model also performs better at low training counts.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAverage PSNR and SSIM of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times 4\\)\u003c/span\u003e\u003c/span\u003e magnification factor for Attention Adaptive Compensation Fusion (ACFM) of Channel Structure\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdaptive Weight Learning Module\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFeature Compensation Attention Module\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eSet5\u003c/p\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSet14\u003c/p\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eBSD100\u003c/p\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eUrban100\u003c/p\u003e \u003cp\u003ePSNR/SSIM\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eACFM-N\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e\u0026times;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e\u0026times;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.27/0.8954\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.64/0.7815\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.57/0.7357\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.13/0.7850\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eACFM-C\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e\u0026times;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e\u0026radic;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.43/0.8980\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.75/0.7843\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.62/0.7388\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.43/0.7955\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eACFM-A\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e\u0026radic;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e\u0026times;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.42/0.8976\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.73/0.7848\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.62/0.7383\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.39/0.7954\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eACFM\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e\u0026radic;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e\u0026radic;\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e32.45/0.8981\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.75/0.7847\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.62/0.7392\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e26.40/0.7959\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFrom the iterative results of the loss function during training in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e, it can be seen that our proposed ACFM has a faster convergence rate and a lower loss value after convergence than the comparative experiments such as ACFM-N, ACFM-C, and ACFM-A. According to the analysis of objective\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eevaluation indicators in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e, it can be concluded that on the Set5 data set, for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\times 4\\)\u003c/span\u003e\u003c/span\u003e magnification, ACFM obtains a better PSNR value than the comparison methods, which verifies that ACFM has better reconstruction and magnification performance.\u003c/p\u003e \u003c/div\u003e"},{"header":"5 Conclusion","content":"\u003cp\u003eIn this paper, we propose a feature compensated adaptive fusion shuffle network (CAFN) for image super-resolution reconstruction. Specifically, CAFN captures contextual feature information by using branch-way feature-compensated shuffled attention model. In addition, CAFN uses an adaptive attention model to attention to the channel structure. It is adaptive in learning the different weight components obtained from different channel structures and enables more efficient attention to feature information of interest. CAFN to compensate for the high frequency components of the information captured by the two branches of the shuffled attention model to obtain more accurate high frequency features. Compared to other state-of-the-art methods, our proposed CAFN achieves competitive performance in terms of image visual quality assessment and quantitative evaluation.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding and/or conflicts of interests/Competing interests\u003c/strong\u003e The authors declare that they have no conflicts of interest to this work. The people involved in the experiment have been informed and formally accepted.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e This research work has been awarded by the National Natural Science Foundation of China (62263021), The Science and Technology Project of Gansu Province (21JR7RA206, 21YF5GA072), the Open Fund project of Gansu Provincial Key Laboratory of Advanced Control for Industrial Process (2022KX07), the Industrial Support Project of Education Department of Gansu Province (2021CYZC-02).\u003c/p\u003e\n"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eJ. Yu, X. Gao, D. Tao, X. Li, and K. Zhang, \u0026quot;A unified learning framework for single image super-resolution,\u0026quot; \u003cem\u003eIEEE Transactions on Neural networks and Learning systems, \u003c/em\u003evol. 25, no. 4, pp. 780-792, 2013.\u003c/li\u003e\n\u003cli\u003eM. Zhang, J. Xin, J. Zhang, D. Tao, and X. Gao, \u0026quot;Curvature Consistent Network for Microscope Chip Image Super-Resolution,\u0026quot; \u003cem\u003eIEEE Transactions on Neural Networks and Learning Systems, \u003c/em\u003e2022.\u003c/li\u003e\n\u003cli\u003eK. Li, S. Yang, R. Dong, X. Wang, and J. Huang, \u0026quot;Survey of single image super‐resolution reconstruction,\u0026quot; \u003cem\u003eIET Image Processing, \u003c/em\u003evol. 14, no. 11, pp. 2273-2290, 2020.\u003c/li\u003e\n\u003cli\u003eS. Jung and J. Kim, \u0026quot;Adaptive and stabilized real-time super-resolution control for UAV-assisted smart harbor surveillance platforms,\u0026quot; \u003cem\u003eJournal of Real-Time Image Processing, \u003c/em\u003evol. 18, no. 5, pp. 1815-1825, 2021.\u003c/li\u003e\n\u003cli\u003eY. Lee, J. Yun, Y. Hong, J. Lee, and M. Jeon, \u0026quot;Accurate license plate recognition and super-resolution using a generative adversarial networks on traffic surveillance video,\u0026quot; in \u003cem\u003e2018 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia)\u003c/em\u003e, 2018, pp. 1-4: IEEE.\u003c/li\u003e\n\u003cli\u003eY. Li, B. Sixou, and F. Peyrin, \u0026quot;A review of the deep learning methods for medical images super resolution problems,\u0026quot; \u003cem\u003eIrbm, \u003c/em\u003evol. 42, no. 2, pp. 120-133, 2021.\u003c/li\u003e\n\u003cli\u003eS. Zhang, Q. Yuan, J. Li, J. Sun, and X. Zhang, \u0026quot;Scene-adaptive remote sensing image super-resolution using a multiscale attention network,\u0026quot; \u003cem\u003eIEEE Transactions on Geoscience and Remote Sensing, \u003c/em\u003evol. 58, no. 7, pp. 4764-4779, 2020.\u003c/li\u003e\n\u003cli\u003eD. Zhang, J. Shao, X. Li, and H. T. Shen, \u0026quot;Remote sensing image super-resolution via mixed high-order attention network,\u0026quot; \u003cem\u003eIEEE Transactions on Geoscience and Remote Sensing, \u003c/em\u003evol. 59, no. 6, pp. 5183-5196, 2020.\u003c/li\u003e\n\u003cli\u003eC. Dong, C. C. Loy, K. He, and X. Tang, \u0026quot;Image super-resolution using deep convolutional networks,\u0026quot; \u003cem\u003eIEEE transactions on pattern analysis and machine intelligence, \u003c/em\u003evol. 38, no. 2, pp. 295-307, 2015.\u003c/li\u003e\n\u003cli\u003eC. Dong, C. C. Loy, and X. Tang, \u0026quot;Accelerating the super-resolution convolutional neural network,\u0026quot; in \u003cem\u003eEuropean conference on computer vision\u003c/em\u003e, 2016, pp. 391-407: Springer.\u003c/li\u003e\n\u003cli\u003eW. Shi\u003cem\u003e et al.\u003c/em\u003e, \u0026quot;Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2016, pp. 1874-1883.\u003c/li\u003e\n\u003cli\u003eK. He, X. Zhang, S. Ren, and J. Sun, \u0026quot;Deep residual learning for image recognition,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2016, pp. 770-778.\u003c/li\u003e\n\u003cli\u003eJ. Kim, J. K. Lee, and K. M. Lee, \u0026quot;Accurate image super-resolution using very deep convolutional networks,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2016, pp. 1646-1654.\u003c/li\u003e\n\u003cli\u003eJ. Kim, J. K. Lee, and K. M. Lee, \u0026quot;Deeply-recursive convolutional network for image super-resolution,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2016, pp. 1637-1645.\u003c/li\u003e\n\u003cli\u003eY. Tai, J. Yang, and X. Liu, \u0026quot;Image super-resolution via deep recursive residual network,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2017, pp. 3147-3155.\u003c/li\u003e\n\u003cli\u003eB. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, \u0026quot;Enhanced deep residual networks for single image super-resolution,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition workshops\u003c/em\u003e, 2017, pp. 136-144.\u003c/li\u003e\n\u003cli\u003eY. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, \u0026quot;Image super-resolution using very deep residual channel attention networks,\u0026quot; in \u003cem\u003eProceedings of the European conference on computer vision (ECCV)\u003c/em\u003e, 2018, pp. 286-301.\u003c/li\u003e\n\u003cli\u003eH. Ren, M. El-Khamy, and J. Lee, \u0026quot;Image super resolution based on fusing multiple convolution neural networks,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition workshops\u003c/em\u003e, 2017, pp. 54-61.\u003c/li\u003e\n\u003cli\u003eJ. Li, F. Fang, K. Mei, and G. Zhang, \u0026quot;Multi-scale residual network for image super-resolution,\u0026quot; in \u003cem\u003eProceedings of the European conference on computer vision (ECCV)\u003c/em\u003e, 2018, pp. 517-532.\u003c/li\u003e\n\u003cli\u003eF. Chollet, \u0026quot;Xception: Deep learning with depthwise separable convolutions,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2017, pp. 1251-1258.\u003c/li\u003e\n\u003cli\u003eA. G. Howard\u003cem\u003e et al.\u003c/em\u003e, \u0026quot;Mobilenets: Efficient convolutional neural networks for mobile vision applications,\u0026quot; \u003cem\u003earXiv preprint arXiv:1704.04861, \u003c/em\u003e2017.\u003c/li\u003e\n\u003cli\u003eX. Zhang, X. Zhou, M. Lin, and J. Sun, \u0026quot;Shufflenet: An extremely efficient convolutional neural network for mobile devices,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2018, pp. 6848-6856.\u003c/li\u003e\n\u003cli\u003eA. Krizhevsky, I. Sutskever, and G. E. Hinton, \u0026quot;Imagenet classification with deep convolutional neural networks,\u0026quot; \u003cem\u003eCommunications of the ACM, \u003c/em\u003evol. 60, no. 6, pp. 84-90, 2017.\u003c/li\u003e\n\u003cli\u003eB. Yang, G. Bender, Q. V. Le, and J. Ngiam, \u0026quot;Condconv: Conditionally parameterized convolutions for efficient inference,\u0026quot; \u003cem\u003eAdvances in Neural Information Processing Systems, \u003c/em\u003evol. 32, 2019.\u003c/li\u003e\n\u003cli\u003eY. Chen, X. Dai, M. Liu, D. Chen, L. Yuan, and Z. Liu, \u0026quot;Dynamic convolution: Attention over convolution kernels,\u0026quot; in \u003cem\u003eProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition\u003c/em\u003e, 2020, pp. 11030-11039.\u003c/li\u003e\n\u003cli\u003eX. Jia, B. De Brabandere, T. Tuytelaars, and L. Van Gool, \u0026quot;Dynamic filter networks for predicting unobserved views,\u0026quot; in \u003cem\u003eProceedings ECCV 2016 workshops\u003c/em\u003e, 2016, pp. 1-2.\u003c/li\u003e\n\u003cli\u003eY. Zhang, J. Zhang, Q. Wang, and Z. Zhong, \u0026quot;Dynet: Dynamic convolution for accelerating convolutional neural networks,\u0026quot; \u003cem\u003earXiv preprint arXiv:2004.10694, \u003c/em\u003e2020.\u003c/li\u003e\n\u003cli\u003eJ. Gu\u003cem\u003e et al.\u003c/em\u003e, \u0026quot;Recent advances in convolutional neural networks,\u0026quot; \u003cem\u003ePattern recognition, \u003c/em\u003evol. 77, pp. 354-377, 2018.\u003c/li\u003e\n\u003cli\u003eY. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, \u0026quot;Residual dense network for image super-resolution,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2018, pp. 2472-2481.\u003c/li\u003e\n\u003cli\u003eJ. Hu, L. Shen, and G. Sun, \u0026quot;Squeeze-and-excitation networks,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2018, pp. 7132-7141.\u003c/li\u003e\n\u003cli\u003eS. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, \u0026quot;Cbam: Convolutional block attention module,\u0026quot; in \u003cem\u003eProceedings of the European conference on computer vision (ECCV)\u003c/em\u003e, 2018, pp. 3-19.\u003c/li\u003e\n\u003cli\u003eE. Agustsson and R. Timofte, \u0026quot;Ntire 2017 challenge on single image super-resolution: Dataset and study,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition workshops\u003c/em\u003e, 2017, pp. 126-135.\u003c/li\u003e\n\u003cli\u003eM. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. A. Morel, \u0026quot;Neighbor embedding based single-image super-resolution using semi-nonnegative matrix factorization,\u0026quot; in \u003cem\u003e2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)\u003c/em\u003e, 2012, pp. 1289-1292: IEEE.\u003c/li\u003e\n\u003cli\u003eR. Zeyde, M. Elad, and M. Protter, \u0026quot;On single image scale-up using sparse-representations,\u0026quot; in \u003cem\u003eInternational conference on curves and surfaces\u003c/em\u003e, 2010, pp. 711-730: Springer.\u003c/li\u003e\n\u003cli\u003eP. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, \u0026quot;Contour detection and hierarchical image segmentation,\u0026quot; \u003cem\u003eIEEE transactions on pattern analysis and machine intelligence, \u003c/em\u003evol. 33, no. 5, pp. 898-916, 2010.\u003c/li\u003e\n\u003cli\u003eJ.-B. Huang, A. Singh, and N. Ahuja, \u0026quot;Single image super-resolution from transformed self-exemplars,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2015, pp. 5197-5206.\u003c/li\u003e\n\u003cli\u003eW.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, \u0026quot;Deep laplacian pyramid networks for fast and accurate super-resolution,\u0026quot; in \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 2017, pp. 624-632.\u003c/li\u003e\n\u003cli\u003eN. Ahn, B. Kang, and K.-A. Sohn, \u0026quot;Fast, accurate, and lightweight super-resolution with cascading residual network,\u0026quot; in \u003cem\u003eProceedings of the European conference on computer vision (ECCV)\u003c/em\u003e, 2018, pp. 252-268.\u003c/li\u003e\n\u003cli\u003eX. He, Z. Mo, P. Wang, Y. Liu, M. Yang, and J. Cheng, \u0026quot;Ode-inspired network design for single image super-resolution,\u0026quot; in \u003cem\u003eProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition\u003c/em\u003e, 2019, pp. 1732-1741.\u003c/li\u003e\n\u003cli\u003eF. Fang, J. Li, and T. Zeng, \u0026quot;Soft-edge assisted network for single image super-resolution,\u0026quot; \u003cem\u003eIEEE Transactions on Image Processing, \u003c/em\u003evol. 29, pp. 4656-4668, 2020.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Deep convolutional networks, Attention mechanism, Channel shuffle, Deep separable convolution, Feature compensation","lastPublishedDoi":"10.21203/rs.3.rs-2404952/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2404952/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eRecently, deep networks with multiple branches or different filtering methods have achieved good performance in image super-resolution, however, they ignore the fact that some high-frequency components are lost during multiple captures of contextual information by a single branch. To solve this problem, we propose a feature-compensated adaptive fusion shuffle network (CAFN) for image super-resolution. Specifically, we propose a feature-compensated shuffle model (FCSM). It uses branch compensation to compensate for the high frequency information lost by single channel feature transfer. Then, we propose an attention adaptive compensation fusion module (ACFM), which adaptively learns the weight information about the channel structure, and improves the branch compensation method to better adapt to the learned weight information. ACFM can further capture contextual dimension information and extract more high-frequency features. Experimental results show that our proposed CAFN has an excellent performance in terms of both visual results and evaluation metrics.\u003c/p\u003e","manuscriptTitle":"Feature Compensated Adaptive Fusion Shuffle Network for Image Super-Resolution Reconstruction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-01-03 15:20:57","doi":"10.21203/rs.3.rs-2404952/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"efb9d3f9-ce8d-4919-8acc-72b474e2362a","owner":[],"postedDate":"January 3rd, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2023-05-14T12:59:30+00:00","versionOfRecord":[],"versionCreatedAt":"2023-01-03 15:20:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-2404952","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2404952","identity":"rs-2404952","version":["v1"]},"buildId":"_2-kVJe1T_tPrBINL-cwx","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}