Multi-scale Self-attention Recursive Hierarchical Network Based on Improved Residual Conv-LSTM

Abstract Super-resolution Reconstruction (SR) refers to the process of obtaining high resolution (HR) images from low resolution (LR) images. In recent years, with the vigorous development of deep learning, SR technology has been more widely applied, especially those based on convolution neural network (CNN). Although some existing methods use multi-scale convolution for feature extraction, most of them improve network performance by deepening network depth, often ignoring the intrinsic correlation between different hierarchic features. As is known to all, the increase of network depth may lead to training difficulties, insufficient feature extraction, feature details loss and other problems, which seriously limit the applications of network models in practice. Based on this, this paper proposes a multi-scale self-attention recursive hierarchical network (M-SRHN) based on improved residual Conv-LSTM. The proposed network model can achieve good reconstruction performance without sacrificing too many resources. Specifically, a multiscale guided learning self-attentional residual block (MGSRB) is first designed. It has four dilated convolutions and an enhanced spatial self-attention mechanism (ESA). This block can adaptively recalibrate the response of feature space by explicitly modeling the interdependence between spaces. Then, an improved residual LSTM (IRC-LSTM) network is proposed to memorize the output features of MGSRB, which can effectively process and store the memorized feature information. Further, a deep pyramid hierarchical module (DPHB) is used to extract more effective hierarchical information, and multiple MGSRBs, IRC-LSTM and DPHB are stacked to form the main framework of our network. Finally, a recursive subpixel reconstruction network is used to reconstruct images. Compared with some state-of-the-art SR methods, the proposed method had better reconstruction performance.