Transformer Tracker based on Multi-level Residual Perception Structure

Abstract Recently, Transformer networks have been used for feature extraction and calculation of similarity in object tracking, replacing the previous use of CNN for feature extraction and cross-attention for feature fusion in Transformer based object tracking algorithms. This new structure is called one stream structure, and has achieved good results. However, the one stream structure of the Transformer tracker has too many network parameters, which limits the tracking speed of the network. For this reason, we have designed a pure one stream Transformer structure that uses soft segmentation operations to significantly reduce model parameters and computational complexity. In order to further improve the accuracy of tracking, we propose a multi-level residual perception structure to enhance the feature information of the target and reduce the background feature information, thereby enhancing the foreground and background discrimination ability of the model. In order to prove the speed and accuracy of our method, we not only compared it with algorithms using deep neural network models, but also compared it with UAV tracking algorithms using shallow networks. Compared with these algorithms, it can be proved that our algorithm not only has fast reasoning speed, but also has very high reasoning accuracy. Experimentally, the UAV123 dataset reached the level of SOTA; the inference speed can reach 130 FPS.