Modality Correlation Aware Sparse Representation and Kiou Tracker for Rgb-infrared Videos

Abstract Visual tracking of objects in videos has a wide range of applications, especially in large-scale applications where it is not practical to attach markers to many objects for conventional, marker-enabled tracking methods. These applications include video surveillance, construction, traffic, logistics, etc. Despite the development of several tracking algorithms for RGB videos over the past ten years, tracking performance and resilience of these systems may suffer significantly when RGB video information is inaccurate due to bad lighting or very low resolution. This study offers a new tracking system that tries to merge data from RGB and infrared modalities for object tracking in order to overcome this problem. The suggested tracking system merges a Kalman filter with an IOU-based track association approach, a machine learning model, and a Kalman-Intersection-Over-Union (KIOU) tracker for object tracking in films. Specifically, the learning model can generate discriminative feature templates for collaborative representations and discrimination in heterogeneous modalities, which can solve the modality discrepancy problem under the proposed modality consistency constraint from both representation patterns and discriminability. The effectiveness of the suggested approach is demonstrated by tests on several difficult RGB-Infrared films.