Advancing Thermal Multi-Object Tracking with Attention and Metric Fusion

Abstract Multiple-Object Tracking (MOT) is a fundamental task in computer vision with many applications. For practical operations, tracking for monitoring with thermal imaging unaffected by lighting conditions is important. However, most MOT methods are proposed to analyze video streams from RGB cameras, while there are few datasets and research on multi-object tracking in infrared image sequences. In this paper, we provide a new infrared dataset for object detection and tracking, which contains small objects and occlusion challenges. We also propose a new robust tracker, which enhances object detection with the strategic integration of the Convolutional Block Attention Module (CBAM) into the YOLOv7 model, along with specialized fusion of IoU, Size, and ReID features during data association to overcome the challenges of thermal images. Our tracker achieves 59.29 HOTA, 73.46 MOTA, and 74.4 IDF1 as a new state-of-the-art on the CAMEL benchmark. The tracker's source code and dataset are publicly available at: https://github.com/aquarter147/TMTV_Thermal_MOT