Integrating Deep Learning-Based Techniques and Multi-Modal Fusion of Thermal and Visual images for improved Object Detection

Multi-modal fusion of thermal and visible imagery enhances object detection in challenging conditions like low light or adverse weather. Thermal imaging excels in surveillance, autonomous navigation, and search-and-rescue by capturing heat signatures independent of lighting. However, thermal images suffer from low contrast, limited texture, and noise, hindering detection. This article proposes a deep learning approach using fused RGB-thermal inputs to improve thermal object detection via You-Only-Look-Once (YOLO) based models. Discrete Wavelet Transform (DWT) and Local Binary Patterns (LBP) are applied for feature extraction and image fusion, combining complementary spatial and textural details. Co-registered night-time RGB and thermal images are pre-processed to maximize modality synergy and annotated using Roboflow for consistent, high-quality labels. Results show multimodal fusion significantly boosts detection accuracy, especially localization precision—critical for real-world deployment. Visually, DWT and LBP fused images enhance interpretability and object boundary clarity over raw inputs. Quantitatively, DWT-fused models outperform both single-modality baselines in mAP and recall, while LBP fusion surpasses RGB-only but slightly underperforms thermal-only detection, highlighting modality-specific advantages. The study demonstrates that multimodal integration strengthens feature robustness and detection reliability in low-visibility environments. It also reviews key datasets and benchmarks in thermal vision research, contributing to the development of more resilient deep learning-based systems for practical applications.