Enhanced Thermal-Only Object Detection via LoRA-Guided Thermal-to-Visible Translation and Cross-Modal Distillation

초록

Thermal infrared (IR) object detection is crucial for robust perception in adverse conditions where visible cameras fail. However, thermal images lack the rich texture and color information inherent in visible data, limiting the performance of detectors trained solely on thermal inputs. To address this, we propose a novel thermal-only object detection framework that bridges the modality gap via LoRA-guided Thermal-to-Visible (T2V) translation and cross-modal knowledge distillation. Specifically, we introduce a parameter-efficient fine-tuning strategy using Low-Rank Adaptation (LoRA) applied to the convolutional layers of a pre-trained T2V generator. This allows the model to synthesize high-fidelity pseudo-RGB images and extract latent semantic features from thermal inputs without heavy computational overhead. These generated cues are then transferred to a thermal-only student detector through a comprehensive distillation framework, which aligns both intermediate feature distributions via Maximum Mean Discrepancy (MMD) and final detection responses. Extensive experiments on the FLIR Aligned and LLVIP datasets demonstrate that our method achieves state-of-the-art performance among monomodal approaches. Notably, our framework exhibits exceptional robustness to illumination changes, maintaining high detection accuracy across both daytime and nighttime scenarios while effectively compensating for the thermal crossover effect. The proposed student model achieves 40.4% mAP on the FLIR dataset, significantly outperforming existing thermal-only methods while requiring only a single thermal sensor at inference. © 2013 IEEE.

키워드