Advancing thermal image object detection for autonomous driving in adverse weather: integrating semi-supervision with a spatial edge-aware attention mechanism

Abstract

Thermal imaging has emerged as a critical sensing modality for object detection in low-visibility driving conditions, where conventional visible-spectrum (RGB) sensors often fail. However, thermal images also exhibit inherent limitations, including low spatial resolution and weak object boundaries. As a result, small or distant objects may be represented by only a few pixels, with faint and ambiguous edge information. To address these challenges, this thesis proposes two novel methodologies that aim to enhance detection robustness and accuracy without increasing computational complexity. The first methodology introduces a Spatial Edge-Aware Attention (SEA) module, integrated into a YOLOv8 general object detection backbone. In this approach, a conventional Sobel filter, together with multilayer convolutional subnetworks, is employed to extract edge features directly from raw thermal images. These edge features are then injected into the SEA module, enabling the proposed detector to emphasize object boundaries and improve the localization of indistinct targets in challenging scenarios. Building upon the first methodology, the second approach introduces a semi-supervised learning framework to further improve detection performance. Specifically, a self-supervised rotation prediction pretext task is employed during a warm-up training phase, allowing the model to learn orientation-invariant representations from unlabeled thermal data. This learned representation is subsequently leveraged during downstream training with limited labeled data, enabling effective semi-supervision. Together, these two methodologies address key limitations of existing thermal object detection techniques for adverse weather conditions. Experimental evaluations conducted on multiple benchmark datasets demonstrate that the proposed approaches achieve improved detection accuracy and robustness while maintaining computational efficiency.