Abstract
Machine perception gathers information about its surroundings through sophisticated sensors. Thermal sensors offer scalable perception in conditions of low visibility such as night time, smoke or fog. However, integrating thermal data with deep learning algorithms is a challenge because of the scarcity of thermal data due to the high cost of thermal sensors. In this paper, we propose for the first time, a deep learning-based framework for thermal video synthesis as an affordable alternative to purchasing costly thermal imaging devices. Here, we have introduced a diffusion model for estimating thermal videos from videos in the visible spectrum. Results show that our Thermal VideoDiff (TVD) is capable of synthesizing high fidelity video samples and captures temperature variations from thermal data effectively. Our work addresses the challenge posed by the scarcity of thermal data, as well as brings deep learning to the domain of infrared video generation, enabling research and development in the infrared domain.
Fundamentals of Thermal Imaging
Incident (I) = Transmitted (T) + Absorbed (A) + Reflected (R) + Emitted (E)
Illustration: When light is incident on an object, a fraction of the radiation is absorbed, another fraction is reflected, and the remainder is transmitted, with subsequent emission. The emitted radiation depends upon factors such as temperature, material properties, and atmospheric conditions. RGB sensors detect the reflected radiation within the visible spectrum (0.4-0.7µm), while infrared sensors capture the emitted radiation (0.7-15µm). Visible and thermal images are fundamentally different and therefore, deriving emissivity solely from reflected radiation is theoretically implausible.
Visible images and their corresponding thermal images from the KAIST and FLIR datasets, illustrating the advantages of thermal imaging in night and low-illumination conditions.
Citation
For more details and additional results, read the full paper.
@article{qazi2024thermal,
title={THERMAL VIDEODIFF (TVD): A DIFFUSION ARCHITECTURE FOR THERMAL VIDEO SYNTHESIS},
author={Qazi, Tayeba and Lall, Brejesh},
booktitle={2023 IEEE International Conference on Image Processing (ICIP)},
year={2024}}