Work

Unsupervised Diffusion-trajectory Segmentation and Multiregion Fusion for Understanding RGB–thermal Scene

Year: 2026

Type: article

Abstract: Multispectral scene understanding remains a critical sensing challenge: visible-spectrum (RGB) image sensors provide high-resolution color and texture cues but fail under poor illumination, while thermal infrared sensors operating on the principle of detecting heat radiation emitted by objects are inherently robust to low-light conditions yet deliver lower spatial resolution and less distinct contrast characteristics.The effective fusion of these two sensor modalities is therefore essential for reliable autonomous perception in real-world environments.To address the sensing limitations of individual modalities and the scarcity of large, densely annotated RGB-thermal (RGB-T) datasets, we propose a hierarchical, multistage RGB-T framework that (1) performs adaptive layer-wise fusion of multiscale features from a dual-stream backbone (ResNet-18 for RGB sensor data, and a custom convolutional neural network with squeeze-and-excitation blocks for thermal sensor data), (2) produces sharp, unsupervised region segmentations by mining latent diffusion trajectories of a pretrained diffusion model, and (3) aggregates region features via a graph-structured hierarchical fusion network for robust scene reasoning.The proposed sensing pipeline preserves complementary local and global cues from both modalities, yields semantically coherent region proposals without manual annotations, and models region-to-region context for stronger scene descriptors.Evaluated on RGB-T benchmarks, the integrated pipeline improves segmentation (Mean Intersection over Union ≈ 0.758) and scene classification (mean accuracy ≈ 88%) over representative baselines, demonstrating superior boundary precision, object separation, and robustness to clutter and occlusion, highlighting its value for sensor-driven autonomous scene understanding systems.

Source: Sensors and Materials

DOI: 10.18494/sam6151

Authors: Muhammad Waqas Ahmed, Tingting Xue, Nouf Abdullah Almujally, Ahmad Jalal, Hui Liu

Institutions: University of Bremen, Nanjing University of Information Science and Technology, Korea University, Air University

Language: English

Topic: Advanced Image Fusion Techniques

Subfield: Media Technology

Field: Engineering

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $748

Funders: Deutsche Forschungsgemeinschaft, Princess Nourah Bint Abdulrahman University, Universität Bremen