DisClass: Classification Model for Disaster-Related Visual and Textual Data
Year: 2026
Type: article
Abstract: Social media platforms continue to generate vast volumes of information during natural disasters. A large number of social media posts are in the form of text and images. There is a need for automated systems that can process this heterogeneous textual and visual content. In crisis settings, such data offers rich but noisy signals about event severity, public distress, infrastructure damage, and emergent needs. Current approaches have attempted to reconcile conflicting and incomplete cross-modal cues. These approaches are usually supervised in nature and require extensive human-annotated data. Collating and cataloguing annotated data during fast-evolving emergencies is an arduous task, and, if performed, can delay disaster response. Our study proposes a unified multi-task learning framework that addresses these limitations through a novel model which includes a Bi-modal Interaction Cross-Attention module that is integrated into a Single-stream And Cross-modal Multi-Task architecture architecture. We call this a BIC-SACMT Pipeline. Our framework introduces a rule-driven pseudo-label generation strategy for automatically inferring urgency and relevance levels from raw text and images. With this model, we attempt large-scale supervision without reliance on manual annotation. The SACMT module facilitates fine-grained alignment between modalities and encapsulates both supportive and suppressive interactions when text and images diverge in meaning. This produces a unified representation that enhances downstream classification across multiple disaster-related tasks. It uses multi-task training, cross-modal fusion, and distribution alignment. Through our model, we aim to demonstrate that robust multi-modal disaster assessment is possible even under noisy supervision. Our goal is to establish that multi-modal pseudo-labeling combined with cross-attention-based fusion can transform unannotated crisis data into useful insights that support emergency responders and policy planners in real time.
Institution: University of Missouri
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204471670
FWCI:
Cites:
Cited by:
Related to: