Work

Estimating urban flooding depth by integrating multimodal image-text data: A segment-level direct preference optimization-based multimodal large language model

Year: 2025

Type: article

Abstract: Urban flood mapping Massive and multi-dimensional social media data provide precious opportunities for the rapid collection and assessment of urban flooding depth. However, effectively and robustly estimating water depth from these multimodal data remains a significant challenge. Although previous studies integrated several existing models, they increase model complexity and hinder joint optimization across different modalities. This paper proposes a Segment-level Direct Preference Optimization-based Multimodal Large Language Model (SDPO-MLLM) for estimating flood depth by integrating image-text data. Our contributions include the design of a hybrid training strategy combining Supervised Fine-Tuning (SFT) and SDPO to reduce inaccurate responses. Additionally, a novel structured workflow is designed, including: (1) dataset preprocessing and construction; (2) event-based extraction of flood location and depth descriptions from text; (3) generation of water depth descriptions from images and videos; (4) classification of water depth descriptions based on multiple reference objects; and (5) quantification of depth categories into numerical values. Empirical experiments are conducted on a dataset containing 2843 text records and 1563 images. The evaluation results show that SDPO-MLLM outperforms other unimodal methods, generating structured and organized results from text, and identifying flooding depth from images based on reference objects. As a case study in Wuhan, Shenzhen and Beijing, the multimodal water depth extracted from social media data is quantified and fused to map and analyze waterlogging-prone areas, demonstrating satisfactory generalization and adaptability of the developed model under various flood scenarios. Our research offers valuable insights for rapid mapping and analysis of urban waterlogging severity.

Source: ISPRS Journal of Photogrammetry and Remote Sensing

DOI: 10.1016/j.isprsjprs.2025.10.013

Authors: Tianyou Chu, Yumin Chen, Rui Zhu, Fei Zeng

Institutions: Wuhan University, Agency for Science, Technology and Research, Institute of High Performance Computing, Chinese People's Liberation Army

Language: English

Topic: Flood Risk Assessment and Management

Subfield: Global and Planetary Change

Field: Environmental Science

Domain: Physical Sciences

SDGs: Sustainable cities and communities, Clean water and sanitation

Open Access status: hybrid

Estimated APC paid: $3,310