Work

Bottom-up building exposure modeling with multimodal earth vision

Year: 2025

Type: article

Abstract: Effective disaster mitigation and management rely on up-to-date exposure models providing detailed and spatially localized information on vulnerability-relevant characteristics of buildings. This study investigates the potential of heterogeneous multimodal geo-image data—incorporating street-level imagery (SLI), very high-resolution optical remote sensing data, and a normalized digital surface model—for generic large-area building characterization. We introduce a deep multimodal multitask learning methodology for the synergistic fusion of multi-sensor data and efficient multi-criteria building classification. The proposed task-wise modality attention (TMA) fusion optimizes multimodal feature representations for individual inference tasks separately according to their specific requirements. To address the challenge of partially missing SLI data ( i.e. , the missing modality problem ), a transformer-based SLI spatial context encoder leverages spatial correlations between structural building attributes and their visual manifestations to make the semantic information from available SLI widely accessible. With the earthquake-prone metropolis Santiago de Chile as test site, the two scenarios— SLI available and SLI missing —are evaluated through a comprehensive experimental cross-comparison of estimated generalization accuracies for classifying buildings according to five target variables: height , lateral load-resisting system material , seismic building structural type , roof shape , and block position . The results underscore the significant potential of the employed modalities and methods. Across the five addressed attributes, covering a total of 35 thematic classes, the most accurate models achieve mean accuracies of 85.19% and 74.96% for data points with and without SLI coverage, respectively. The presented data and methods allow to generate an area-wide building exposure model with a unique combination of thematic resolution, spatial detail and coverage.

Source: ISPRS Journal of Photogrammetry and Remote Sensing

DOI: 10.1016/j.isprsjprs.2025.10.036

Authors: Patrick Aravena Pelizari, Christian Geiß, Hannes Taubenböck

Institutions: University of Bonn, Deutsches Zentrum für Luft- und Raumfahrt e. V. (DLR), University of Würzburg

Language: English

Topic: Automated Road and Building Extraction

Subfield: Ocean Engineering

Field: Engineering

Domain: Physical Sciences

SDG: Sustainable cities and communities

Open Access status: hybrid

Estimated APC paid: $3,310