Work

Multimodal GeoAI: An integrated spatio-temporal topic-sentiment model for the analysis of geo-social media posts for disaster management

Year: 2025

Type: article

Abstract: The analysis of online communication on social networks has become a central research interest to improve disaster management. Especially geo-referenced textual posts have been investigated extensively using techniques from Geospatial Artificial Intelligence (GeoAI), topic modelling and sentiment analysis. However, workflows are traditionally sequential with independent processing steps, limiting their ability to capture interconnections between modalities and risking chains of dependencies. To overcome these limitations, we introduce an integrated GeoAI model called the Joint Spatio-Temporal Topic-Sentiment (JSTTS) model. Our proposed method combines semantic, sentiment, spatial and temporal knowledge into continuous feature vectors and is capable of computing geographically delineated sentiment-associated clusters of topics with meaningful location and temporal information. The properties of the JSTTS model were validated experimentally and the approach was evaluated against a comparable sequential workflow. Overall, our JSTTS model achieved higher topic quality scores with an average of 0.145 compared to 0.034 for the sequential workflow and higher average sentiment uniformity with an average of 0.89 versus 0.73. At the same time, both approaches exhibited similar spatial and temporal variance. As a secondary result, the Geographic Growing Self-Organising Map (Geo-GSOM) was developed to cluster multimodal feature vectors meaningfully in geographic space. It was evaluated on artificial training data where it reproduced up to 90% of the spatial autocorrelation while the non-spatial Growing Self-Organising Map (GSOM) only achieved 55%. The learned neuron grid can also be interpreted geographically. The utility of the JSTTS approach is demonstrated through a case study on the 2021 Ahr Valley flooding in Western Germany, where it identified interpretable multimodal clusters, a subset of which proved relevant for disaster management. The approach can be extended to other use cases and adapted for different modalities, holding potential for numerous follow-up studies.

Source: International Journal of Applied Earth Observation and Geoinformation

DOI: 10.1016/j.jag.2025.104540

Authors: David Hanny, Bernd Resch

Institutions: University of Salzburg, Interdisciplinary Transformation University Austria, Harvard University

Language: English

Topic: Sentiment Analysis and Opinion Mining

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $3,550

Funders: Bundesministerium für Klimaschutz, Umwelt, Energie, Mobilität, Innovation und Technologie, HORIZON EUROPE Framework Programme