Connecting Science and Stories: A Human-AI Method for Multi-Modal Knowledge Capture that Integrates Sensors, Community Narratives, and Computational Decision Models
Year: 2025
Type: conference-paper
Abstract: Traditional decision support systems often fail to bridge scientific models with community knowledge, creating an implementation gap. The AI-enabled Modeling project (AIM) presents a novel approach that semantically links computational narratives with geospatial sensor data and physics-based models across domains. By leveraging large language models, semantic model integration, and multi-modal analysis, we establish technical services that enable integration of qualitative and quantitative knowledge systems. Our framework transforms unstructured stakeholder narratives into model configurations, generating culturally relevant representations of past events for complex socio-environmental challenges like hurricane impacts. The AIM project leverages, DataX, a data portal built on reusable science gateway components, incorporating CKAN-based data discovery, semantic variable mapping, interactive geospatially sensed visualization through Potree point cloud rendering, and a model integration platform. We demonstrate how unstructured interview transcripts can be systematically linked to data or physics-based models through a combination of natural language processing, standardized scientific vocabularies, and reusable computational cookbooks. This infrastructure enables transdisciplinary teams to connect lived experiences with observational data and computational simulations for technical integration in complex socio-environmental systems analyses.
Authors: Pierce, Suzanne A, Mobley, William, Fletcher, Lydia A. M., Vendelev, Vladislav, Osorio, Maximiliano
Language: English
Subfield: Geology
Field: Earth and Planetary Sciences
Domain: Physical Sciences
SDG: Climate action
Open Access status: green
OpenAlex ID: W7095061873
FWCI:
Cites:
Cited by:
Related to: