Applying and adapting established frameworks for geospatial data quality to evaluate Generative AI in geographic information tasks
Year: 2025
Type: article
Abstract: Challenges such as hallucination and non-determinism are well known to undermine the reliability of Generative AI (GenAI) applications. Despite this, their use across multiple fields and domains continues to rapidly grow. Many GenAI applications now generate or interact with geospatial information – leading to the potential proliferation of non-human generated geospatial data. Because such tools can be developed without specialist geographic expertise, there is a growing risk of diminished oversight and declining geospatial data quality. This paper presents an approach for evaluating the geospatial information produced by large language models (LLMs) using established frameworks for geospatial data quality. We apply this framework in two experiments that assess the completeness, accuracy, precision, and consistency of LLM outputs: geoparsing and route finding. Interpreting these results through ontological and teleological perspectives reveals fundamental limitations in how LLMs represent and reason about space. The study highlights the need for responsible evaluation and adaptation of data quality standards as GenAI becomes increasingly embedded within geographic information science.
Authors: Joseph Shingleton, Ana Basiri
Institutions: University of Glasgow, The Alan Turing Institute
Language: English
Subfield: Geography, Planning and Development
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
Funder: UK Research and Innovation
Awards: G3007901435, G3741620067
OpenAlex ID: W4417165972