LLM-Assisted Detection and Community-Based Review of Promotional Texts in OpenStreetMap: An Empirical Study
Year: 2026
Type: article
Abstract: Promotional and SEO-oriented free-text content can undermine the neutrality and quality of OpenStreetMap (OSM). Because such text is multilingual and context-dependent, purely rule-based detection is challenging. This study evaluates large language models (LLMs) for identifying promotional content in OSM description=* tags and investigates their use in a community-based quality assurance workflow. A classification prompt was developed using a pilot dataset and evaluated on a balanced, manually annotated multilingual dataset of 1500 descriptions. Thirty-five open-weight model configurations were compared using precision, recall, F1-score, and runtime, and two conventional text-classification baselines were evaluated for classification performance. The best-performing model, llama3.3:70b, achieved 95.03% precision, 91.87% recall, and an F1-score of 93.42%, outperforming the evaluated conventional baselines. This model was then applied to 16,156 newly added or modified OSM descriptions in a real-world workflow. It identified 2738 descriptions as potentially promotional, resulting in 2599 public changeset comments. Within a 28-day follow-up period, 79.5% of analyzed commented cases were modified or removed, compared with 27.3% during a later comparison period without workflow-generated comments. Because the two conditions were temporally separated and non-randomized, this difference should be interpreted as an observational association.
DOI: 10.3390/ijgi15110514
Author: Pascal Neis
Institution: University of Applied Sciences Mainz
Language: English
Subfield: Geography, Planning and Development
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,292
OpenAlex ID: W7236476378
