Work

When Discourse Pressures Conflict: Information Structure in Vision-Language Model Outputs

Year: 2026

Type: conference-paper

Abstract: Vision-language models (VLMs) are increasingly evaluated for whether they identify the right visual content, but little is known about whether they express such content in a discourse-appropriate form.We address this research gap using information structure (IS), testing whether VLMs distinguish discourseold Topics from discourse-new Foci in visually grounded question answering.We exploit Hungarian, a language in which Topic and Focus map onto dedicated syntactic positions, making IS choices observable in text.Comparing six VLMs with human participants, we find that models produce IS-relevant constructions, but over-regularise this sensitivity.Under the interacting pressures of discourse status, grammatical role (preference for subject Topics) and definiteness (preference for indefinite Foci), humans choose variable strategies for IS realisation.VLMs, by contrast, collapse onto narrow response templates, resembling mode collapse (Kirk et al., 2024).Our findings suggest that VLM evaluation should look beyond content accuracy to how content is packaged for the discourse.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/2026.emnlp-main.1499

Authors: Marcell Fekete, Johannes Bjerva, Tamás Káldi

Institutions: Aalborg University, Eötvös Loránd University, Max Planck Institute for Psycholinguistics, Hungarian University of Sports Science

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: Novo Nordisk, Nemzeti Kutatási Fejlesztési és Innovációs Hivatal, National Research, Development and Innovation Office

Award: G8046319854