Work

Cross‐Modal Activation, Integration, and Validation When Reading Illustrated Texts: Evidence From Reading Times and Eye‐Tracking

Year: 2026

Type: article

Abstract: ABSTRACT The experiments investigated cognitive processes involved in reading illustrated text, focusing on cross‐modal activation, integration, and validation between text and picture. Using an adapted contradiction paradigm, both studies manipulated, within a single illustrated narrative, whether a previously presented picture was consistent or inconsistent with a later target sentence. Experiment 1 ( N = 285) employed sentence‐by‐sentence presentation and key‐press reading times, conceptually replicating earlier findings. Experiment 2 ( N = 143) extended the paradigm to a more naturalistic context, presenting short paragraphs on each page and using eye‐tracking. Across both experiments, inconsistencies led to longer processing times: Experiment 1 revealed prolonged reading times for the spillover sentence; Experiment 2 showed longer processing of the target sentence. Increased gaze transitions between the target sentence and the picture presented on the same page were observed. These results provide preliminary evidence that readers engage in cross‐modal activation, integration, and validation processes when reading illustrated texts.

Source: Applied Cognitive Psychology

DOI: 10.1002/acp.70277

Authors: Pauline Frick, Emely Hoch, Anne Schüler

Institutions: Leibniz-Institut für Wissensmedien, University of Tübingen, University of Zurich

Language: English

Topic: Reading and Literacy Development

Subfield: Developmental and Educational Psychology

Field: Psychology

Domain: Social Sciences

Open Access status: hybrid

Estimated APC paid: $2,900

Funder: Deutsche Forschungsgemeinschaft