Work

Human Sentence Processing: Recurrence or Attention?

Year: 2021

Type: conference-paper

Abstract: Recurrent neural networks (RNNs) have long been an architecture of interest for computational models of human sentence processing.The recently introduced Transformer architecture outperforms RNNs on many natural language processing tasks but little is known about its ability to model human language processing.We compare Transformer-and RNNbased language models' ability to account for measures of human reading effort.Our analysis shows Transformers to outperform RNNs in explaining self-paced reading times and neural activity during reading English sentences, challenging the widely held idea that human sentence processing involves recurrent and immediate processing and provides evidence for cue-based retrieval.

Source: Workshop on Cognitive Modeling and Computational Linguistics (CMCL)

DOI: 10.18653/v1/2021.cmcl-1.2

Authors: Danny Merkx, Stefan L. Frank

Institution: Radboud University Nijmegen

Language: English

Topic: Neurobiology of Language and Bilingualism

Subfield: Cognitive Neuroscience

Field: Neuroscience

Domain: Life Sciences

Open Access status: hybrid

Funder: Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Awards: G2084304408, G4627615187