Work

‘Indicatements’ that character language models learn English morpho-syntactic units and regularities

Year: 2018

Type: conference-paper

Abstract: Character language models have access to surface morphological patterns, but it is not clear whether or how they learn abstract morphological regularities.We instrument a character language model with several probes, finding that it can develop a specific unit to identify word boundaries and, by extension, morpheme boundaries, which allows it to capture linguistic properties and regularities of these units.Our language model proves surprisingly good at identifying the selectional restrictions of English derivational morphemes, a task that requires both morphological and syntactic awareness.Thus we conclude that, when morphemes overlap extensively with the words of a language, a character language model can perform morphological abstraction.

Source: BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP)

DOI: 10.18653/v1/w18-5417

Authors: Yova Kementchedjhieva, Adam Lopez

Institutions: University of Copenhagen, University of Edinburgh

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W2891343966

FWCI: 0.817

Cites: 14

Cited by: 11

Related to: 10