Analogies minus analogy test: measuring regularities in word embeddings
Year: 2020
Type: conference-paper
Abstract: Vector space models of words have long been claimed to capture linguistic regularities as simple vector translations, but problems have been raised with this claim. We decompose and empirically analyze the classic arithmetic word analogy test, to motivate two new metrics that address the issues with the standard test, and which distinguish between class-wise offset concentration (similar directions between pairs of words drawn from different broad classes, such as France--London, China--Ottawa, ...) and pairing consistency (the existence of a regular transformation between correctly-matched pairs such as France:Paris::China:Beijing). We show that, while the standard analogy test is flawed, several popular word embeddings do nevertheless encode linguistic regularities.
Authors: Louis Fournier, Emmanuel Dupoux, Ewan Dunbar
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: Empirical Foundations of Linguistics : data, methods, models, Paris Sciences et Lettres, Frontières en cognition, G4303677324, Speech perception and learning in the geometric typology of phonological inventories, PaRis Artificial Intelligence Research InstitutE, G782245399, Université de Paris
OpenAlex ID: W3092565990