Year: 2020
Type: conference-paper
Abstract: The recent paradigm shift to contextual word embeddings has seen tremendous success across a wide range of down-stream tasks.However, little is known on how the emergent relation of context and semantics manifests geometrically.We investigate polysemous words as one particularly prominent instance of semantic organization.Our rigorous quantitative analysis of linear separability and cluster organization in embedding vectors produced by BERT shows that semantics do not surface as isolated clusters but form seamless structures, tightly coupled with sentiment and syntax.
Authors: David Yenicelik, Florian Schmidt, Yannic Kilcher
Institution: ETH Zurich
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W3102970018
