Year: 2021
Type: preprint
Abstract: In recent years, transformer-based language models have achieved state of the art performance in various NLP benchmarks. These models are able to extract mostly distributional information with some semantics from unstructured text, however it has proven challenging to integrate structured information, such as knowledge graphs into these models. We examine a variety of approaches to integrate structured knowledge into current language models and determine challenges, and possible opportunities to leverage both structured and unstructured information sources. From our survey, we find that there are still opportunities at exploiting adapter-based injections and that it may be possible to further combine various of the explored approaches into one system.
Source: arXiv (Cornell University)
Authors: Colon-Hernandez, Pedro, Havasi, Catherine, Alonso, Jason, Huggins, Matthew, Breazeal, Cynthia
Institutions: MIT Media Lab, Massachusetts Institute of Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3126974869