Transformer-Based Siamese Textual Encoder for Knowledge Base Completion
Year: 2020
Type: dissertation
Abstract: Knowledge graph is constructed as a directed graph and regarded as a kind of knowledge base. Its nodes and edges represent named entities and relations between the entities, respectively. Knowledge graph is normally denoted as a collection of triples, i.e., (head/subject entity, relation, tail/object entity), for feasible processing. However, incompleteness or sparsity issue usually exists in knowledge graph since the knowledge graph are constructed via human labeling. Therefore, 𝘭𝘪𝘯𝘬 𝘱𝘳𝘦𝘥𝘪𝘤𝘵𝘪𝘰𝘯 task is proposed to complete a knowledge graph via predicting the missing links. \nPrevious approaches for link prediction on knowledge graph are mainly based on graph embedding, such as translation-based embedding approaches, convolutional-based embedding approaches, and graph convolutional network (GCN-based) approaches. These traditional graph embedding-based approaches straightforwardly learn the embeddings by considering a knowledge base’s structure but are inherently vulnerable to the graph’s sparseness or incompleteness issue. In contrast, some recent approaches, named textual encoding approaches here, learn the embeddings from a natural language processing perspective. That is, except for the structural information in a knowledge graph, the semantic information of the entities and relations, like their texts, mentions, and descriptions, are taken into account to enrich the embeddings of the knowledge graph. Hence, they capture such structured knowledge from the semantic perspective and employ deep neural text encoder to model graph triples in semantic space, but fail to trade off the contextual features or model’s efficiency. \nTherefore, in line with previous textual encoding approaches that represent graph components like entities or triples into latent semantic space, we propose a Siamese textual encoder operating on each graph triple from knowledge base, where the contextual features between a head/tail entity and a relation are well-captured to highlight relation-aware entity embedding while a Siamese structure is also adapted to avoid combinatorial explosion during inference of link prediction. \nIn the experiments, the proposed approach reaches the best or comparable performance on two link prediction datasets. And further compared to its baseline (i.e., a state-of-the-art textual encoding approach), our approach can accelerate the inference procedure by one or two orders of magnitude with an even better quality of predictions.
Author: Mengyao Li
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3193495838
