Work

Distilling Knowledge Based on Curriculum Learning for Temporal Knowledge Graph Embeddings

Year: 2024

Type: conference-paper

Abstract: Lower-dimensional temporal knowledge graph embedding (TKGE) models are crucial for practical applications and resource-limited scenarios, although existing models employ higher-dimensional embeddings in training. In this paper, we propose a new framework for distilling TKGE models via an easy to hard pedagogical principle. The framework utilizes a learnable curriculum temperature (CT) module to optimize and guide the knowledge distillation process dynamically, ensuring that the entire procedure adheres to the principle. It also employs a self-adaptive attention mechanism to endeavor to achieve efficient transfer of knowledge from higher-dimensional models to lower-dimensional ones. Evaluation on various TKGE models and datasets demonstrates the proposed approach significantly reduces the model's parameters without noticeably affecting its performance.

Source: ACM International Conference on Information and Knowledge Management (CIKM)

DOI: 10.1145/3627673.3679896

Authors: Bin Zhang, Jiayin Li, Yuanfei Dai

Institutions: Nanjing Tech University, Fujian Normal University

Language: English

Topic: Advanced Graph Neural Networks

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: bronze

Funders: National Natural Science Foundation of China, Government of Jiangsu Province, Natural Science Foundation of Jiangsu Province