Work

Efficient Embedding Initialization via Dominant Eigenvector Projections

Year: 2025

Type: conference-paper

Abstract: The embedding layer is essential in deep learning, transforming high-dimensional data into compact representations. However, growing datasets and model sizes pose challenges in training time, memory, and generalization. We propose a scalable method for embedding initialization via spectral dimensionality reduction using dominant eigenvector projections.

DOI: 10.1145/3731599.3767541

Authors: Quentin Petit, Chong Li, Nahid Emad, Jack Dongarra

Institutions: Université Paris Sciences et Lettres, Centre de Recherche en Informatique, École Nationale Supérieure des Mines de Paris, Huawei Technologies (France), Université Paris-Saclay, Maison de la Simulation, University of Tennessee at Knoxville

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4416004080

FWCI:

Cites: 29

Cited by: 1

Related to: