Work

Faster and Memory-Efficient Training of Sequential Recommendation Models for Large Catalogs

Year: 2026
Type: article
Abstract: Transformer-based sequential recommender systems are retrained frequently, yet their training is dominated by cross-entropy (CE) over item catalogs that can reach millions of items. Its logit tensor scales with catalog size × batch size × sequence length, forcing practitioners to trade these quantities against each other or use negative sampling. However, how to allocate memory among negatives, batch size, and sequence length remains an open question. We study this trade-off with nearly 1,000 SASRec training runs on six datasets and find that no single memory-related hyperparameter should be maximized in isolation. Scaling both the number of negatives and batch size outperforms maximizing only one, while the choice between full CE and CE with negative sampling is dataset-dependent. We further introduce a memory-efficient and GPU-optimized replacement for CE with negative sampling that avoids materializing the logit tensor. It speeds up training by up to 3.5× and reduces memory by 4–24× relative to PyTorch CE with negative sampling. Reallocating memory saved by our replacement and Cut Cross-Entropy, which we adapt to sequential recommender systems, into larger batches, longer sequences, and more negatives improves NDCG@10 by up to 54.7% under the same memory budget. We release the Triton kernel and a public dataset of all training runs at https://github.com/On-Point-RND/MemoryEfficientSRS .
Language: English
Open Access status: bronze
OpenAlex ID: W4414591557
FWCI: 2.197
Cites: 42
Cited by: 2
Related to: 10