LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Year: 2026
Type: preprint
Abstract: We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3\% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems.
Source: arXiv (Cornell University)
Authors: Xiong, Lee, Chen, Zhirong, Mayuranath, Rahul, Qiu, Shangran, Ozdemir, Arda, Li, Lu, Hu, Yang, Li, Dave, Ren, Jingtao, Cheng, Howard, Herrera, Fabian Souto, Agiza, Ahmed, Epshtein, Baruch, Aggarwal, Anuj, Ulziisaikhan, Julia, Wang, Chao, Ramasamy, Dinesh, Doshi, Parshva, Reddy, Sri, Overwijk, Arnold
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7126095296
Cites:
Cited by:
Related to: