Work

Federated LLM Pre-Training on Mobile Phones

Year: 2025

Type: conference-paper

Abstract: Over the past decades, billions of mobile phones have become the primary interface to the Internet, accessing vast amounts of private user data. These devices are idle most of the time or become obsolete after a few years. Pre-training Large Language Models (LLMs) on ubiquitous mobile phones offers a promising way to utilize both private data and idle computing power. In this work, we propose the first federated LLM pre-training framework for mobile devices and demonstrate that it can potentially achieve wall-clock training time comparable to centralized pre-training.

Source: International Conference on Mobile Systems, Applications, and Services

DOI: 10.1145/3711875.3736664

Author: Dongqi Cai

Institutions: Beijing University of Posts and Telecommunications, University of Cambridge

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Royal Academy of Engineering, National Natural Science Foundation of China

Awards: G7483642958, G8529090286