Get started
Work

On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View

Year: 2025
Type: preprint
Abstract: Federated Learning (FL) enables training models across decentralized data silos while preserving client data privacy. Recent research has explored efficient methods for post-training large language models (LLMs) within FL to address computational and communication challenges. While existing approaches often rely on access to LLMs' internal information, which is frequently restricted in real-world scenarios, an inference-only paradigm (black-box FedLLM) has emerged to address these limitations. This paper presents a comprehensive survey on federated tuning for LLMs. We propose a taxonomy categorizing existing studies along two axes: model access-based and parameter efficiency-based optimization. We classify FedLLM approaches into white-box, gray-box, and black-box techniques, highlighting representative methods within each category. We review emerging research treating LLMs as black-box inference APIs and discuss promising directions and open challenges for future research.
DOI: 10.48550/arxiv.2508.16261
Authors: Guo, Tao, Wang, Junxiao, Huo, Fushuo, Cui, Laizhong, Guo, Song, Gui, Jie, Tao, Dacheng
Institutions: Shenzhen University, Guangzhou University, Hong Kong Polytechnic University, Hong Kong University of Science and Technology, Southeast University, Nanyang Technological University
Language: English
Open Access status: green
OpenAlex ID: W4417060269
Cites:
Cited by:
Related to: