Work

Sequential Federated Fine-Tuning of Large Language Models for Clinical Outcome Prediction in Intensive Care Units

Year: 2026
Type: preprint
Abstract: Large language models (LLMs) show promise for clinical outcome prediction in the intensive care unit (ICU), but collaborative fine-tuning across hospitals is constrained by privacy and data-governance concerns. Federated learning (FL) enables collaborative training without exchanging raw data, but conventional parallel FL requires coordinated participation and centralized aggregation, which can be challenging when institutions differ in training availability. We develop and evaluate a novel framework called sequential federated fine-tuning (SFF) framework that enables efficient and privacy-aware collaborative clinical outcome prediction in ICUs across multiple hospitals. By transferring low-rank adapters sequentially from one hospital to the next, SFF supports asynchronous learning, allowing institutions to remain offline until their turn while another institution performs local training. Using 45,149 ICU stays from 27 hospitals in the eICU Collaborative Research Database (eICU-CRD), we applied low-rank adaptation (LoRA) to Qwen-2.5-7B-Instruct and LLaMA-3.1-8B-Instruct and compared SFF with centralized fine-tuning, local fine-tuning, zero-shot prompting, and federated averaging (FedAvg) for in-hospital mortality and three-class length-of-stay (LoS) prediction. SFF achieved performance close to centralized fine-tuning while outperforming local training and FedAvg. For mortality prediction, SFF reached AUROCs of 86.08% and 89.24% for Qwen-2.5-7B and LLaMA-3.1-8B, respectively; for LoS classification, it reached 75.32% and 76.57%. The advantage over FedAvg was observed across most hospital cohorts and remained evident when local training data were limited. Our work advances the practical, privacy-aware deployment of LLMs in multi-institutional clinical settings for collaborative clinical outcome prediction in ICUs.
Language: English
Open Access status: green
OpenAlex ID: W7235505461
Cites: 46
Cited by:
Related to: 10