LIFT: Client-Shared Latent Space for Federated Learning in Heterogeneous Environments
Year: 2026
Type: conference-paper
Abstract: Federated learning (FL) has emerged as a promising paradigm for training machine learning models in distributed and privacy-sensitive environments. However, data distribution shifts across clients and the network and latency overhead from multiple client-server communication rounds are still open challenges. Most efforts in literature focus on modifying the FL process itself, for instance through alternative weight aggregation methods, client selection, or clustering strategies, while comparatively little attention has been given to the the underlying neural network architecture. In this paper, we propose LIFT (Latent Inference with Feature Transfer), a tailored FL model architecture for robust vision-based classification tasks. Our approach combines a frozen pretrained backbone for feature extraction with a generative component (e.g., a variational autoencoder) that incorporates a discriminative layer operating on a client-shared semantic representation, such as a linear classifier in the latent space. The pretrained backbone significantly limits the number of trainable parameters, reducing the communication overhead; at the same time, the generative component captures the semantic structure of the data, improving robustness to non-IID client distributions. We conduct a large-scale experimental evaluation to demonstrate the effectiveness of our propose approach across a wide range of configurations, including increasing numbers of clients (up to 100), different backbone architectures (CNN- and transformer-based), multiple aggregation strategies (FedAvg and FedProx), and three datasets with varying levels of complexity and heterogeneity. Our findings demonstrated that our approach consistently outperforms baseline FL models, achieving average accuracy improvements of 1% over fine-tuned visual transformer (the best-performing among the baseline set) models and 8% over non–fine-tuned counterparts, while reducing communication costs by up to 72×.
Institution: University of Bologna
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204452868