Work

Communication-Computation Co-Optimized Federated Learning for Efficient Large-Model Embedding Training

Year: 2025

Type: article

Abstract: With the rapid development of the Industrial Internet of Things (IIoT) and intelligent manufacturing, massive amounts of heterogeneous and non-independent, identically distributed (non-IID) data are continuously generated in industrial environments. Large models have demonstrated strong generalization and transfer capabilities, offering new possibilities for predictive maintenance, anomaly detection, and intelligent decision-making in IIoT scenarios. However, the deployment of such models in industrial environments faces challenges due to resource constraints in communication and computation. To address this problem, this paper proposes a collaborative optimization framework that integrates client-side feature learning, a hierarchical client–edge–cloud federated aggregation, and network-computing resource scheduling for efficient large-model embedding training. A parameter search method based on the Kepler Optimization Algorithm (PSKOA) is introduced to jointly optimize the three interdependent dimensions: client-side model structure parameter, federated aggregation parameters, and scheduling strategy. Evaluations demonstrate that the proposed method significantly reduces model loss by 41.7% and shortens training time by 13.4% compared to the traditional Genetic Algorithm-based method. Additionally, the proposed method achieves 12.5% lower model loss and 3.1% faster training time compared to the Particle Swarm Optimization-based method. These results highlight that the proposed method effectively enhances both training efficiency and convergence performance by jointly optimizing communication, computation, and model structure, making it a practical and scalable solution for large-model embedding training in resource-constrained IIoT environments.

Source: Mathematics

DOI: 10.3390/math13233871

Authors: Yingying Luo, Xi Jin, Changqing Xia, Chi Xu, Yiming Sun

Institutions: Shenyang Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Northeastern University

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: gold

Estimated APC paid: $3,137