EdgeFormer: Latency-Aware Collaborative Multi-Head Attention of Transformer Inference in Edge Networks
Year: 2026
Type: conference-paper
Abstract: Recent breakthroughs in Transformer-based large models, have driven widespread tasks, yet their reliance on centralized cloud deployment raises significant privacy risks due to sensitive data exposure.While edgebased collaborative inference offers a privacypreserving alternative, existing methods face critical limitations: static model partitioning cannot adapt to dynamic edge resource fluctuations, and rigid multi-head attention handling overlooks semantic-critical prioritization and parallelism.We propose EdgeFormer, a latency-aware framework for distributed Transformer inference in resource-constrained edge networks.EdgeFormer dynamically allocates model blocks across devices via efficiencystorage trade-off optimization and introduces collaborative Multi-Head Attention (cMHA), which distributes semantic-critical attention heads across devices while pruning redundant ones under real-time constraints.We further develop LiScore, a composite metric integrating attention diversity and latency costs, alongside a similarity-based retrieval method to reduce recomputation overhead.Extensive experiments demonstrate that EdgeFormer achieves up to 2.01× inference acceleration over state-of-theart baselines with ≤1.06% accuracy loss, maintaining robustness under varying edge conditions.
Authors: Yiming Yao, Jianwei Niu, Bin Dai, Tao Ren
Language: English
Topic: IoT and Edge/Fog Computing
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166798822