Comparison of Vision Transformer and ResNet under Federated Learning in Human Action Recognition
Year: 2025
Type: article
Abstract: This study investigates the adaptability of ResNet50 and Vision Transformer (ViT) in Federated Learning (FL) versus centralized training for human action recognition, addressing data privacy and distributed data silos in real-world applications. Employing the Kaggle dataset (about 12,000 images across 15 actions), the thesis standardized preprocessing (bilinear resizing, flip/rotation augmentation, ImageNet normalization) and conducted controlled experiments. Centralized training utilized full-batch gradient descent, while FL simulated non-independent and identically distributed (non-IID) data partitioning with Federated Averaging (FedAvg) aggregation. The results indicate that ViTs achieved 87.30% accuracy in FL, surpassing Residual Network (ResNet) by 8.01%. The train-validation gap was significantly reduced (1.67% vs. 11.35%), revealing improved generalization. Overfitting was limited through phased aggregation, which also enabled late-stage convergence exceeding. ViT’s global attention proved more adaptable to heterogeneous data distributions, and FL’s distributed optimization enhanced model resilience. These results highlight the complementary strengths of ViT and FL, especially for privacy-sensitive applications like healthcare, where preserving data sovereignty is essential.
Author: Yicheng Li
Institution: Shanghai Lixin University of Accounting and Finance
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4412184050