Get started
Work

A Unified Method for Patch Pruning in Vision Transformers to Achieve Efficient Model Deployment

Year: 2025
Type: preprint
Abstract: Vision Transformers (ViTs) have demonstrated remarkable performance in computer vision tasks, but their computational cost remains a significant challenge due to the quadratic complexity of selfattention mechanisms. Patch pruning has emerged as an effective strategy to reduce this computational burden by selectively removing image patches, thereby decreasing the number of input tokens and improving efficiency. In this survey, we provide a comprehensive review of patch pruning methods for ViTs, exploring various pruning strategies, including heuristic-based, attention-guided, reinforcement learning, and adaptive tokenization approaches. We discuss how these methods can be integrated with different ViT architectures, such as standard ViTs, Swin Transformers, and data-efficient models like DeiT, to achieve a balance between computational efficiency and model accuracy. Experimental results from benchmark datasets, including ImageNet, CIFAR-100, and COCO, are presented, demonstrating that patch pruning can significantly reduce FLOPs and improve inference speed with minimal loss in accuracy. We also highlight the challenges and open problems in the field, such as the trade-off between pruning ratio and performance, as well as the real-world hardware considerations for efficient deployment. Finally, we outline promising future research directions, including self-supervised pruning, multi-task learning, and hybrid approaches combining pruning with other compression techniques. This survey aims to provide valuable insights and guide future advancements in patch pruning for Vision Transformers, contributing to more efficient and scalable deep learning models for computer vision.
Source: HAL
Language: English
Open Access status: green
OpenAlex ID: W4415248466
Cites:
Cited by:
Related to: 10