Work

CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference

Year: 2025

Type: conference-paper

Abstract: In response to the growing interest in large multimodal models, we introduce Cross-Attention Token Pruning (CATP), an accuracy-preserved token pruning method. Our approach leverages cross-attention layers in multimodal models, exemplified by BLIP-2, to extract valuable information to determine token importance. CATP employs a refined voting strategy across model heads and layers. In evaluations, CATP achieves up to 12.1 X higher accuracy than existing token pruning methods, addressing the trade-off between computational efficiency and model precision.

Source: IEEE Conference on Artificial Intelligence (CAI)

DOI: 10.1109/cai64502.2025.00191

Authors: Ruqi Liao, Chuqing Zhao, Jin Li, Weiqi Feng, Yi Lyu, Bingxian Chen, Haochen Yang

Institutions: Harvard University, University of Wisconsin–Madison, Stanford University

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4412082472

FWCI: 2.344

Cites: 7

Cited by: 3

Related to: 10