Work

FCL-ViT: Task-aware attention tuning for Continual Learning

Year: 2025

Type: article

Abstract: Continual Learning (CL) involves adapting the prior Deep Neural Network (DNN) knowledge to new tasks, without forgetting the old ones. However, modern CL techniques focus on provisioning memory capabilities to existing DNN models rather than designing new ones that are able to adapt according to the task at hand. This paper presents the novel Feedback Continual Learning Vision Transformer (FCL-ViT) that uses a feedback mechanism to generate real-time dynamic attention features tailored to the current task. The FCL-ViT operates in two Phases. In phase 1, the generic image features are produced and determine where the Transformer should attend on the current image. In phase 2, task-specific image features are generated that leverage dynamic attention. To this end, Tunable self-Attention Blocks (TABs) and Task Specific Blocks (TSBs) are introduced that operate in both phases and are responsible for tuning the TABs attention, respectively. The FCL-ViT surpasses state-of-the-art performance on Continual Learning compared to benchmark methods, while retaining a small number of trainable DNN parameters. • A novel CL framework using TABs and TSBs as key building blocks. • Operates in two phases: generic image features and task-specific features. • Task-specific features generated by dynamically reprogramming TABs. • FCL-ViT outperforms state-of-the-art on CL image classification benchmarks.

Source: Pattern Recognition Letters

DOI: 10.1016/j.patrec.2025.08.003

Authors: Anestis Kaimakamidis, Ioannis Pitas

Institution: Aristotle University of Thessaloniki

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $2,500

Funders: European Commission, Toyota Motor Engineering and Manufacturing North America, HORIZON EUROPE Framework Programme