Work

Forgetting of task-specific knowledge in model merging-based continual learning

Year: 2025

Type: preprint

Abstract: This paper investigates the linear merging of models in the context of continual learning (CL). Using controlled visual cues in computer vision experiments, we demonstrate that merging largely preserves or enhances shared knowledge, while unshared task-specific knowledge rapidly degrades. We further find that merging models from an incremental training process consistently outperforms merging models trained in parallel.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2507.23311

Authors: Hess, Timm, van de Ven, Gido M, Tuytelaars, Tinne

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: European Commission, Fonds Wetenschappelijk Onderzoek, Vlaamse regering

Award: Incrementally learning new classes with generative classification

OpenAlex ID: W4414920255

Cites:

Cited by:

Related to: