Work

One Task Vector is not Enough: A Large-Scale Study for In-Context Learning

Year: 2026

Type: conference-paper

Abstract: In-context learning (ICL) enables Large Language Models (LLMs) to adapt to new tasks using few examples, with task vectors, defined as specific hidden state activations hypothesized to encode task information.Existing studies are limited by small-scale benchmarks, restricting comprehensive analysis.We introduce QUITEAFEW, a novel dataset of 3,096 diverse few-shot tasks, each with 30 input-output pairs derived from the Alpaca dataset.Experiments with Llama-3-8B and Qwen-3-8B on QUITEAFEW reveal: (1) task vector performance peaks at an intermediate layer, (2) effectiveness varies significantly by task type, and (3) complex tasks rely on multiple, subtaskspecific vectors rather than a single vector, suggesting distributed task knowledge representation.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2026.acl-srw.127

Authors: Pavel Tikhonov, Ivan Oseledets, Elena Tutubalina

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Ministero dello Sviluppo Economico, Russian Academy of Sciences

Award: G1061213447

OpenAlex ID: W4414855550