Beyond Task-Level Context: Class-Conditional Context Vectors for Implicit In-Context Learning
Year: 2026
Type: conference-paper
Abstract: Implicit In-Context Learning compresses demonstration examples into a single context vector and injects it into the model's activation space, achieving few-shot-level performance at near zero-shot inference cost.However, existing approaches typically aggregate demonstrations from all classes into a shared, tasklevel context vector, capturing global task information but without explicitly preserving finegrained, class-conditional semantic distinctions.In this work, we propose Class-Conditional Context Vectors (C 3 V), a simple yet effective extension to implicit in-context learning that explicitly models class-specific contextual information by constructing separate context vectors for each class, without relying on explicit prompts.These class-conditional context vectors are additively injected into the model's residual streams in a single forward pass, enabling contextual contributions to be modulated in a class-aware manner while keeping the backbone frozen.We evaluate C 3 V on multiple text classification benchmarks across several families of large language models.Experimental results demonstrate that C 3 V consistently outperforms task-level context vector baselines, and achieves higher average accuracy than conventional few-shot learning on most models.
Authors: Jianxin Zhang, Yilu Hu, Teng Liu, Pei Guo, Juntao Li
Institution: Soochow University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166817644