Work

Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steering

Year: 2026

Type: conference-paper

Abstract: Linear concept vectors effectively steer LLMs, but existing methods suffer from noisy features in diverse datasets that undermine steering robustness.We propose Sparse Autoencoder-Denoised Concept Vectors (SDCV), which selectively keep the most discriminative SAE latents while reconstructing hidden representations.Our key insight is that concept-relevant signals can be explicitly separated from dataset noise by scaling up activations of top-k latents that best differentiate positive and negative samples.Applied to linear probing and differencein-mean, SDCV consistently improves steering success rates by 4-16% across six challenging concepts, while maintaining topic relevance.

Source: Findings of the Association for Computational Linguistics: EACL

DOI: 10.18653/v1/2026.findings-eacl.40

Authors: Haiyan Zhao, Xuansheng Wu, Fan Yang, Bo Shen, Ninghao Liu, Mengnan Du

Institutions: New Jersey Institute of Technology, University of Georgia, Wake Forest University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W7140141392

FWCI:

Cites:

Cited by:

Related to: 10