Abstract
Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate a per-layer steering direction from contrastive data and apply it on the layer's entire representation space, which may couple the intervention to off-target properties present in the contrastive data and degrade unrelated capabilities. To mitigate this issue, we introduce LocUS (Localized Unembedding Steering), a method which grounds activation steering to the model's own output vocabulary subspace. By identifying a property-specific linear subspace within the unembedding matrix, LocUS enforces a geometric constraint that restricts the steering transformation to a specific subspace and at the same time localizes its application to a sparse subset of attention heads. Extensive evaluations across three model families on toxicity mitigation, sentiment redirection and sycophancy suppression show that LocUS matches or outperforms state-of-the-art baselines while intervening on under 6% of parameters and better preserving general capability.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Tallini, I., Basile, L., Maiorca, V., Locatello, F., & Cazzaniga, A. (2026). LocUS: Head Selection and Subspace Projection for Targeted Activation Steering. https://omanscience.com/en/articles/locus-head-selection-and-subspace-projection-for-targeted-activation-steering
MLA 9
Tallini, Irene, et al. "LocUS: Head Selection and Subspace Projection for Targeted Activation Steering." https://omanscience.com/en/articles/locus-head-selection-and-subspace-projection-for-targeted-activation-steering.
Chicago (author–date)
Tallini, Irene, Lorenzo Basile, Valentino Maiorca, Francesco Locatello, and Alberto Cazzaniga. 2026. "LocUS: Head Selection and Subspace Projection for Targeted Activation Steering." https://omanscience.com/en/articles/locus-head-selection-and-subspace-projection-for-targeted-activation-steering.
Harvard
Tallini, I., Basile, L., Maiorca, V., Locatello, F. and Cazzaniga, A. (2026) 'LocUS: Head Selection and Subspace Projection for Targeted Activation Steering', Available at: https://omanscience.com/en/articles/locus-head-selection-and-subspace-projection-for-targeted-activation-steering.
Vancouver
Tallini I, Basile L, Maiorca V, Locatello F, Cazzaniga A. LocUS: Head Selection and Subspace Projection for Targeted Activation Steering. https://omanscience.com/en/articles/locus-head-selection-and-subspace-projection-for-targeted-activation-steering
IEEE
I. Tallini, L. Basile, V. Maiorca, F. Locatello, and A. Cazzaniga, "LocUS: Head Selection and Subspace Projection for Targeted Activation Steering," https://omanscience.com/en/articles/locus-head-selection-and-subspace-projection-for-targeted-activation-steering.