Value Locking: Preserving Safety Alignment via Orthogonal Gradient Projection During Post-Deployment Fine-Tuning
Year: 2026
Type: preprint
Abstract: The rapid deployment of open-weight Large Language Models (LLMs) has introduced a critical vulnerability: when edge users fine-tune these models on local data, the deterministic safety guardrails (RLHF/DPO alignment) embedded during pre-training are rapidly degraded or entirely overwritten. Value Locking is proposed as a formal mathematical framework to preserve safety alignment during post-deployment continual learning. By extracting the geometric "safety Gestalt"—the principal activation subspaces corresponding to aligned rejection behaviors—it is demonstrated how downstream parameter updates can be continuously projected onto the orthogonal complement of this safety manifold. Using Orthogonal Gradient Projection (OGP), fine-tuning gradients that attempt to overwrite established safety boundaries are strictly nullified. This provides a deterministic structural constraint that prevents gradient updates from modifying the identified safety subspace, formally guaranteeing retention for activations within the estimated span(U_safety), without requiring the fine-tuner to store or process the original safety dataset.
Author: Lemos, Jero
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204196512
Cites:
Cited by:
Related to: