Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
Year: 2025
Type: conference-paper
Abstract: Large Language Models (LLMs) are vulnerable to jailbreak attacks that exploit weaknesses in traditional safety alignment, which often relies on rigid refusal heuristics or representation engineering to block harmful outputs.While they are effective for direct adversarial attacks, they fall short of broader safety challenges requiring nuanced, context-aware decision-making.To address this, we propose ReAsoning-enhanced Fine-TunIng fOr iNterpretAble LLM Safety (RATIONAL), a novel framework that trains models to engage in explicit safe reasoning before response.Fine-tuned models leverage the extensive pretraining knowledge in selfgenerated reasoning to bootstrap their own safety through structured reasoning, internalizing context-sensitive decision-making.Our findings suggest that safety extends beyond refusal, requiring context awareness for more robust, interpretable, and adaptive responses.Reasoning is not only a core capability of LLMs but also a fundamental mechanism for LLM safety.RATIONAL employs reasoningenhanced fine-tuning, allowing it to reject harmful prompts while providing meaningful and context-aware responses in complex scenarios.
Institution: Carnegie Mellon University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Funder: DeepMind
OpenAlex ID: W4412887948