ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
Year: 2026
Type: conference-paper
Abstract: While LLM-based agents can interact with environments via invoking external tools, their expanded capabilities also amplify security risks.Monitoring step-level tool invocation behaviors in real time and proactively intervening before unsafe execution is critical for agent deployment, yet remains underexplored.In this work, we first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents.We then develop a guardrail model, TS-Guard, using multitask reinforcement learning.The model proactively detects unsafe tool invocation actions before execution by reasoning over the interaction history.It assesses request harmfulness and action-attack correlations, producing interpretable and generalizable safety judgments and feedback.Furthermore, We introduce TS-Flow, a guardrail-feedback-driven reasoning framework for LLM agents, which reduces harmful tool invocations of ReAct-style agents by 65% on average and improves benign task completion by approximately 10% under prompt injection attacks.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166886473