Causal-Graph-Guided Reinforcement Learning for LLM Security Testing: An Empirical Study
Year: 2026
Type: preprint
Abstract: This preprint investigates whether structural causal information derived from adversarial interaction trajectories can improve reinforcement-learning-based black-box security testing of large language models. The study combines PPO-based adaptive prompt search with an interpretable six-factor representation derived from red-teaming trajectories. Structural relationships among these factors are estimated using Fast Causal Inference (FCI) and represented through a Partial Ancestral Graph (PAG). A Structural Causal Model (SCM) is subsequently used to provide model-based structural guidance during reinforcement learning through intermediate feedback and causal-graph-guided action selection. Under the reported experimental configuration, the causally guided approach achieved a mean attack success rate of 61.54%, compared with 19.33% for the archived RL-based baseline, while reducing API and token usage. Because the primary evaluations were not fully prompt-aligned, this headline comparison is treated as descriptive rather than as a paired statistical comparison. The work contributes an empirical investigation of how causal structure can be incorporated into adaptive LLM security testing while maintaining an interpretable representation of adversarial prompt characteristics.
Author: Zeinaliashtiyani, Parisa
Institution: University of Naples Federico II
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7214429707
Cites:
Cited by:
Related to: