Year: 2026
Type: conference-paper
Abstract: Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-judge pipelines, but strong judges can still be costly to deploy at scale.We present a multi-agent judge that uses small language models (SLMs) in structured debates among critic, defender, and judge agents.To evaluate these systems, we introduce ForesightSafety Human-Annotated JailBench (FS-HAJailBench), a human annotated jailbreak benchmark with 11,100 labeled interactions spanning diverse attack methods and target models.FS-HAJailBench provides fine-grained ground truth for measuring both target-model robustness and judge reliability.On this benchmark, our SLM-based framework improves over matched small-model baselines and prior multi-agent judges, while approaching GPT-4o-level agreement with lower estimated unit cost.Ablation results show that three debate rounds provide the best accuracy-efficiency trade-off among the current Qwen3-14B variants.These results indicate that structured debate is a practical way to improve scalable LLM safety evaluation.The source code is available at https://github.com/Beijing-AISI/ ForesightSafety-HAJailBench.
Authors: Dachuan Lin, Guobin Shen, Zihao Yang, Tianrong Liu, Dongcheng Zhao, Yi Zeng
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7104869295