AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Year: 2025
Type: preprint
Abstract: Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems.
Source: arXiv (Cornell University)
Authors: He, Yun, Li, Wenzhe, Zhang, Hejia, Li, Songlin, Mandyam, Karishma, Khosla, Sopan, Xiong, Yuanhao, Wang, Nanshu, Peng, Xiaoliang, Li, Beibin, Bi, Shengjie, Patil, Shishir G., Qi, Qi, Feng, Shengyu, Katz-Samuels, Julian, Pang, Richard Yuanzhe, Gonugondla, Sujan, Lang, Hunter, Yu, Yue, Qian, Yundi, Fazel-Zarandi, Maryam, Yu, Licheng, Benhalloum, Amine, Awadalla, Hany, Faruqui, Manaal
Institutions: Princeton University, Carnegie Mellon University
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4416246346