Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs’ Instruction Following Capability
Year: 2026
Type: conference-paper
Abstract: The capability to precisely adhere to instructions is a cornerstone for Large Language Models (LLMs) to function as dependable agents in real-world scenarios.However, confronted with complex prompts, LLMs frequently encounter difficulties in fulfilling all specified requirements within a single response.Drawing inspiration from recent advancements in Chain-of-Thought (CoT) prompting and self-correction methodologies, we introduce Meeseeks 1 , a fully automated iterative instruction-following benchmark equipped with an integrated feedback mechanism.Meeseeks identifies erroneous components in model responses and provides corresponding feedback accurately, thereby iteratively guiding the model toward selfcorrection.The dataset contains over 700 curated instances annotated by 32 distinct capability tags in Chinese and English.Extensive experimental results reveal that different state-of-the-art commercial and opensource LLMs exhibit vastly disparate performance, and even after 20 turns of iterative feedback-driven self-correction, nearly all models demonstrate suboptimal performance.We conducted comprehensive analysis and uncovered numerous common issues prevalent in current state-of-the-art models, as well as several counterintuitive phenomena.Meeseeks has been open-sourced on https://github.com/ ADoublLEN/Meeseeks.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166838090