LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation
Year: 2025
Type: conference-paper
Abstract: The LLMSR@XLLM25 formulates a lowresource structural reasoning task that challenges LLMs to generate interpretable, step-bystep rationales with minimal labeled data.We present Less is More, the third-place winning approach in the LLMSR@XLLM25, which focuses on structured reasoning from only 24 labeled examples.Our approach leverages a multi-agent framework with reverseprompt induction, retrieval-augmented reasoning synthesis via GPT-4o, and dual-stage reward-guided filtering to distill high-quality supervision across three subtasks: question parsing, CoT parsing, and step-level verification.All modules are fine-tuned from Meta-Llama-3-8B-Instruct under a unified LoRA+ setup.By combining structure validation with reward filtering across few-shot and zero-shot prompts, our pipeline consistently improves structure reasoning quality.These results underscore the value of controllable data distillation in enhancing structured inference under low-resource constraints.Our code is available at https://github.com/JhCircle/ Less-is-More.
Authors: Jiahao Yuan, Xingzhe Sun, Xing Yu, Jingwen Wang, Dehui Du, Zhiqing Cui, Zixiang Di
Institutions: East China Normal University, University of Reading
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4412944148