Year: 2026
Type: preprint
Abstract: Recent advances in Large Language Models (LLMs) have enabled agentic systems capable of solving complex tasks through multi-turn planning, tool use, verification, and memory updates. However, learning agentic systems remains difficult due to two fundamental challenges, i.e., (1) long-horizon credit assignment, where supervision is available only at the final outcome, and (2) imbalanced data distributions, where dominant data patterns bias optimization and weaken adaptation to rare but informative reasoning behaviors. In this paper, we propose Fair Multi-Level Preference Optimization (Fair-MPO or $Φ$-MPO), a new preference optimization framework for agentic learning. We first show that Multi-Level Preference Optimization provides a principled and more computationally efficient framework for long-horizon reasoning. Then, we introduce a Fair Multi-Level Objective that addresses imbalance in agentic learning. We provide a comprehensive theoretical analysis demonstrating that our approach addresses both long-horizon reasoning and data imbalance. Our experiments on agentic reasoning benchmarks demonstrate that our approach achieves State-of-the-Art (SOTA) performance.
Source: arXiv (Cornell University)
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: Collaborative Research: BIO-AI: Human-in-the-loop Artificial Intelligence System for High Throughput Automated Identification of Ground-dwelling Arthropods, SCH:Large-scale Multi-Modality Learning System to Identify Tobacco Addiction and Predictive Analytics via Social Media Platforms, CAREER: Addressing Hallucinations for Trustworthy Large-scale Multi-Modality Learning, E-RISE Rll: Arkansas Smart Transportation Research Incubator through Data Engineering and Science
OpenAlex ID: W7214745725
Cites:
Cited by:
Related to: