Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling
Year: 2025
Type: preprint
Abstract: The effective training of Large Language Models (LLMs) for function calling faces a critical challenge: balancing exploration of complex reasoning paths with stable policy optimization. Standard methods like Supervised Fine-Tuning (SFT) fail to instill robust reasoning, and traditional Reinforcement Learning (RL) struggles with inefficient exploration. We propose \textbf{EGPO}, a new RL framework built upon Group Relative Policy Optimization (GRPO), designed to address this challenge directly. The core of EGPO is an entropy-enhanced advantage function that integrates the entropy of the model's Chain-of-Thought (CoT) into the policy gradient computation. This encourages the generation of diverse reasoning strategies. To maintain optimization direction, the entropy bonus is carefully constrained by a clipping mechanism. Complemented by a strict, binary reward signal, EGPO effectively guides the model towards discovering structured and accurate tool invocation patterns. On the challenging Berkeley Function Calling Leaderboard (BFCL), a 4B-parameter model trained with EGPO sets a new state-of-the-art among models of comparable size, surpassing a range of strong competitors, including GPT-4o and Gemini-2.5.
Source: arXiv (Cornell University)
Authors: Hao, Bingguang, Xu, Zengzhuang, Wang, Maolin, Wen, Yuntao, Chen, Yicheng, Peng, Cunyin, Chen, Long, Wang, Dong, Zhao, Xiangyu, Gu, Jinjie, Zhuang, Chenyi, Zhang, Ji
Institutions: City University of Hong Kong, Southwest Jiaotong University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4415280673