Year: 2026
Type: conference-paper
Abstract: Steering large language model (LLM) reasoning via high-level reasoning actions offers a promising approach to improve robustness and interpretability.However, existing action-based paradigms, ranging from training-free prompting to static plan retrieval or prediction, often fail to consistently outperform standard generation because their planners tend to degenerate into repetitive loops or fixed patterns.We propose PG-HAP (Policy-Guided High-Level Action Planning), a lightweight stepwise plannerexecutor framework that learns to select reasoning actions dynamically while keeping the executor LLM fully frozen.The planner is trained with reinforcement learning to optimize answer correctness.To prevent degeneration, we introduce two targeted mechanisms: (i) an Action-Dependency Logit Mask that enforces valid transitions to avoid redundancy, and (ii) an Action Diversity Reward that discourages mode collapse by promoting varied action sequences.Across mathematical and commonsense reasoning benchmarks, PG-HAP improves accuracy over strong baselines while producing less redundant, more adaptive trajectories.This demonstrates that learning high-level planning alone can substantially strengthen reasoning without expensive end-to-end model tuning. 1
Authors: Jianpeng Zhou, Qisheng Hu, Jiahai Wang, Wenya Wang
Institutions: Sun Yat-sen University, Nanyang Technological University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G1368166787, G8800154292
OpenAlex ID: W7166833587