Reinforcement Learning Approaches for Code Generation: a Comparative Analysis
Year: 2026
Type: conference-paper
Abstract: The rapid advancement of Large Language Models (LLMs) has transformed automated code generation. However, generating functionally correct, efficient, and maintainable code remains a challenging task. Traditional supervised fine-tuning often fails to optimize multiple software quality attributes simultaneously. This study investigates two reinforcement learning (RL) approaches for fine-tuning code generation models: Pairwise Filtering Proximal Policy Optimization (PF-PPO) and Group Relative Policy Optimization (GRPO). PF-PPO employs prefer-ence learning through noisy pairwise comparisons, while GRPO utilizes group-wise relative ranking with a median baseline. Instead of using unstable LLM-judge rewards, we have employed a static code quality metric, which combines cyclomatic complex-ity and readability scores. Experiments were conducted on the DeepSeekCoder-1.3B-Instruct model using the highly recognized MBPP benchmark. Results show that GRPO achieves a 31% improvement in functional correctness $\left(R_{\text {UT }}\right)$, outperforming PF –PPO (21% improvement). Both methods show limited gains in static code quality, highlighting trade-offs between correctness and maintainability. Our work provides a comparative analysis of preference-based and group-relative RL strategies under constrained computational settings, offering insights for future RL-based code generation research.
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7202115341