Work

A Center-Guided Reinforcement Learning Method for Hyperparameter Optimization and Its Application to Relation Extraction

Year: 2026

Type: article

Abstract: Hyperparameter optimization (HPO) aims to identify high-quality model configurations under a limited evaluation budget. To address mixed search spaces, sparse feedback, and low sample efficiency in reinforcement learning (RL)-based HPO, a Center-Guided Reinforcement Learning (CGRL) method is proposed. In CGRL, the policy output is reformulated from a configuration to be directly evaluated into a search center that defines a promising region, decoupling region-level guidance from exact configuration selection. A mixed candidate pool is generated around the center, and a promising candidate for real evaluation is selected by a Random Forest surrogate model. Meanwhile, a process-aware reward provides dense and informative feedback for policy learning. Experiments on 20 Yet Another Hyperparameter Optimization (YAHPO) Gym environments validate the effectiveness of CGRL. Compared with random search (RS), Tree-structured Parzen Estimator (TPE), Sequential Model-based Algorithm Configuration 3 (SMAC3), a Proximal Policy Optimization baseline (PPO-basic), Hyperparameter Optimization by Reinforcement Learning (Hyp-RL), and Q-Learning for Hyperparameter Optimization (HyperQ-Opt), CGRL achieves the best average rank of 1.800 in terms of the final best objective value, versus 6.000, 3.600, 2.200, 4.450, 6.350, and 3.600, respectively. For Low-Rank Adaptation (LoRA) HPO for relation extraction (RE) from ancient Chinese historical documents, CGRL improves Macro-F1 by 8.66%, 3.10%, 3.18%, and 5.13% on the validation set relative to RS, TPE, SMAC3, and PPO, respectively, and by 11.15%, 2.03%, 3.11%, and 9.59% on the test set. These results demonstrate the effectiveness of CGRL for limited-budget HPO and its applicability to practical RE tasks.

Source: Machine Learning and Knowledge Extraction

DOI: 10.3390/make8090264

Authors: Yangbin Tan, Liping Mo, Yu Yan

Institution: Jishou University

Language: English

Topic: Machine Learning and Data Classification

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,172

Funders: National Natural Science Foundation of China, Natural Science Foundation of Hunan Province