Work

Behavior of Hyper-Parameters for Selected Machine Learning Algorithms: An Empirical Investigation

Year: 2022

Type: preprint

Abstract: Hyper-parameters (HPs) are an important part of machine learning (ML) model development and can greatly influence performance. This paper studies their behavior for three algorithms: Extreme Gradient Boosting (XGB), Random Forest (RF), and Feedforward Neural Network (FFNN) with structured data. Our empirical investigation examines the qualitative behavior of model performance as the HPs vary, quantifies the importance of each HP for different ML algorithms, and stability of the performance near the optimal region. Based on the findings, we propose a set of guidelines for efficient HP tuning by reducing the search space.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2211.08536

Authors: Bhattacharyya, Anwesha, Vaughan, Joel, Nair, Vijayan N.

Language: English

Topic: Machine Learning and Data Classification

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4309301085

Cites:

Cited by:

Related to: 10