Year: 2026
Type: article
Abstract: Accurate split selection in regression trees and random forests is critical for predictive performance. In regression trees, splits are selected by minimizing within-node variances weighted by node proportions. These weights directly influence the impurity reduction criterion. We modify the standard weighting scheme by applying a power transformations to node proportions and examine how this modification influences split selection and predictive accuracy. Based on the proposed weighting framework, we develop four algorithmic variants of random forest, referred to as FlexForest, that introduce flexibility at the node, tree, and forest levels, along with a randomized weight-selection strategy. The proposed framework is evaluated via simulations across a range of data-generating mechanisms, including varying sample sizes, noise levels, functional relationships, and covariate dependence structures and distributions. Performance is assessed using relative prediction error against the default weighting scheme, and the results demonstrate considerable improvements, with the tree-level flexible method achieving improved or comparable predictive performance relative to the standard random forest in nearly all scenarios.
Source: Mspace (University of Manitoba)
Author: Mendis, Heshani
Institution: University of Manitoba
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221217349