Primal and Dual Optimization Strategies for Entropy-Based Regression: A Monte Carlo Comparison
Year: 2026
Type: article
Abstract: Generalized cross-entropy regression can be formulated as either a constrained primal optimization problem over probability distributions or a dual optimization problem over Lagrange multipliers. Although the two formulations are theoretically equivalent under the conditions considered, their performance depends on the optimization strategy. This study compares two primal and 10 dual routines targeting the generalized maximum entropy (GME) estimator to inform practical optimizer selection. The optimization formulations use the weighted generalized cross-entropy framework, which includes generalized cross-entropy and GME as particular cases. The Portland cement benchmark and 1000 simulated datasets varying sample size, predictor conditioning, and signal-to-noise ratio assess the routines under GME. We compare elapsed time and coefficient-recovery mean squared error (MSE), alongside interface-reported termination, common feasibility checks, and a separate dual-multiplier moment-residual diagnostic. The results reveal substantial differences in computational efficiency but smaller differences in coefficient-estimation accuracy among most routines. dual.lbfgsb3c has the lowest median time, while dual.BFGS combines competitive time and coefficient recovery and was selected as GCEstim’s default. Coefficient recovery and reported termination should be considered alongside numerical residuals. The comparison reflects a practical use case in which users do not tune each optimizer separately, yet relative performance at matched numerical accuracy remains to be established.
Source: Entropy
DOI: 10.3390/e28111219
Authors: Jorge Cabral, Pedro Macedo, Vera Afreixo
Institution: University of Aveiro
Language: English
Subfield: Applied Mathematics
Field: Mathematics
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7225426404