Year: 2025
Type: article
Abstract: Bayesian optimization (BO) is a memory-intensive algorithm that requires training and evaluating an expensive objective function. In contrast to previous works that use an offline memory estimation to make BO memory-efficient, we propose a robust and simple online memory estimation method that requires training a model only for the first two iterations of the first epoch. Our memory estimation method is then integrated with a simple, performance-based surrogate model of BO in a seamless (or in sync) mode that enforces memory efficiency even if it does not bypass a preset threshold. The online memory estimation method has been evaluated on two different datasets, showing that it is more accurate than the existing offline method ( $$2.19\times $$ 2.19 × for MNIST and $$3.51\times $$ 3.51 × for CIFAR datasets). Furthermore, compared to a memory-unaware baseline, the enhanced BO has no loss of accuracy and is $$11.31\times $$ 11.31 × memory-efficient for a simple CNN-based image classification, and $$5.03\times $$ 5.03 × memory efficient but $$9.27\times $$ 9.27 × slower for a more complex LSTM-based text classification (useful for a resource-constrained environment where delay is tolerable but memory is scarce), while it is $$2.6\times $$ 2.6 × memory efficient but $$1.23\times $$ 1.23 × slower on a pretrained ResNet50 model.
Source: Complex & Intelligent Systems
Authors: Befekadu Bekuretsion, Wolfgang Menzel, Solomon Teferra
Institutions: Addis Ababa University, Universität Hamburg
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,490
OpenAlex ID: W4416677628