Optimization of the architecture and hyperparameters of multimodal neural networks based on evolutionary algorithms
Year: 2026
Type: article
Abstract: This paper develops and investigates AEB-MO (Adaptive Evolutionary Budgeting for Multimodal Optimization), a method for optimizing the architectures and hyperparameters of multimodal neural networks. The proposed method combines joint encoding of architectural and training parameters, adaptive adjustment of the mutation probability based on search stagnation, and dynamic allocation of the computational budget according to the uncertainty of previous evaluations and changes in the set of nondominated solutions. The optimization problem is formulated as a multi-objective task that maximizes classification performance in terms of Macro-F1 while minimizing the number of model parameters and inference time. The experimental evaluation was conducted on the FinMultiTime multimodal financial dataset, incorporating numerical financial indicators, textual news data, and Japanese candlestick chart images. According to the reported results, AEB-MO achieved a Macro-F1 score of 0.863, exceeding the result obtained by NSGA-II by 0.017. The optimized model contained 26.4 million parameters and achieved an inference time of 12.7 ms, representing reductions of 31.6% and 31.0%, respectively, compared with the initial architecture. Ablation analysis demonstrated the contributions of adaptive mutation and computational budget allocation to classification performance and the computational cost of the search process. The results highlight the potential of the proposed method for jointly optimizing predictive performance and computational efficiency in multimodal neural networks.
Source: InterConf
Authors: Valerii Rudoi, Serhii Minukhin
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Keywords: multimodal neural networks, neural architecture search, hyperparameter optimization, multi-objective evolutionary algorithms, adaptive mutation probability, computational budget allocation, model compression, inference latency reduction, financial time series classification, candlestick charts
Open Access status: hybrid
OpenAlex ID: W7234170352
