Stage-aware Bayesian-guided evolutionary algorithm for resource constrained serverless MapReduce scheduling
Year: 2026
Type: article
Abstract: Serverless computing features instant elasticity and fine-grained billing. It has emerged as a prominent paradigm for big data processing such as log analysis and monitoring. However, scheduling MapReduce jobs under serverless resource constraints is challenging because the stateless nature of functions conflicts with MapReduce stage dependencies, which increases scheduling complexity. Existing research has not adequately captured these structural constraints, causing traditional evolutionary algorithms to explore the search space inefficiently. This study addresses the MapReduce scheduling problem under serverless resource constraints with the objective of minimizing the makespan. Accordingly, a stage-aware Bayesian-guided evolutionary algorithm (BGA) is proposed that, unlike traditional evolutionary algorithms, guides evolution through Bayesian learning from elite patterns. First, a mixed-integer programming model is formulated to unify temporal and precedence constraints. Second, a Bayesian probability model for assigning tasks to function instances guides evolutionary sampling using elite patterns, reducing inefficient search. Finally, dependency-aware crossover is integrated with adaptive variable neighborhood search to ensure feasibility and enhance local exploitation. Experimental results show that BGA consistently achieves the lowest makespan among compared methods, with Relative Percentage Deviation (RPD) remaining within 2.40%-3.92%.
Source: Tsinghua Science & Technology
Institution: Liaocheng University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7236353895
