Year: 2026
Type: article
Abstract: The Python package mlrl-testbed provides a command line utility designed to support researchers in conducting reproducible machine learning experiments.It offers a straightforward, easily configurable, and extensible workflow that supports the full experimental lifecycle:• Loading a dataset.• Splitting it into training and test sets.• Training one or more models.• Evaluating the models' predictive performance.• Saving experimental results to output files.By default, mlrl-testbed executes a single experiment using a given dataset and parameter setting.However, it can also be operated in the following modes:• Batch mode: Allows running multiple independent experiments with varying datasets and parameter settings.Installing the optional package mlrl-testbed-slurm enables experiments to be run via the Slurm Workload Manager 1 .• Read mode: Allows inspecting the results of previous experiments and saving them to new output files.When viewing results obtained in batch mode, results are automatically aggregated across different experiments.• Run mode: Allows re-running previously conducted experiments with the option to partly override their configuration.Experiments for which results are already available can be skipped.Originally developed to support work on the BOOMER algorithm (Rapp et al., 2021;Rapp, 2021), mlrl-testbed has since evolved into a standalone utility for empirical machine learning studies.
DOI: 10.21105/joss.10609
Author: Michael Rapp
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7225997631
