A Reproducible Framework for Benchmarking Machine Learning Operations (MLOps) Infrastructures: Comparing Bare-Metal and Orchestrated Machine Learning Workflows
Year: 2025
Type: article
Abstract: As machine learning (ML) systems grow in complexity, supporting infrastructure must balance computational performance with scalability, reproducibility, and operational manageability. This technical report presents a reusable, Infrastructure-as-Code benchmarking artefact designed to systematically evaluate ML workflows across bare-metal and Kubernetes-based environments. Leveraging Terraform, Proxmox VE, and Kubeflow Pipelines, the artefact enables consistent, automated deployment and telemetry collection for arbitrary ML workloads under controlled hardware conditions. To demonstrate its applicability, we apply the artefact to a representative task: training a ResNet model on the CIFAR-10 dataset across both environments. The experiments validate the artefact’s ability to measure performance, resource utilization, and orchestration overheads transparently. Rather than establishing the superiority of one infrastructure over another, this work contributes a practical, extensible tool for researchers and practitioners seeking reproducible, infrastructure-aware benchmarking of ML workflows.
Authors: Julian Kramer, Tianxiang Lu
Institution: IU International University of Applied Sciences
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid:
OpenAlex ID: W4416985249