Towards A Modular End-To-End Machine Learning Benchmarking Framework
Year: 2025
Type: conference-paper
Abstract: Machine learning (ML) benchmarks are crucial for evaluating the performance, efficiency, and scalability of ML systems, especially as the adoption of complex ML pipelines, such as retrieval-augmented generation (RAG), continues to grow. These pipelines introduce intricate execution graphs that require more advanced benchmarking approaches. Additionally, collocating workloads can improve resource efficiency but may introduce contention challenges that must be carefully managed. Detailed insights into resource utilization are necessary for effective collocation and optimized edge deployments. However, existing benchmarking frameworks often fail to capture these critical aspects.
Authors: Robert Bayer, Ties Robroek, Pinar Tözün
Institution: IT University of Copenhagen
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Funder: Universitas Brawijaya
Award: G8244615741
OpenAlex ID: W4408894417