Work

ELIXIR-STEERS M3.3 Use cases for benchmarking workflows identified

Year: 2025

Type: article

Abstract: In the framework of ELIXIR-STEERS, task 3.3 aims to extend the OpenEBench community-led evaluation feature. This extension will enable the technical benchmarking of workflows used for common analyses in the Life Sciences domain based on reference datasets. For that purpose, from the ELIXIR-STEERS Task 3.3 perspective, a software has been developed to obtain a set of technical metrics. Such metrics should enable the assessment of efficiency and environmental impact of software executions. This software is the Execution Metrics Collector, which gathers low-level technical metrics from Linux processes, which are then used to derive high-level indicators, including computational carbon footprint estimations. The Technical Metrics live document describes the low and high-level community-led metrics. The identification of use cases for benchmarking workflows has been carried out in collaboration with ELIXIR-STEERS WP2, several ELIXIR Communities and other relevant Life Sciences communities, and is presented in Milestone 3.3 report. From the use cases identified, we will obtain low- and high-level metrics that will enable assessing the workflow executions' carbon footprint impact. We will connect those workflows with popular managers systems like Galaxy to gather usage. Altogether will provide us with valuable information about reproducibility and the reuse of workflows by the different scientific communities in ELIXIR and, potentially, beyond.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.15363544