Building an efficient Continuous Integration workflow on HPC systems
Year: 2024
Type: article
Abstract: Continuous Integration (CI) is a central cornerstone of modern software development practices, but its usage in High Performance Computing (HPC) contexts brings a set of challenges that are yet to be tackled fully across different libraries and computing centers. We want to share our own path in the development of a modern HPC library, moving from a single development node to multiple nodes and onto SLURM-managed HPC systems. We will cover the following topic areas: Container image generation and deployment Rootless container execution and custom executors Interaction with SLURM through a custom executor or ephemeral runners Accounting and security model Build performance profiling and optimization Test performance and split build/test execution What do we need from computing center operators? A recording of this session is available on YouTube: https://youtu.be/xchS7wef0L0
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6949554615
FWCI:
Cites:
Cited by:
Related to: