Revisiting HDF5 SWMR and file versioning features
Year: 2025
Type: article
Abstract: In this talk, we will present an overview and demonstration of two HDF5 features implemented via virtual file drivers (VFDs), both currently in a prototype stage: the full single-writer/multiple-reader capability (VFD SWMR) and HDF5 versioning (also known as the Onion VFD). The VFD SWMR feature enables file modifications during writing and provides guarantees on the maximum latency before new data becomes available to readers. The Onion VFD allows tracking of modifications to an HDF5 file between open/close calls while preserving—or providing access to—the file's state prior to a given set of changes. Although the Onion VFD has been released and is available in current maintenance versions, a more robust implementation is still in development. In this presentation, we aim to gather feedback from the HDF5 user community on both VFDs and to share our roadmap for developing production-ready versions of these features.
Authors: Pourmal, Elena, Breitenfeld, M. Scot
Institution: The HDF Group
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6931132986
FWCI:
Cites:
Cited by:
Related to: