Work

Parallel I/O analysis in distributed deep learning applications on high-performance computing

Year: 2025

Type: article

Abstract: Distributed deep learning (DDL) applications generate heavy input/output (I/O) workloads that can create bottlenecks in high-performance computing (HPC) systems. Their optimal I/O configuration depends on factors such as access patterns, storage hardware, dataset size, and execution scale. This study proposes a systematic methodology for characterizing and optimizing I/O behavior in DDL applications, represented through the deep learning I/O benchmark (DLIO), and validated with the real DeepGalaxy application. We evaluate access modes, file formats, and Lustre file system configurations, demonstrating that stripe counts optimized for the access pattern and application scale can reduce I/O and execution times, achieving up to 18 GiB/s of bandwidth and a 5X increase in IOPS. HDF5 provides balanced performance, while TFRecord stands out in bandwidth-intensive scenarios. Shared access minimizes contention and improves scalability in multi-node executions. The results are consolidated into configuration guidelines that offer practical recommendations for practitioners to tune DDL applications for efficient execution in HPC environments.

Source: The Journal of Supercomputing

DOI: 10.1007/s11227-025-07986-1

Authors: Edixon Parraga, Betzabeth Leon, Sandra Mendez, Dolores Rexachs, Emilio Luque

Institutions: Universitat Autònoma de Barcelona, Barcelona Supercomputing Center

Language: English

Topic: Advanced Data Storage Technologies

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,290

Funders: Universitat Autònoma de Barcelona, European Regional Development Fund, Agencia Estatal de Investigación