Work

Performance of a high-order MPI-Kokkos accelerated fluid solver

Year: 2025

Type: article

Abstract: This work discusses the performance of a modern numerical scheme for fluid dynamical problems on modern high-performance computing (HPC) architectures. Our code implements a spatial nodal discontinuous Galerkin (NDG) scheme that we test up to an order of convergence of eight. It is temporally coupled to a set of Runge-Kutta (RK) methods of orders up to six. In order to target modern hardware involving many-core Central Processing Units (CPUs) and accelerators such as Graphic Processing Units (GPUs) we use the Kokkos library in conjunction with the Message Passing Interface (MPI) to run our single source code on various NVidia and AMD GPU systems. By means of one- and two-dimensional simulations of simple test equations we find that the higher the order the faster is the code. Eighth-order simulations attain a given global error with much less computing time than third- or fourth-order simulations. The RK scheme has a smaller impact on the code performance and a classical fourth-order scheme seems to generally be a good choice. The code performs very well on all considered HPC GPUs. We observe very good scaling properties up to 64 AMD MI250x GPUs. The many-CPU performance is also very good and perfect weak scaling is observed up to many hundreds of CPU cores using MPI. We note that small grid-size simulations are faster on CPUs than on GPUs while GPUs win significantly over CPUs for simulations involving more than $10^7$ degrees of freedom ($\approx 3100^2$ grid points). When it comes to the environmental impact of numerical simulations we estimate that GPUs consume less energy than CPUs for large grid-size simulations but more energy on small grids. Further, we observe a tendency that the more modern is the GPU the larger needs to be the grid in order to use it efficiently. This yields a rebound effect because larger simulations need longer computing times and in turn more energy that is not compensated by the energy efficiency gain of the newer GPUs.

Source: Computer Physics Communications

DOI: 10.1016/j.cpc.2025.109873

Authors: Filipp Sporykhin, Holger Homann

Institutions: Centre National de la Recherche Scientifique, Université Côte d'Azur, Lagrange Laboratory, Observatoire de la Côte d’Azur

Language: English

Topic: Parallel Computing and Optimization Techniques

Subfield: Hardware and Architecture

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $4,160

Funders: Centre Informatique National de l’Enseignement Supérieur, Agence Nationale de la Recherche, Grand Équipement National De Calcul Intensif, Université Côte d’Azur