Generalizable coordination of large multiscale workflows: challenges and learnings at scale
Year: 2021
Type: conference-paper
Abstract: The advancement of machine learning techniques and the heterogeneous architectures of most current supercomputers are propelling the demand for large multiscale simulations that can automatically and autonomously couple diverse components and map them to relevant resources to solve complex problems at multiple scales. Nevertheless, despite the recent progress in workflow technologies, current capabilities are limited to coupling two scales. In the first-ever demonstration of using three scales of resolution, we present a scalable and generalizable framework that couples pairs of models using machine learning and in situ feedback. We expand upon the massively parallel Multiscale Machine-Learned Modeling Infrastructure (MuMMI), a recent, award-winning workflow, and generalize the framework beyond its original design. We discuss the challenges and learnings in executing a massive multiscale simulation campaign that utilized over 600,000 node hours on Summit and achieved more than 98% GPU occupancy for more than 83% of the time. We present innovations to enable several orders of magnitude scaling, including simultaneously coordinating 24,000 jobs, and managing several TBs of new data per day and over a billion files in total. Finally, we describe the generalizability of our framework and, with an upcoming open-source release, discuss how the presented framework may be used for new applications.
Authors: Harsh Bhatia, Francesco Di Natale, Joseph Y. Moon, Xiaohua Zhang, Joseph R. Chavez, Fikret Aydin, Chris Stanley, Tomas Oppelstrup, Chris Neale, Sara Kokkila Schumacher, Dong H. Ahn, Stephen Herbein, Timothy S. Carpenter, Sandrasegaram Gnanakaran, Peer-Timo Bremer, James N. Glosli, Felice C. Lightstone, Helgi I. Ingólfsson
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: hybrid
Funders: U.S. Department of Energy, International Business Machines Corporation, Nvidia, National Institutes of Health, Office of Science, National Nuclear Security Administration, National Cancer Institute, Advanced Scientific Computing Research, Lawrence Livermore National Laboratory, Oak Ridge National Laboratory, Los Alamos National Laboratory
Awards: G121915436, G1356433355, G14554683, G1645119126, G1716777315, G203625172, G2145270357, G2569679037, G2584197162, G286898171, G2988778103, G3005847338, G314071323, G4017386982, G4094752214, G4343739756, G4370606643, G4423657506, G5685548091, G5713387433, G5738753633, G574795485, G5840121450, G6086734019, G6139676266, G632693473, G675320460, G69704028, G69709960, G7005983164, G7186599013, G75892134, G8269158468, G8906985441, G928841278, G991281326
OpenAlex ID: W3210258158