Democratizing Scientific Machine Learning through Expert-Encoded Workflow Templates
Year: 2025
Type: conference-paper
Abstract: The increasing complexity of AI/ML software stacks and diverse accelerator hardware on High-Performance Computing (HPC) systems creates a significant productivity barrier for domain scientists. Configuring optimized, reproducible AI workflows often requires specialized systems expertise that researchers may not possess. To tackle this, we introduce Drona, a workflow engine that encapsulates expert knowledge into reusable, domain-specific templates. Drona guides researchers through complex setups via a simple web interface, dynamically generating optimized and transparent workflows. We demonstrate Drona's power and flexibility through an in-depth analysis of a Hugging Face template that automates the model inference and fine-tuning lifecycle, from proactive resource validation and distributed training configuration to seamless reproducibility. Drona lowers the barrier to entry for advanced AI/ML on HPC, improves scientific reproducibility, and empowers a wider community of researchers to leverage accelerated computing.
Authors: Kryvenko, Andrii, Pennings, Marinus, Liu, Honggao
Institution: Texas A&M University
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7095039943
FWCI:
Cites:
Cited by:
Related to: