Comparative analysis of workflow management systems in production genomics research and the clinic
Year: 2020
Type: other
Abstract: The changing landscape of genomics research and clinical practice warrants computational pipelines that efficiently orchestrate complex analyses of large data volumes across heterogeneous environments. We performed a systematic evaluation of the popular bioinformatics Workflow Management Systems (WfMS): Nextflow, CWL and WDL and some of their executors, along with Swift/T, a WfMS commonly used in high-scale physics applications. A genomic variant calling pipeline and a scalability-testing framework were developed in each and run locally, on an HPC cluster and in the cloud. This allowed contrasting the four in terms of language expressiveness, modularity, scalability, robustness, reproducibility, interoperability, ease of development, and adoptability. This evaluation helps guide the choice of the appropriate WfMs as driven by the interplay of the workflow language, executor and computational ecosystem, as well as the collaborative practices of our field. We also arrived at a set of design considerations to further improve WfMS deployment in research and clinical settings. As the community and its needs continue to evolve along with computational infrastructure, WfMS will evolve, especially those with permissive licenses. Just as the dataflow paradigm and containerization are now widely used, this field will continue to see innovations elsewhere, such as big data technologies and portability.
Authors: Azza Ahmed, Joshua Allen, Saurabh Baheti, Tajesvi Bhat, Matthew A Bockol, Prakruthi Burra, Travis M Drucker, Steven N Hart, Jacob R Heldenbrand, Matthew Hudson, Michael Kalmbach, Gregory Kapraun, Eric W Klee, Katherine Kendig, Matthew Kendzior, Nathan R Mattson, Christian A Ross, Sami M Sharif, Eric D Wieben, Mathieu Wiepert, Ramshankar Venkatakrishnan, Faisal M Fadlelmola, Liudmila S Mainzer
Institution: University of Illinois Urbana-Champaign
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W3110675604
Cites:
Cited by:
Related to: