Script of Scripts: A pragmatic workflow system for daily computational research
Year: 2019
Type: article
Abstract: Computationally intensive disciplines such as computational biology often require use of a variety of tools implemented in different scripting languages and analysis of large data sets using high-performance computing systems. Although scientific workflow systems can powerfully organize and execute large-scale data-analysis processes, creating and maintaining such workflows usually comes with nontrivial learning curves and engineering overhead, making them cumbersome to use for everyday data exploration and prototyping. To bridge the gap between interactive analysis and workflow systems, we developed Script of Scripts (SoS), an interactive data-analysis platform and workflow system with a strong emphasis on readability, practicality, and reproducibility in daily computational research. For exploratory analysis, SoS has a multilanguage scripting format that centralizes otherwise-scattered scripts and creates dynamic reports for publication and sharing. As a workflow engine, SoS provides an intuitive syntax for creating workflows in process-oriented, outcome-oriented, and mixed styles, as well as a unified interface for executing and managing tasks on a variety of computing platforms with automatic synchronization of files among isolated file systems. As illustrated herein by real-world examples, SoS is both an interactive analysis tool and pipeline platform suitable for different stages of method development and data-analysis projects. In particular, SoS can be easily adopted in existing data analysis routines to substantially improve organization, readability, and cross-platform computation management of research projects.
Source: PLoS Computational Biology
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,350
Funders: Michael and Susan Dell Foundation, Gordon and Betty Moore Foundation, Mary Kay Foundation, Mary K. Chapman Foundation, Cancer Prevention and Research Institute of Texas, University of Chicago, National Institutes of Health, University of Texas MD Anderson Cancer Center, National Human Genome Research Institute, National Cancer Institute
Awards: Matthew Stephens Investigator Award, A Proteomics and Metabolomics Core Facility at the University of Texas MD Anderson Cancer Center, G293715275, Computational tools for sequence-based large-scale epidemiology studies, Statistical Methods for Population Genomics and "Next-gen" Sequencing Data, G3746653997, Integrative Pipeline for Analysis & Translational Application of TCGA Data (GDAC), G6110703196, Cancer Center Support Grant
OpenAlex ID: W2915479314