Peer Review Report For: DREAMTools: a Python package for scoring collaborative challenges [version 1; peer review: 3 approved with reservations]
Year: 2015
Type: peer-review
Abstract: DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of September 2015, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform https://www.synapse.org. Availability : DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools.
Authors: Thomas Cokelaer, Mukesh Bansal, Christopher Bare, Erhan Bilal, Brian M. Bot, Elias Chaibub Neto, Federica Eduati, Mehmet Gönen, Steven M. Hill, Bruce Hoff, Jonathan R. Karr, Robert Küffner, Michael P. Menden, Pablo Meyer, Raquel Norel, Abhishek Pratap, Robert J. Prill, Matthew T. Weirauch, James C. Costello, Gustavo Stolovitzky, Julio Saez-Rodriguez
Institutions: European Bioinformatics Institute, Wellcome Trust, Columbia University, Sage Bionetworks, IBM (United States), Oregon Health & Science University, MRC Biostatistics Unit, Icahn School of Medicine at Mount Sinai, Helmholtz Munich, Institute of Bioinformatics and Systems Biology, IBM Research - Almaden, Cincinnati Children's Hospital Medical Center, University of Colorado Anschutz, Universitätsklinikum Aachen, RWTH Aachen University
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4416618033
Cites:
Cited by:
Related to: