Author response: Mars, a molecule archive suite for reproducible analysis and reporting of single-molecule properties from bioimages
Year: 2022
Type: peer-review
Abstract: Article Figures and data Abstract Editor's evaluation Introduction Results Discussion Methods Data availability References Decision letter Author response Article and author information Metrics Abstract The rapid development of new imaging approaches is generating larger and more complex datasets, revealing the time evolution of individual cells and biomolecules. Single-molecule techniques, in particular, provide access to rare intermediates in complex, multistage molecular pathways. However, few standards exist for processing these information-rich datasets, posing challenges for wider dissemination. Here, we present Mars, an open-source platform for storing and processing image-derived properties of biomolecules. Mars provides Fiji/ImageJ2 commands written in Java for common single-molecule analysis tasks using a Molecule Archive architecture that is easily adapted to complex, multistep analysis workflows. Three diverse workflows involving molecule tracking, multichannel fluorescence imaging, and force spectroscopy, demonstrate the range of analysis applications. A comprehensive graphical user interface written in JavaFX enhances biomolecule feature exploration by providing charting, tagging, region highlighting, scriptable dashboards, and interactive image views. The interoperability of ImageJ2 ensures Molecule Archives can easily be opened in multiple environments, including those written in Python using PyImageJ, for interactive scripting and visualization. Mars provides a flexible solution for reproducible analysis of image-derived properties, facilitating the discovery and quantitative classification of new biological phenomena with an open data format accessible to everyone. Editor's evaluation This is a valuable paper that reports an open-source platform for the storage and processing of single-molecule, camera-based, imaging data. The development and testing of the platform are very compelling and the platform will facilitate data sharing and reproducibility and will be of great interest to practitioners of single-molecule imaging experiments, both experienced and new to the field. The work represents significant and important steps towards unifying and standardizing how the field stores and processes data and expanding the base of researchers who can easily employ single-molecule imaging methods. https://doi.org/10.7554/eLife.75899.sa0 Decision letter Reviews on Sciety eLife's review process Introduction Reproducible analysis of bioimaging data is a major challenge slowing scientific progress. New imaging techniques generate datasets with increasing complexity that must be efficiently analyzed, classified, and shared. This challenge has gained wide recognition (Carpenter et al., 2012; Eliceiri et al., 2012; Lerner et al., 2021; Meijering et al., 2016; Ouyang and Zimmer, 2017), which has led to the development of software frameworks for curating images (Allan et al., 2012; Kvilekval et al., 2010) and analysis workflows (Rubens et al., 2020), metadata reporting standards (Goldberg et al., 2005) and the creation of public archives to increase data availability (Williams et al., 2017). However, few standards exist for the reproducible analysis and reuse of image-derived properties and those that have been developed for single molecule fluorescence (Greenfeld et al., 2015; Ingargiola et al., 2016) offer limited options for adaption to other experimental configurations. There is an increasingly powerful toolkit to quantitatively follow the time evolution of the position, shape, composition, and conformation of individual cells and complexes, but these precious information-rich observations are generated in heterogenous formats that do not provide easy, transparent access to key features. As a consequence, the promise of new technologies is often unrealized, and new biological phenomena remain undiscovered in existing datasets, due to the lack of tools that enable robust classification and interactive exploration. Single-molecule techniques provide access to rare intermediates in complex, multistage molecular pathways. Multicolor fluorescence imaging has revealed the conformational dynamics of membrane transport (Akyuz et al., 2013; Erkens et al., 2013), molecular states underlying assembly and transcription by RNA polymerase (Baek et al., 2021; Duchi et al., 2016), and DNA replication dynamics (Duderstadt et al., 2016; Scherr et al., 2018; Ticau et al., 2015). These approaches have been combined with spatial tracking on biological structures to clarify how exchange events and conformational changes modulate the function of motor proteins, as well as replication, transcription, and DNA repair machineries (Crickard et al., 2020; Lewis et al., 2020; Niekamp et al., 2021; Scherr et al., 2022). Biological macromolecules are routinely attached to microspheres which allow for controlled studies of the forces and torques involved in basic biological reactions (Agarwal and Duderstadt, 2020; Dulin et al., 2013; Neuman and Nagy, 2008; Revyakin et al., 2006). And finally, improvements in the sensitivity of camera sensors, fluorophore brightness, and new illumination strategies Gao et al., 2012; Tokunaga et al., 2008 have enabled time-resolved studies of biological processes in live cells. These developments have revealed frequent exchange of factors during normal operation of replisomes (Beattie et al., 2017; Kapadia et al., 2020) and the dynamics of the transcription-factor target site search (Chen et al., 2014). The discovery of new biological phenomena from these multidimensional observations, depends on long, multistage image analysis workflows, followed by careful grouping and feature classification. Images are typically preprocessed, to correct for non-uniform beam profiles and filtered to enhance detection of individual molecules and structures over the background. Additional tools are then used to follow the properties of individual biomolecules through time. These often provide some form of tabular data containing aggregated results, which are either manually evaluated or filtered using a feature unique to the desired group of observations. Typically, at this point, results can no longer be re-evaluated in the context of the original images due to unidirectional transformations or lack of interactive image evaluation software. The final data are frequently migrated to another platform ideally suited for the generation of publication-quality figures. This necessitates further data restructuring during transfers between software platforms. Large datasets are downsized to cope with limited storage by removal of rejected observations in ways that irreversibly alter datasets. As rejection criteria change and additional data must be incorporated, precious time and reproducibility are lost. This process is often repeated for each experimental application and research group, leading to substantial duplication of efforts. Recognizing these issues in common single-molecule image processing workflows, we developed Mars, which provides a collection of Fiji/ImageJ2 (Pietzsch et al., 2012; Rueden et al., 2017; Schindelin et al., 2012) commands and integrations with well-established Fiji workflows for processing images and image-derived properties based on a versatile Molecule Archive architecture (Figure 1). This architecture allows for seamless virtual storage, merging, and multithreaded processing of very large datasets. A simple, yet powerful interface allows for fast and easy access to subsets of records based on any biomolecule property. This framework allows for the same data structure to be used from initial image processing all the way to the generation of final figures. All image metadata and biomolecule records are assigned universally unique ids (uids) upon creation, which together with comprehensive logging, ensures the history of each record remains traceable through long and complex analysis workflows involving numerous data merging steps. Non-destructive tag-based filtering ensures no observations are lost and updates to rejection criteria only require retagging of processed records. These design principles facilitate the reproduction of analysis workflows and ensure Molecule Archives provide a comprehensive, transparent format for the deposition of final datasets. Figure 1 Download asset Open asset Overview of Mars workflows. The process of reproducible data analysis with Mars starting from image processing to iterative rounds of classification and filtering to the final stage of data exploration and deposition into a public database. The modular design of commands and minimal definitions of image metadata and biomolecule records ensure flexibility that facilitates the development of varied Mars workflows. To illustrate the range of image processing and kinetic analysis tasks that can be accomplished, we present the results of three applications. We demonstrate the versatility of Mars record types in representing both single molecules as well as large macromolecular structures by tracking single RNA polymerases on long DNAs. We benchmark a multichannel fluorescence integration workflow using well-established single-molecule FRET frameworks. And finally, we show how high-throughput imaging of DNA-tethered microspheres reveals the results of complex topological transformations induced by controlled forces and torques. In each example, we highlight image processing commands and methods for accessing and manipulating Molecule Archives using scripts. Each workflow provides a basic framework that can be further adapted to custom applications by the introduction of additional processing steps and modification of the scripts provided. The graphical user interface provided by the Mars Rover facilitates workflow refinement through interactive data exploration and manual classification with multichannel image views. Taken together, these features make Mars a powerful platform for the development of imaging-processing workflows for the discovery and quantitative characterization of new biological phenomena in a format that is open to everyone. Results Common pitfalls in single-molecule image-processing workflows Single-molecule imaging platforms provide an increasingly powerful toolkit to observe complex biological systems, but obtaining quantitative information depends on multistage analysis workflows with many potential pitfalls. The core design principles of Mars were developed to avoid common issues. Before introducing the architecture of Mars in depth, we will illustrate the benefit of Mars with a practical example. For those not yet experienced with single-molecule imaging, this example should help to clarify how Mars improves data readability and workflow reproducibility to avoid issues that frequently arise during dataset transformations when analysis steps are not properly documented and annotated. Many common single-molecule experiments involve recording images of the position and intensity of fluorescent labels attached to biomolecules as a function of time. For example, consider the simple experiment of fluorescently labeled polymerases traveling along DNA molecules during synthesis (covered in greater depth in workflow 1). Experimenters typically want to track polymerase position and fluorescence intensity in consecutive images. From the raw images, they would like to determine the polymerase synthesis rate, product sizes, and number of polymerases on each DNA molecule. This is accomplished by finding the locations of fluorescent spots in individual images, followed by linking spots from the same molecules over time. In a typical single-molecule workflow, tracking results are generated in a single table with columns for the image number, molecule position, and an index number for each track. Several filters are used to select moving polymerases that travel a minimum distance and have an intensity consistent with a single dye. After this automated filtering, tracks are manually checked to remove those with tracking errors. In a typical workflow, these filtering steps remove rejected tracks from the table. Finally, a conversion factor that relates DNA length to polymerase coordinates is used to calculate the synthesis rate and product length. The process is then repeated for many videos generating a final merged table with reindexed global track numbers. This typical workflow has several pitfalls that increase the analysis time, make discovery of new phenomena more difficult, lead to irreproducibility, and, in the worst case, result in mistakes. The most significant problem is that every step is unidirectional. Tracks rejected during automated or manual filtering are removed. As a consequence, if any filtering criteria change, the analysis must be repeated from the start. Moreover, no framework is provided to easily document which scripts and thresholds were used and why tracks were manually rejected. The most significant pitfall arises from merging multiple videos and reindexing track numbers. This makes it impossible to evaluate the final dataset in the context of the original raw videos and their metadata. For example, when a mistake is discovered in a subset of experiments, such as poor polymerase labeling, incorrect temperature, illumination, or contamination, there is no way to easily remove the subset of experiments from the final dataset since the tracks have no identity linking them to the original videos. Beyond these immediate practical pitfalls, the structure of the workflow makes it difficult to ask complex questions that relate one biomolecule property to another and quickly re-evaluate the dataset based on a new model. For example, how does the synthesis rate depend on the number of polymerases? How does polymerase number influence the frequency of pauses? What if there are large differences in DNA extension between molecules or videos that require unique correction factors. The entire analysis and filtering would need to be done again. Mars was developed to address these common pitfalls that arise in workflows like the example provided. First, Mars provides image processing commands streamlined for single-molecule applications that generate Molecule Archives containing a record of the analysis steps. Filtering is done using tags without the removal of rejected data to avoid unidirectional workflows. When filter criteria change only the tags need to be updated without redoing the analysis. Different subsets of existing tags may also be used to study complex differences between subpopulations such as polymerase clusters. The uids given to records upon creation, together with unique metadata ids, ensure a traceable history when datasets are merged, so that final datasets can always be evaluated in the context of the original raw images. The biomolecule storage and graphical interface in Mars are built to simplify the exploration of complex relationships based on different biomolecule properties. If experimental problems are discovered in a subset of videos, they can easily be filtered using the metadata ids stored with the records. All of these elements provide endless reslicing possibilities using the same dataset that overcome common pitfalls in single molecule image processing workflows. Molecule Archive architecture Molecule Archives provide a flexible standard for storing and processing image-derived properties adaptable to a broad range of experimental configurations. Molecule Archives contain three record types: Properties, Metadata, and Molecule (Figure 2). Each type is defined in an interface, independent of implementation details. This abstraction ensures Molecule Archives support a variety of biomolecule, metadata, and property implementations. Moreover, this allows for the creation of new implementations that seamlessly work with the existing code base, algorithms, and user interface. To simplify the mechanics of record retrieval and the process of dataset merging, all Molecule and Metadata records are assigned human-readable, base58-encoded universally uids. Storing records using uids reduces indexing requirements, facilitates scalable processing using uid-to-record maps that support multithreaded operations, and ensures the traceability of records through analysis workflows. Figure 2 with 1 supplement see all Download asset Open asset Molecule Archive structure. Schematic representation of the structure of Molecule Archives consisting of three types of records: Properties, Metadata, and Molecule. The single Properties record contains global information about the Molecule Archive contents, the Metadata records store information about the images used for biomolecule analysis (e.g. image dimensions, the analysis log), and the Molecule records store molecule-specific information (e.g. position over time, intensity). Molecule Archives contain a single Properties record with the type of the Molecule Archive and global information about the Molecule Archive contents. This includes the number of Metadata and Molecule records and unique names used for tags, parameters, regions, positions, and table columns. The Properties record stores global Molecule Archive comments typically containing important information about the analysis strategy and naming scheme for tags, parameters, regions, positions, and other fields to orient a new researcher that did not perform the original analysis. To improve the organization and readability of comments, the Mars Rover also provides a convenient Markdown editor. Metadata records contain experimental information about the images used for biomolecule analysis. To ensure compatibility with the broadest set of image formats and maximum reusability, image dimensionality, timepoints, channels, filters, camera settings, and other microscope information are stored in OME format (Goldberg et al., 2005). Molecule Archives do not store the raw images, only the image-derived properties and the source directory containing the project. Nevertheless, interactive image views linked to molecule records are supported through big data viewer (BDV) integration (Pietzsch et al., 2015) with HD5 and N5 (Stephan Saalfeld et al., 2022) formatted images. The image file locations, coordinate transforms, and other settings are stored in Metadata records. Global events in time influencing all biomolecules are documented using Metadata regions and positions that are available for kinetic analysis and scripting. Global experimental conditions are stored as parameters in the form of key-value pairs (e.g. buffer composition and temperature). Metadata records are categorized using tags to filter biomolecules by whole experiments. The Metadata records also contain a log where all commands and settings used for processing are recorded, thus maintaining the entire history of data processing throughout the analysis. Molecule records contain fields for convenient storage of common image-derived properties of biomolecules. This includes a table that typically contains the position and intensity over time, the uid of the Metadata record containing the primary experimental information, and the index of the image containing the biomolecule. Events of interest in time can be marked with regions and positions (e.g. activity bursts, dye bleaching), which allow for event-specific calculations and kinetic analyses. Calculated global biomolecule properties are stored as parameters in the form of key-value pairs (e.g. mean intensity, distance traveled, and mean position). Mars comes with kinetic change point (KCP) commands for unbiased identification of distinct linear regimes and steps (e.g. polymerase synthesis rate, FRET states, and dye bleaching) (Hill et al., 2018), which are stored in segment tables. And finally, Molecule records are categorized for later analysis using custom tags and notes for manual assessments. Molecule Archives can be created, saved, and reloaded in a variety of formats in multiple environments both desktop-based and without a user interface for parallel processing on high-performance clusters. Molecule Archives are saved in JSON format using the field schema outlined in Figure 2. By default, Molecule Archives are written to single files with a yama extension using smile encoding and compression of molecule table data to reduce file size, but can also be saved and reopened in plaintext JSON. Mars supports processing of very large datasets, that typically do not fit in physical memory using a virtual storage mode in which records are retrieved only on-demand supported by a simple filesystem-backed record hierarchy. This architecture provides a powerful and flexible framework for multistep analysis workflows involving very large datasets. Mars Rover–interactive molecule feature exploration and image views The discovery of new biological phenomena using single-molecule techniques often relies on manual exploration of individual biomolecules in primary images together with image-derived measurements. To simplify this process, we developed a user interface in JavaFX, called the Mars Rover, that provides access to all information stored in Molecule Archives. The Mars Rover is integrated into Fiji with windows available for all open Molecule Archives. Open Molecule Archives are available as inputs and outputs in Fiji/ImageJ2 commands and in supported scripting languages. In the Mars Rover, Molecule Archive windows contain tabs and subpanels that provide complete access to all fields of Molecule and Metadata records as well as global properties, comments, and interface settings (Figure 2—figure supplement 1). A customizable global dashboard provides information about the Molecule Archive contents and scriptable chart widgets that can be adapted to specific workflows. The Molecule tab features an interactive chart panel that reveals the time evolution of biomolecule properties and provides region and position highlighting tools. To ensure image-derived measurements of biomolecule properties are evaluated together with primary observations, interactive image views linked to molecule records are supported through BDV integration (Pietzsch et al., 2015). The Mars Rover was written to provide extensive possibilities for customization. All tabs and panels are defined in interfaces, independent of implementation details. This facilitates extension of the Mars Rover to support custom icons and display elements based on Molecule Archive type. This will enable further refinement and the development of workflow-specific displays by extending the core architecture in the future. Commands for image processing and biomolecule analysis Mars comes with a collection of several dozen Fiji/ImageJ2 commands for common single-molecule image processing and analysis tasks (Table 1). This includes commands to and track through time intensity and in images. Commands to correct for non-uniform beam and to region of interest with filtering In to image there are Molecule Archive commands for merging, and datasets as well as kinetic change point analysis. Finally, Mars has commands to interoperability with other common This includes for the single molecule dataset (Greenfeld et al., and files from experiments. 1 Mars of Fiji/ImageJ2 commands the analysis of image-derived biomolecule data in can be on the Mars in an the position can be a DNA molecules in an the position of both of the molecule can be a and tracks in in images classification by and tracks their of the intensity of a over all the intensity of a over all in an image with for the beam image intensity the of consecutive from to or from to to long linear such as DNA several individual videos into one a single with the information stored along the a Molecule a virtual Molecule Archive from an opened table with a index into a Molecule DNA a DNA Molecule Archive from a single Molecule Archive and a of DNA in the the of the DNA molecules to search for molecules in the single Molecule Archive that with this multiple Molecule Archives in a single into multiple virtual Molecule Archives in a single into a to the molecule to time to time as in the metadata or by a time and for the given a Molecule Archive and a to all molecules in the new columns for each molecule the between the regions for all molecules in the Molecule Archive and the as a molecule the on a molecule table and the as a molecule a or table to the a based on in a the of a based on the any table to the any table to the linear regions or steps in single-molecule This molecule and for linear a single change point in a single-molecule The is a table with the and fit or the the in a specific region of interest in all single-molecule that can be used as for the change point from one region of a image to data in file format using a and the data to Molecule Archive dataset files in plaintext format and the data to Molecule Archive Commands in a of the of Fiji when Mars is using the When a is are with a to the desired The image in Fiji is used as by image processing provide possibilities to processing image and fluorescence integration commands generate Molecule Archives as outputs which open in new windows when the can then be using the Mars Rover user interface in all open Molecule Archive windows and all analysis work can be saved to as one file with a yama extension containing the primary Molecule Archive data and a file with the extension containing Mars Rover settings used to the and dashboard The typical Mars workflow with the creation of a Molecule Archive by processing an image followed by biomolecule feature analysis and This typically the creation of custom analysis scripts in Fiji that Molecule Archives. are often by the yama file in a and using one of the Python and for common workflows are available on the Mars The format of ImageJ2 commands ensures they can be used in many different with or without a graphical user interface To facilitate these methods have been for all settings and example scripts for Commands can be combined into larger scripts that support multistep analysis workflows on high-performance clusters. This facilitates a from a workflow development using the graphical user interface of Fiji to high-performance parallel processing of many experiments in environments a graphical user interface. interoperability Fiji provides a comprehensive open-source platform for scientific image analysis containing well-established software for common imaging processing These technologies are integrated in a modular as that can be combined in Mars commands and data structures are integrated into
Source: eLife
Language: English
Subfield: Biophysics
Domain: Life Sciences
Open Access status: gold
Estimated APC paid: $2,500
OpenAlex ID: W4295338630