From Lost in Translation to Paradise Found: Enabling Protein Biomarker Method Transfer by Mass Spectrometry
Year: 2014
Type: article
Abstract: Recently, Begley and Ellis delivered a sobering message to laboratories around the world (1). The lack of meaningful progress in preclinical cancer research was highlighted by the irreproducibility of >70% of published studies. The authors also crystallized the importance of full disclosure and the validation of critical scientific discoveries for industry-wide improvement. Translation of novel biomarkers into clinical care for the evaluation of therapeutic safety and efficacy has been slow (2), partly because of the cost and complexity of immunoassay development. The potential for liquid chromatography–tandem mass spectrometry (LC-MS/MS)3 to streamline the translation of novel protein biomarkers is profound (3). Most LC-MS/MS-based protein assays incorporate denaturation and proteolytic digestion of proteins in the sample into peptides (traditionally called “bottom-up” proteomics). These preparative steps destroy potentially interfering proteins into peptides that can be resolved and ignored by LC-MS/MS (4). Inclusion of stable isotope–labeled internal standard proteins or peptides (which may be cleavable) in each sample enables correction for matrix effects, including sample-related digestion variability and/or ion suppression, both significant analytical benefits compared with immunoassays. Downstream members of the scientific community are hopeful about translating important preliminary findings into clinical practice; however, success has been hampered by a lack of transparency and insufficient validation. Consequently, LC-MS/MS-based clinical protein analysis has predominantly focused on improved analytical measurement for well-established biomarkers (5). This is despite “fit-for-purpose” criteria for enablement (6, 7) and published recommendations for analytical validation (8), based primarily on U.S. Food and Drug Administration guidance (9). Although assays used in preclinical research are generally not held to the same standards as assays used in the immediate care of patients, which are governed by CLIA-88 and by extension many Clinical Laboratory Standards Institute consensus documents, published fundamental discovery experiments and biomarker verification studies spawn costly research programs. To advance our efforts as a community, LC-MS/MS protein quantification data used to support published research findings should be from properly designed studies (10) and accompanied by a standard operating procedure that includes sufficient detail to facilitate assay reproduction in other laboratories. The careful definition of the measurand is essential (11). Appropriate method validation should include experiments that evaluate and document key analytical performance characteristics. To this end, we present and discuss a minimal list of experiments (Table 1) that would allow downstream users of novel biomarkers to carefully evaluate their quality and potential reproducibility. Although certain experiments are uniquely associated with bottom-up LC-MS/MS proteomics work flows, we believe that there are universally applicable concepts described within this document that should be applied to alternate technologies in biomarker translation. Given the lessons of the past (1), we cannot overemphasize the importance of this level of transparency and rigor in the publication of novel scientific discoveries. Best practice acceptance criterion as defined by Lee et al. (7), acknowledged as a hybrid of immunoassays and LC-MS/MS validation criteria derived from DeSilva et al. (8). Best practice acceptance criterion as defined by Lee et al. (7), acknowledged as a hybrid of immunoassays and LC-MS/MS validation criteria derived from DeSilva et al. (8). For the vast majority of clinical and preclinical assays, precise measurements facilitate longitudinal monitoring of disease, resolution of the disease continuum with confidence, and hypothesis verification. The imprecision of an assay can be assessed within batches (repeatability) or longitudinally and between laboratories (reproducibility). We propose the use of 2 pools, presumably with different concentrations, for the marker of interest: a disease pool comprising equal volumes of known disease samples (ideally n = 20) and a healthy pool (n = 400, derived from Ichihara et al. (12)). We have recently collaborated to enable a commercial source of pools from healthy controls (EDTA plasma and serum, men or women 20–50 years old, Golden West Biologicals). An estimation of total variability per pool should be determined from 5 individual replicates of each pool assayed each day for 5 days. The mean intra-assay (CVintra) and inter-assay (CVinter) CVs for each pool should be calculated. CVintra includes all 5 replicates (1–5) per pool measured in a single day. CVinter is determined for each replicate individually across all 5 days (CV for replicate 1 across 5 days, replicate 2 across 5 days, etc.). The total variability is determined by use of the mean CVintra (across 5 days) and the mean CVinter (across all 5 replicates) by the sum of squares, CVtotal = (CVintra2 + CVinter2)1/2, and is reported for each pool. The determination of mean interassay protein concentration (intermc) for these pools (i.e., the mean of 25 results for each pool) will be used for subsequent validation experiments (the concentration measured in each sample is calculated as the ratio of the endogenous peak area to the spiked internal standard peak area multiplied by the concentration of internal standard spiked into the samples; if appropriate, results from multiple peptides are averaged for each sample). If internal standard peptide spiked after digestion is used in the calculation of the concentration, it is unlikely to be entirely accurate because of incomplete proteolytic digestion and nonlinearity of the ratio when it deviates from 1.0, but it will provide a frame of reference for subsequent experiments. We further propose that all subsequent experiments described in this paper be determined by use of triplicate samples, with CV reported at each concentration for each experiment. Accuracy is often difficult to achieve for protein assays because of the lack of standard reference materials and assays, particularly for novel biomarkers. We propose the use of the interassay mean concentration determined for the healthy pool (or disease pool where biomarkers are normally absent) as a calibration material in preclinical experiments. The intrinsically normalizing size of the healthy pool offers a concentration anchor point (intermc) for comparative accuracy purposes to improve repeatability and reproducibility concordance (13, 14). The majority of preclinical research studies incorporate isotope-labeled internal standard peptides (ISs) after digestion. However, the influence of proteolytic peptide formation/degradation relative to IS and its effect on assay bias must be determined (13). The disease and healthy pools are proteolysed with IS addition predigestion (ISpre), and protein concentrations are compared to intermc with IS addition postdigestion (ISpost). Estimation of bias for protein determination due to peptide degradation during the proteolysis step is calculated as (ISpre − ISpost)/ISpost, expressed as a percentage. This experiment should be performed at least twice, but can be eliminated if internal standards are routinely added predigestion. Whereas the imprecision of a preclinical assay is important in distinguishing diseased from healthy individuals or one pathophysiologically important state from another, a narrow analytical dynamic range can make this difficult. To evaluate linearity, we propose a 5-point mixing scheme. The study includes the disease and healthy pools described above, together with 3:1, 1:1, and 1:3 admixtures of these pools before sample preparation. Admixture recoveries should be calculated against expected protein concentrations generated via linear extrapolation of expected disease and healthy pool intermc results (from the 5 replicate/5 day experiment above) and the ratio of admixtures (e.g., expected 1:1 mixture concentration = mean of disease intermc and healthy intermc). This experiment should be performed at least twice and highlights the analytical capability for disease differentiation at the individual analyte level, together with a preliminary determination of matrix effects. Dilution studies of the healthy pool are used to estimate the lower limit of quantification when analyte is present (disease pool when analyte is absent). The healthy pool should be gravimetrically diluted (serial 2- to 5-fold dilutions) with analyte-free surrogate or alternate species matrix until analyte is no longer quantifiable. This experiment should be performed at least twice; recovery (accounting for dilution) and imprecision should be reported. In addition to evaluating for matrix effects by use of linearity, we also propose to evaluate the effects of common clinical interferences. A test kit containing supraphysiological interferences has recently been commercialized for this study (Assurance Interference Test Kit, Sun Diagnostics). Evaluation of bias is performed for lipemia (triglycerides 3000 mg/dL or 33.9 mmol/L), hemolysis (hemoglobin 500 mg/dL), icterus (bilirubin 20 mg/dL or 342 μmol/L), and hyperproteinemia (total protein 12 g/dL). Influence of clinical interferents (determined as percentage bias) is performed by spiking interferents into the healthy pool, measuring the protein concentration, and comparing to the healthy intermc, accounting for dilution in the expected concentration. When the spiked interferent contains the protein analyte, the concentration of analyte in the spiked interferent should be determined from a 1:1 admixture of the interferent and an analyte-free matrix. This concentration should be used to determine the contribution to the measured concentration of the interferent-spiked healthy pool and subtracted to evaluate for bias. In routine clinical LC-MS/MS assays of small molecules, transition ratio monitoring (ratio of quantifying transition peak area to the qualifying transition peak area) is used to document selectivity of the approach, identifying samples with isotopic/isobaric interferences and thereby providing confidence in concentration assignment (15). This has been expanded to include proteins (5, 16), whereby alternative products of the same peptide generated by the mass spectrometer can be used to confirm the identity of the molecule being quantified. For each of the validation studies performed, we propose the disclosure of transition ratio monitoring results. In addition, transition ratio monitoring results must be disclosed for all samples assayed during preclinical studies. In routine bioanalytical assay validation (9), assessment of stability requires purified analyte for generation of fresh calibration standards to assay samples both before and after stressed storage. We propose a relative bias approach (against intermc) by use of both disease and healthy pools for stability evaluation. Frozen storage bias is assessed through the analysis of pool aliquots assayed 30 days after generation of intermc. Sample handling stability bias should be determined on pre-extracted samples after storage of aliquots at room temperature (20–24°C) for 4 h, refrigeration (4–8°C) for 24 h, and up to 2 freeze–thaw cycles. Postextraction stability should be determined for both pools after storage in the autosampler (>24 h, reinjecting aliquots if feasible), freezing (>72 h, if routine), and extract freeze–thaw for 1 and 2 cycles. Because many preclinical studies rely on biobanked materials, it should be noted that at least 3 freshly acquired samples should be evaluated for stability of 1 freeze–thaw cycle (assay fresh, freeze for >12 h, thaw for >2 h, reassay, and compare). For complete transparency, we propose that authors submit processed analytical data to a web-based repository, such as Panorama/Skyline (16), to enable detailed critical review of published results and the human influence in data reduction (10). A meticulous description of key reagents used in each assay should be included in the supplemental data section together with the standard operating procedures used to perform the preclinical studies. The proposed framework will enable us as a community to fully evaluate the potential of novel biomarkers published in the literature. If those biomarkers are truly discriminatory, we can improve patient care. liquid chromatography–tandem mass spectrometry isotope-labeled internal standard peptide.
Source: Clinical Chemistry
Authors: Russell P Grant, Andrew N Hoofnagle
Institution: University of Washington
Language: English
Subfield: Spectroscopy
Field: Chemistry
Domain: Physical Sciences
Keywords: protein biomarkers, method transfer, analytical method validation, LC-MS/MS, immunoassay comparison, reproducibility, matrix effects, ion suppression, isotope-labeled internal standards, assay reproducibility, preclinical cancer research, biomarker validation, standard operating procedures, CLIA 88, FDA guidance, shotgun proteomics, immunoassay development
Open Access status: green
OpenAlex ID: W2110958540
