We evaluated three different types of assays (Fig. 1a) currently used for protein biomarker studies for their respective figures of merit and practical considerations. We found that each platform delivered varying degrees of depth and range for each individual analyte measured (Fig. 2), even in the cases where the two immuno-based assays claim to measure the absolute values of the same protein (Fig. 3). We found that platforms had inherent technical variation evaluated with technical replicates (Fig. 4); however, we were still able to detect biological signal that generally surpassed technical variation (Fig. 5a, c). Despite that most variability was captured by biological effects (Fig. 5c), paired and unpaired comparisons of pre-diagnostic case and control samples yielded 32 significant proteins (Fig. 6) with altered expression levels, with a limited subset previously reported to be associated with cancer. No proteins, however, retained statistical significance after false discovery correction. We conclude that technical, practical, and study design considerations are essential to maximize information obtained in limited pre-diagnostic samples of cancer patients.
The motivation for this study was to perform an exploratory investigation of technical and study design implications for identifying potential protein biomarkers in a small sample (n = 10) of pre-diagnostic breast cancer biospecimens. The pre-diagnostic plasma samples from the BCFR in this study – and human clinical cancer biospecimens in general – are often relatively limited in number and amount and difficult to come by. Thus one often arrives, as we did in this study, at a situation wherein there are multiple available technologies to analyze precious samples. It is therefore critical to understand the implications of technical choices, and to carefully design experiments to maximize information obtained.
In this study, we selected three different types of assays (Fig. 1a) currently used for protein biomarker studies. We evaluated each type assay for their respective merit and practical considerations. We leveraged specimens from matched sisters who remained cancer-free as controls and compared them to pre-diagnostic specimens of sisters who developed breast cancer within 2 years of providing the blood sample. Therefore, in addition to comparatively assessing the technical platforms, we also evaluated whether a paired design would bring more statistical power to account for confounding biological variability. Though modest in sample size, this study serves as a model pilot study and practical strategy to evaluate important factors before designing a larger study.
Each of the three methods evaluated in this study comes with their own advantages and disadvantages. The unbiased shotgun LC-MS/MS method not only avoids a constrained pre-selection of analytes of interest, but at the same time can also quantify many more proteins than targeted panels such as the Myriad-RBM and Olink platforms (Fig. 2a). However, LC-MS/MS and most other quantitative mass spectrometry-based methods provide relative rather than absolute quantitation. Absolute measurements require the use of internal standards, which is not feasible in an untargeted study of hundreds of proteins. In recent years, emerging mass spectrometry-based technologies [16] have started to build quantitative assays for larger numbers of proteins; however, these technologies require either a priori knowledge of analytes of interest, or peptide spectral libraries that still rely on data dependent acquisition methods as exhibited in this study. The LC-MS/MS approach is also more sensitive towards higher abundance proteins, thus impacted by a relatively large amount of missing data compared to the antibody-based approaches due to the inherent under-sampling of the mass spectrometer.
In contrast to the LC-MS/MS method, the Myriad-RBM and Olink approaches provide absolute measurements of specific protein levels (Additional file 1: Table S2) which are useful for targeted rather than exploratory studies. These platforms provide respective datasets with protein measurements for nearly every analyte in every sample, with little missing data (Fig. 2b-c). However, though the subset of proteins that were measured by both platforms provided mostly concordant values (Fig. 3b), there remained a clear discrepancy in absolute measurements between the platforms (Fig. 3a) which raises concerns about the reliability of these reported absolute values. This discrepancy is possibly due to differential calibrations used by the different technologies. Moreover, for protein measurements that were not concordant between the two antibody-based methods, it is possible that corresponding antibodies may not be measuring the same epitope of a protein and are henceforth capturing different protein forms. Another disadvantage of these antibody-based technologies is that antibodies can either be inherently biased due to their affinity to substrates, or unavailable for potential novel protein biomarkers.
In addition to the technical features of these assays, there are several practical considerations for each approach. The LC-MS/MS approach was the most time-consuming approach due to time needed to perform fractionation to overcome the large dynamic range of proteins in the blood and the time required for LC-MS/MS data acquisition. The Olink assays required the smallest volume of plasma samples of all the platforms, which is an important practical consideration when working with the precious and limited pre-diagnostic samples. Lastly, each assay comes at a different cost which may determine the feasibility of performing assays in duplicate, or even triplicate.
The three assay platforms provide both complementary and mutually exclusive information. Therefore, practical considerations and priorities may determine the choice of assay for evaluating clinical samples for biomarker discovery. All assays used in this this study had some inherent technical variation. We also chose to evaluate technical variation of the antibody-based Olink platform rather than the Myriad-RBM platform largely because the Olink assay requires significantly smaller volume of our limited biospecimens. We found some technical variation in the Olink assay, but it was not correlated with the type of analyte or measured protein abundance (Fig. 4b-c), thus establishing reliable scalability of the assay across all protein targets.
Despite technical variation detected with our triplicate analysis, we observed that the signal from biological variation of protein levels between samples was larger than the technical variation (Fig. 5a-b). Interestingly, each protein analyte in the same antibody-based platform has its own technical and biological variation (Fig. 5c), which cannot necessarily be predicted a priori, probably owing to inherent biases in the antibodies, or to confounding biological noise. For each analyte in a protein measurement study, it is thus important to evaluate the technical and biological variation to account for confounding in the analytical signal. Our findings therefore support the utility of replicates in studies to assess analyte variation.
Lastly, in addition to assessing the inherent variability within and between the different protein assays, we also evaluated whether a matched case and control sisters study design would yield more statistical power in uncovering significant early diagnostic biomarkers. We initially hypothesized that sisters of the breast cancer cases would serve as well-matched controls as they are naturally controlled for race/ethnicity and a large proportion of genetic background. Moreover, our choice to evaluate pre-diagnostic samples eliminates bias associated with sample collection as both case and control samples were collected under the same conditions without knowledge of cancer diagnosis at the time of collection.
However, the unbiased principal component analysis exhibited that even with our careful study design choices to minimize bias, samples showed relatively large individual variance along their two principal component axes in the antibody-based datasets. Samples from biological sisters did not generally appear to be more similar to each other than to other individual samples (Additional file 3: Figure S1B). Hence it was not surprising to discover that the case and control samples were not well separated in the two-dimensional space of components that captured a substantive amount of the overall variance (>90%) (Additional file 3: Figure S1A).
Henceforth, we focused our analysis efforts on identifying individual protein analytes with potential diagnostic power. While a small number of proteins showed statistically significant differences between cases and controls; the changes were small (less than two-fold) and the number of proteins showing differences was small (Fig. 6). This finding is not necessarily unexpected. Given that the samples were collected from women up to 2 years before they were diagnosed with breast cancer, it is highly likely that the analytical signal in the blood plasma associated with breast cancer cannot be detected above the biological noise that arises from inter-individual variation.
We observed in Fig. 6a that the paired statistical analysis in the Olink platform provided more significant analytes than the unpaired analysis, suggesting that for the Olink platform, the paired design had greater statistical power when evaluating each protein analyte independently. However, also shown in Fig. 6a, we observed that there were some significant analytes that were mutually exclusive to the paired and unpaired analysis in the Myriad-RBM platform. If evaluating this platform in isolation, these results echo our initial PCA (Additional file 3: Figure S1); it appears that on the Myriad-RBM platform, the sisters are not more similar to each other than across all the samples both when protein levels are analyzed independently, and in covariance. This result may be explained by factors including, but not limited to, noise introduced by technical variation that was not evaluated for this platform, coupled with person-to-person variation from the samples. The Myriad-RBM oncology panel was chosen specifically for this study, and our results suggest that these protein measurements show little utility in pre-diagnostic breast cancer samples. This suggests that there is still a need for the unbiased exploratory LC-MS/MS methods that yield novel significant biomarker candidates as seen in Fig. 6b. Although the protein analytes in Fig. 6b have not been validated, they serve as a proof of concept that an untargeted approach can yield candidates that would otherwise be missed by targeted panels applied to noisy clinical samples.