Performance of the MasSpec Pen for Rapid Diagnosis of Ovarian Cancer.

In our previous study, we analyzed a sample set including normal ovarian and HGSC tissue using the MasSpec Pen coupled to an Orbitrap mass spectrometer ( 21 ). Here, 2 additional sample sets were obtained and analyzed using the same approach ( Table 1 ). Fig. 2 shows representative MasSpec Pen mass spectra from the 3 different tissue types from the second sample set. A variety of small metabolites, such as glutamate ( m/z 146.045), ascorbate ( m/z 175.025), and glutathione ( m/z 306.077), as well as complex glyocerophospholipid (GP) species, such as glycerophosphoethanolamine (PE) P-18:0_20:4 ( m/z 750.546), glycerophosphoserine (PS)_18:0 18:1 ( m/z 788.546), and glycerophosphoinositol (PI) 18:0_20:4 ( m/z 885.551), were detected. Identification and corresponding mass errors are provided in Table S2 in the online Data Supplement for species tentatively assigned using high mass accuracy and tandem MS/MS measurements. Representative MasSpec Pen MS/MS spectra of ovarian tissue samples are shown in Fig. S2 in the online Data Supplement . The molecular profiles obtained from normal and HGSC from the second and the third sample sets presented similar trends to those previously observed from tissues analyzed in the first sample set (see Fig. S3 in the online Data Supplement ). In particular, mass spectra from normal ovarian tissue were characterized by high relative abundance of ascorbate and other small metabolites, whereas GP species appeared at higher relative abundances in the mass spectra from HGSC tissues ( Fig. 2 ). The mass spectra obtained from LGSC also presented a rich variety of GP species at higher relative abundances than in the mass spectra from normal tissues. Further, when comparing LGSC to HGSC, qualitative changes in the relative abundances were observed between the mass spectra of the 2 cancer subtypes ( Fig. 2 ). For example, higher relative abundances of plasmalogen PE species, such as m/z 722.514 and m/z 750.546, were observed in the mass spectra from LGSC, whereas glutamate and glycerophosphatidic acid (PA) 18:0 18:1 ( m/z 701.514) were at higher relative abundances in HGSC tissue. Cosine analysis was performed to evaluate the similarity in the mass spectra between the 3 sample sets for the same tissue class. Note that cosine analysis provided similarity values ranging from 0 (dissimilar) to 1 (identical). Similarity values of 0.76 ± 0.06 and 0.80 ± 0.06 were achieved from HGSC and normal data, respectively, showing data comparability between the 3 sample sets for the same tissue type. Next, we evaluated if the molecular information obtained from MasSpec Pen analysis using the Orbitrap mass spectrometer was predictive of normal, LGSC, and HGSC across different sample sets. In our previous study ( 21 ), a classifier for HGSC diagnosis was built from a sample set 1, a single sample batch, and evaluated using leave-one-out cross-validation only ( 21 ). Here, the histologically validated mass spectra obtained from the second set of samples were combined with the first sample set data to improve and validate the statistical classifiers (n = 43 normal, n = 36 HGSC). First, the combined data set was randomly split into a training set (n = 32 normal, n = 28 HGSC) to build the statistical model, and a validation set (n = 11 normal, n = 8 HGSC) to evaluate performance and test for possible overfitting. Using the training set, 100% clinical sensitivity, 96.8% clinical specificity, and 98.3% accuracy was achieved, which are an improvement over previously published results of 100% clinical sensitivity, 89.7% specificity, and 94.7% overall accuracy. In the validation set, 100% clinical sensitivity, specificity, and accuracy were obtained ( Fig. 3 ). Using our previous model (built on sample set 1 alone) to predict on sample set 2, we achieved 100% clinical sensitivity, 75.0% specificity, and 90.9% overall accuracy (see Table S3 in the online Data Supplement ), which was an overall lower performance than what we achieved by including samples from different batches in the training sample set. Note that tissue samples were grouped based on tissue type and, as such, the effect of sample source (or tissue bank) on method performance was not evaluated. Next, a statistical classifier to discriminate between normal and serous cancer (SC) samples was also built by combining the HGSC and LGSC samples into a single class (n = 43 normal, n = 43 cancer). Similarly, the model exhibited high performance in both training (93.9% clinical sensitivity, 96.9% specificity, and 95.4% accuracy) and validation sample sets (100% clinical sensitivity, 90% specificity, and 95.0% accuracy). To further test the performance of the statistical models (HGSC vs normal and SC vs normal), prediction was performed on the third sample set ( Fig. 3 ). Results are reported for samples of clear diagnosis by pathology (n = 28 normal, n = 24 HGSC), corresponding to 91.7% clinical sensitivity, 92.9% specificity, and 92.3% overall accuracy, with 4 out of 52 samples misclassified. A 3-class statistical model was also created to differentiate between normal ovarian, HGSC, and LGSC tissues. Because of the limited LGSC sample size (n = 7), the model was built using cross-validation on the entire data set. Despite the introduction of the new LGSC subtype, all HGSC and normal ovarian samples were correctly classified (n = 79, 100% accuracy). For LGSC classification, an accuracy of 71.4% was achieved, with 2 out of the 7 samples misclassified as HGSC. Collectively, the 3-class model showed strong overall performance for cancer diagnosis, resulting in an overall accuracy of 97.7%. A summary of all results, including confusion matrices, classification performance, and number of metabolic species included in the models is provided in Table S4 in the online Data Supplement . Among the species selected to generate the statistical models, ascorbate ( m/z 175.024) and taurine ( m/z 124.006) were selected as important for the discrimination of normal tissue, and lactate ( m/z 201.038), glutathione ( m/z 306.077), and the PI 18:0_20:4 ( m/z 885.551) were characteristic of HGSC and the combined cancer class. For the 3-class model, PE P-18:0_20:4 at m/z 750.546 was given a positive weight for the characterization of LGSC, in agreement with the trends in relative abundances observed in the representative mass spectra shown in Fig. 2 . Gluconate ( m/z 195.051) and glutamate ( m/z 146.046) were also selected as important for discrimination of LGSC and HGSC samples. FT (n = 15) and peritoneum (n = 17) tissue samples were also analyzed with the MasSpec Pen and compared to ovarian cancer tissues. Mass spectra obtained from peritoneum tissue were commonly characterized by fatty acid (FA) species, detected both as monomers, such as FA 18:1 ( m/z 281.247), and dimers, such as FA 18:0–16:0 ( m/z 537.487). Chlorinated triacylglyceride (TG) species were also detected at higher mass range, such as TG 52:3 ( m/z 891.720), reflecting on the fat content characteristic of peritoneal and omental tissues. High abundance of complex lipids, such as PI, PE, and PS species, were observed from the lipid and metabolite profiles obtained from FT samples ( Fig. 4A ). A list of tentatively identified species with mass errors is provided in Table S5 in the online Data Supplement . Molecular models to discriminate peritoneum and FT tissue from ovarian cancer samples from batch 1 and 2 were built by leave-one-out cross-validation analysis ( Fig. 4B ). A detailed summary of all classification results, including the number of metabolic species included in the models, is provided in Table S6 in the online Data Supplement . All peritoneum (n = 15) and FT (n = 11) samples were correctly classified when compared to HGSC and ovarian SC samples (100% clinical specificity). Clinical sensitivity values of 88.9% and 90.7% were achieved for HGSC (n = 36) and SC overall (n = 43) vs FT samples, respectively. When compared to peritoneum samples, lower clinical sensitivities were achieved for HG (80.6%) and SC (83.7%). Interestingly, many of the features selected by the models to characterize HGSC and ovarian SC tissue from FT and peritoneum tissue were similar to those used for discrimination against normal ovarian tissue, such as lactate, glutamate, or PI 38:4. Yet, ascorbate was given a positive weight for ovarian cancer classification. Hexose ( m/z 215.031) was selected as predictive of both FT and peritoneum samples, whereas a species at m/z 267.073, tentatively identified as inosine, a purine nucleoside, was only associated to FT tissue. The MasSpec Pen was coupled to a LIT and used to evaluate a subset of HGSC and normal ovarian samples (n = 20 normal, 20 HGSC). Note that the LIT provides lower performance for mass analysis with limited resolving power (1600 at m/z 885.7) and mass accuracy when compared to the resolving power (72 000 at m/z 885.547) and mass accuracy (<5 ppm) achieved with Orbitrap mass analyzers. Fig. 5A shows that the mass spectra obtained for the same HGSC sample analyzed by both systems are highly comparable, although a higher number of resolved lipid species were detected by Orbitrap analysis, as anticipated. Distinctive metabolic profiles for HGSC and normal ovarian samples were obtained by MasSpec Pen and LIT analysis (see Fig. S4 in the online Data Supplement ). Similar to the Orbitrap data, HGSC mass spectra were characterized by the high relative abundance of GP species, whereas normal samples displayed considerably lower lipid abundance and higher relative abundances of small metabolites. Statistical classifiers were built to evaluate the predictive performance for HGSC diagnosis based on LIT data. The data was randomly split into a training (n = 30) and validation sets (n = 10), yielding excellent classification performance, with 100% clinical sensitivity, specificity, and accuracy in both sets ( Fig. 5B ). For adequate comparison, a new model was built using Orbitrap data from the same set of samples analyzed using the LIT (n = 40). High specificity was achieved in both the training (93.3%) and validation sets (100%) by the new Orbitrap model, whereas lower clinical sensitivity was observed for the training (86.7%) and validation (80.0%) sets, respectively, with 3 HGSC samples misclassified as normal. Note that these 3 samples had been correctly classified by the model built using Orbitrap data acquired from the larger training set (n = 60). A detailed comparison of the results obtained for the Orbitrap and ion trap models is provided in Table S7 in the online Data Supplement .

In total, 160 deidentified ovarian frozen tissue samples were obtained from the Cooperative Human Tissue Network (CHTN; Table 1 ). Samples were requested by the research team and collected from different tissue banks within the CHTN network in 3 different batches, the first in the fall of 2015, the second in the fall of 2017, and the third in the spring of 2018. Samples were selected following criteria based on tissue diagnosis (HGSC or LGSC) by gross anatomy and pathological evaluation, and specimen size (>200 mg). FT and peritoneum tissues (32 samples, 29 patients) were received as deidentified frozen specimens from the MD Anderson and CHTN Tissue Banks, or prospectively collected as fresh specimens during endometriosis surgery performed by MB (under approved IRB protocol#: 2017–08-0087) at the Seton Medical Center (Austin, TX; see Table S1 in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol65/issue5 . Patients undergoing endometriosis surgery were consented. Information on disease severity (other than grade for ovarian cancer) and alternative diagnoses were not considered in our study. Samples were analyzed at room temperature in random order. The analyzed area of the tissue was demarcated and registered through optical images. The same demarcated tissue or a parallel piece was frozen and sectioned at 10–16 μm using a CryoStar NX50 cryostat (Thermo Fisher Scientific), stained by standard hematoxylin and eosin procedure, and evaluated by expert pathologists (not blinded from previous clinical information) to confirm diagnosis of the analyzed area (authors J.L. and S.L.). Of note, the final diagnoses were performed after MasSpec Pen analyses ( Fig. 1 ). Only samples with clear diagnosis (n = 164) were used for statistical analysis (see Fig. S1 in the online Data Supplement ). The MasSpec Pen with a 2.7-mm pen tip diameter was used for analysis, using the same procedures and system previously described (see Methods in the online Data Supplement ) ( 21 ). The MasSpec Pen uses a water droplet to extract molecules from tissues upon contact, which are then analyzed by an Orbitrap mass spectrometer and statistical classifiers, resulting in a total analysis time of approximately 10 s. A new MasSpec Pen device was used for each of the tissue analyses described in this study to prevent any potential carryover between samples. Experiments were performed on a Q Exactive Orbitrap and an LTQ mass spectrometer (Thermo Fisher Scientific). Orbitrap analyses were performed from m/z 120–1800 at a resolving power of 140000 at m/z 200. Ion trap analyses were performed from m/z 120–1000. Tandem MS of selected ions was performed during MasSpec Pen analyses using high-energy collisional dissociation in the Q Exactive. Of note, the MasSpec Pen Orbitrap analysis of sample set 1 (n = 57) and the resulting data were previously described ( 21 ), whereas all the remaining samples and analyses were new to this study. Orbitrap and ion trap data were binned, normalized, and filtered before being subjected to lasso ( 34 ) (see Methods in the online Data Supplement ). Data from sample sets 1 and 2 were randomly split between training and validation sets. Classification performance was measured by sensitivity, specificity, and accuracy. The “pROC” R package was used to select an optimal threshold from the receiver operating characteristic curve ( 35 ). Cosine similarity analysis was performed in the CRAN R language using average mass spectra for each tissue type analyzed in each sample set.

Validation of the diagnostic capabilities of the MasSpec Pen is paramount for potential clinical translation. Utilizing a cohort of 138 histologically validated ovarian cancer and normal tissue samples, we demonstrate here the predictive power of the statistical models built from MasSpec Pen analyses. Machine learning and classification algorithms are not parametrically constrained and thus are more susceptible to overfitting than other more traditional methods ( 36 ). Overfitting occurs when a model uses spurious correlations within a single data set rather than relationships within the population. Here, statistical validation and evaluation of model overfitting were performed using training (to fit the model), validation (to provide an unbiased evaluation while tuning parameters such as data normalization methods), and test (to provide an unbiased evaluation of a final model) sets of samples. Improved performance was achieved for HGSC diagnosis using a new training set that combines samples from different batches, compared to what we have previously reported using a single sample batch ( 21 ). These results show that including samples from different batches in the training set helps improve the predictive power of the classifiers. High prediction accuracies were also observed in both the validation and test sets, indicating that the model was not overfitting to the training set. Similar results were obtained for the normal vs SC classifier, demonstrating that the MasSpec Pen and machine learning provide robust predictive models for ovarian cancer diagnosis. Discrimination between LGSC and HGSC tissue was also explored. Owing to the lower clinical occurrence of LGSC, sample availability was limited. Nevertheless, trends in lipid and metabolite composition between the 2 cancer types were and reflected by the moderate classification accuracy achieved. Collection of additional LGSC tissues is ongoing to continue MasSpec Pen validation for ovarian cancer subtyping. Four molecular models were also built to discriminate HGSC and SC from healthy FT and peritoneum samples, yielding 100% clinical specificity and sensitivity from 80.6% to 90.7% for cancer diagnosis. Assessment of the mass spectra from the cancer samples misclassified as FT revealed unusually high abundance of m/z 215.031, weighted toward FT classification, and an overall low lipid abundance in samples misclassified as peritoneum. Interestingly, 3 out of the 7 cancer tissues misclassified as peritoneum were obtained from ovarian cancer metastasis to the omentum. Segregation of primary or metastatic samples will be evaluated to investigate differences in molecular information that may be correlated to misclassification, as well as refinement of the statistical models. Nonetheless, the overall accuracies (>86%) achieved demonstrate the MasSpec Pen potential to differentiate SC from surrounding normal ovarian and abdominal healthy tissues. The features chosen by the lasso models as predictive for ovarian cancer diagnosis and subtyping were identified as biologically relevant species. For example, ascorbic acid, which has an important role in normal ovarian functions ( 37 ), was selected as important for normal ovarian tissue characterization, whereas GP lipids, such as PI 18:0_20:4, were at high relative abundance in cancer tissue and selected as predictive markers. Gluconate and glutamate were selected as important for the discrimination of LGSC and HGSC samples. Similar trends and species have also been reported using desorption electrospray ionization MS ( 26 ). Moreover, increased abundance of glutamate has been reported previously in invasive ovarian carcinomas compared to borderline tumors by gas chromatography and MS analyses ( 38 ). Another interesting feature selected was inosine at m/z 267.073, which was associated with FT tissue. Inosine is rapidly metabolized from adenosine, a nucleoside known to modulate neurotransmission and contractile responses in the FTs ( 39 ), and has been previously detected in oviduct cells ( 40 ). Smaller and lower-cost equipment could facilitate translation of MS technologies to the clinic ( 30 ). LIT mass spectrometers are attractive for clinical use due to their scalability, tolerance for higher operating pressure, and lower cost ( 31 ). However, LIT analyzers provide limited mass resolution, which could potentially compromise performance. To demonstrate that the MasSpec Pen can be compatible with a mass spectrometer platform other than an Orbitrap, and to further investigate how classification performance is affected by collecting data with lower performance mass analyzers, we coupled the MasSpec Pen to a LIT system. Using a smaller sample set than what collectively explored for Orbitrap analysis, we achieved 100% accuracy for HGSC classification in training and validation. The higher performance achieved compared to the Orbitrap data could be related to the absence of batch effects in the LIT data and will be further investigated. Overall, similarities between the detected metabolic profiles and statistical models observed between Orbitrap and LIT data support the validity and robustness of the predictive molecular species and further support that the MasSpec Pen provides consistent molecular analysis across MS systems. More testing of the LIT and other mass spectrometers is needed across multiple sample sets and cancer subtypes. In conclusion, our study demonstrates high performance and versatility of the MasSpec Pen technology and statistical models across different sample sets and MS platforms, as well as feasibility for cancer subtyping. The MasSpec Pen leverages on its nondestructive nature for direct and gentle analysis of tissues and the sensitivity, specificity, and speed provided by MS for untargeted and accurate molecular diagnosis. Further, its simple design, ease of operation, and biocompatibility are attractive for clinical use. Thus, we envision the MasSpec Pen to be employed in the operating room (OR) for in vivo and/or ex vivo use in conjunction with standard surgical tools. Yet, this study represents a first step toward validating the performance of the MasSpec Pen for ovarian cancer diagnosis. Several challenges still need to be addressed and further validation pursued to demonstrate the usefulness of the MasSpec Pen in its envisioned application in the OR. Although our experiments performed on ex vivo tissues obtained from tissue banks yield diagnostic molecular results, in vivo and freshly excised tissues in the OR may present slightly different molecular profiles. Moreover, extensive analyses of benign ovarian disease and tissues with varied histologic composition are needed to further demonstrate its value in more complex surgical pathology cases. As such, in vivo OR experiments utilizing the MasSpec Pen for ovarian cancer diagnosis are currently planned to further evaluate its diagnostic capabilities. Performance evaluation across different institutions, instrumentation, and users are also envisioned to further validate our results and evaluate its usefulness in patient care.