{"paper_id":"0956ca1d-d91a-41c5-b82b-7b6d262c7282","body_text":"Characterization of non-monotonic relationships\nbetween tumor mutational burden and clinical outcomes\nJordan Anaya1, Julia Kung 2, and Alexander S. Baras 1,3,4\n1Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA\n2Applied Health Sciences Informatics, Johns Hopkins University School of Medicine, Baltimore, MD, USA\n3The Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore,\nMD, USA\n4Bloomberg∼Kimmel Institute for Cancer Immunotherapy, Sidney Kimmel Comprehensive Cancer Center, Johns\nHopkins University School of Medicine, Baltimore, MD, USA\nCorresponding author:\nAlexander S. Baras1,3,4\nEmail address: baras@jhmi.edu\nABSTRACT\nPotential clinical biomarkers are often assessed with Cox regressions or their ability to differentiate two groups of\npatients based on a single cutoff. However, both of these approaches assume a monotonic relationship between the\npotential biomarker and survival. Tumor mutational burden (TMB) is currently being studied as a predictive biomarker for\nimmunotherapy, and a single cutoff is often used to divide patients. In this study we introduce a two-cutoff approach\nthat allows splitting of patients when a non-monotonic relationship is present, and explore the use of neural networks to\nmodel more complex relationships of TMB to outcome data. Using real-world data we find that while in most cases the\ntrue relationship between TMB and survival appears monotonic, that is not always the case and researchers should be\nmade aware of this possibility.\nINTRODUCTION\nWhen searching for features predictive of survival across different cancer types researchers often use Cox regression1–3.\nThe Cox model provides a high level of interpretability in the form of the regression coefficients, but these coefficients\nsimply describe the linear relationship of the feature to the predicted log partial hazard4. Often there isn’t a clear reason to\nassume a linear (and more importantly monotonic) relationship; furthermore, neural nets have previously been proposed as\na more flexible approach in this context5, with some recent applications appearing in the literature6–9.\nTumor mutational burden (TMB) is a commonly investigated biomarker in the context of immunotherapy10–16, and its\nprognostic value has also been investigated in the context of heterogeneous treatments17,18. When investigating TMB as a\nbiomarker researchers often bin patients into a “TMB low” group and a “TMB high” group, which implicitly assumes a\nmonotonic relationship of TMB with survival, independent of the number of bins used. In this case the relationship is\nassumed to be a step function with patients below a certain threshold having a certain risk and patients above the threshold\nhaving another. The monotonic assumption is that change in risk only increases (or only decreases) with the value of the\npredictor variable in question.\nSimilar to how we previously leveraged the flexibility of neural networks to more optimally model the calibration of\nnext-generation sequencing (NGS) gene panel-derived estimates of exomic TMB19, we wondered whether such flexible\nmodeling approaches could be applied to characterizing the relationship of TMB with clinical outcomes data. While a\nsingle cutoff approach may work well for monotonic relationships, it would be expected to be suboptimal in the case of a\ntruly non-montonic relationship. In such a scenario the lowest and highest TMB values would have similar risk, and the\nmoderate values would be associated with a different risk. In this study we explored different approaches to attempt to\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\nbetter characterize these more complex relationships and investigated whether such relationships exist in the context of\nTMB and clinical outcomes data, both in the prognostic sense and predictive sense.\nRESULTS\nGiven the long right-tailed distribution of TMB (Supplemental Figure 1A), when modeling the relationship of this data to\nthe log partial hazard some form of a transformation will generally be required, since these extreme values multiplied by\nthe estimated model coefficient will generate unrealistic partial hazards. We can demonstrate this by fitting a Cox model\nto untransformed TMB values for uterine corpus endometrial carcinoma (UCEC) and the corresponding survival data in\nthe The Cancer Genome Atlas (TCGA) dataset20. Generating predicted survival curves for several different TMB values\n(Supplemental Figure 1B), we see that what the field would consider large differences in TMB result in minimal differences\nin the curves, and at the highest TMB values patients are predicted to survive longer than humanly possible.\nApproaches with Simulated Data\nOne common approach for transforming TMB is to define a “TMB low” group and a “TMB high” group, which can be\ndescribed as transformation via a step function with a given cutoff. This cutoff can be defined by taking the median TMB\nvalue, or some other percentile of the data; alternatively, a cutoff can be found that optimizes some metric difference\nbetween the two groups. We demonstrate what the optimal cutoff approach looks like for simulated survival data with a\nmonotonic relationship (Figure 1A, B). In this simulated data the TMB values are real (the values of the UCEC TCGA\nsamples), but the survival data is generated according to a simple linear mapping of logged TMB values to log risks\n(representing a monotonic relationship). As expected, in every simulation a single cutoff approach correctly identified a\n“TMB high” group with a significant hazard ratio.\nWhile a single cutoff approach works well for this data, we wondered what would happen with a simple but non-\nmonotomic relationship. Further, we wanted to understand what approach could be used to identify this type of relationship.\nInstead of searching for a single cutoff we decided to search for two cutoffs, and merge the lowest and highest groups into a\nsingle group. Two cutoffs have been proposed before\n21, but in those contexts three groups were generated where the “TMB\nmid” group was expected to have a risk in between that of “TMB low” and “TMB high”, which retains the monotonic\nnature of the relationship. In contrast, in our two-cutoff approach we assign the same risk to the “TMB low” and “TMB\nhigh” groups with the “TMB mid” displaying a different survival risk, which represents a non-monotonic relationship that\nsome describe as a “Goldilocks effect”.\nLinear\nData Non-monotonic Data\nSimulated\nDataset Model LL score C-index LL score C-index\n1\nGround Truth -.895 .601 -.812 .696\nCox-PH -.895 .601 -.857 .669\nFCN -.896 .601 -.817 .689\n5 Ground Truth -.771 .617 -.879 .664\nCox-PH -.771 .617 -.915 .611\nFCN -.772 .617 -.879 .658\nTable 1. Simulated data metrics. Log-likelihood and C-indexes for the test folds of two representative monotonic and\nnon-monotonic simulated datasets for either a Cox model or a neural network.\nIn the case of a monotonic relationship, we would expect grouping the highest and lowest values together would result in\npoor correlation, and when we applied the new two-cutoff approach to the monotonic data the statistical significance was\noften lower than the single-cutoff approach (Figure 1C), with the “TMB mid” group often having the higher hazard ratio.\nHowever, in the case of a non-monotonic relationship, grouping the lowest and highest values is exactly what should be\ndone and when these two different approaches were applied to data simulating a non-monotonic relationship of TMB with\nsurvival we found the two-cutoff approach correctly associated a middle group with a larger hazard ratio, and typically did\nso with greater statistical significance than the single-cutoff approach (Figure 1D, E, F).\n2/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\n0 2 5 10 20 40 64\nTMB\n0.0\n0.5\n1.0\n1.5\n2.0Log Partial Hazard\n0 2 5 10 20 40 64\nTMB\n-1.0\n0.0\n1.0\n2.0\nLog Partial Hazard\nA\n0.125 0.25 0.5 1.0 2.0 4.0 8.0\nLL-test <= TMB\n34.4 1.58\n29.7 2.35\n28.7 3.36\n26.6 2.08\n14.1 1.65\n38.4 2.03\n30.0 2.38\n28.9 3.32\n18.9 5.53\n22.9 2.94\n23.3 1.65\n42.3 5.4\n23.7 2.23\n28.1 2.96\n35.4 2.5\n0.125 0.25 0.5 1.0 2.0 4.0 8.0\nLL-test <= TMB <=\n32.3 6.43 17.8\n49.4 4.72 30.05\n48.8 3.36 19.12\n38.5 4.7 18.51\n24.7 10.26 17.52\n62.7 2.5 24.38\n45.6 4.7 19.67\n48.9 3.32 24.88\n29.0 6.89 30.05\n38.1 9.47 28.85\n39.1 6.81 25.95\n69.4 5.4 20.55\n38.3 7.36 16.81\n50.2 6.81 28.87\n50.9 4.42 17.06\n0.125 0.25 0.5 1.0 2.0 4.0 8.0\nLL-test <= TMB\n31.7 17.78\n19.0 10.88\n13.1 26.47\n28.1 14.09\n28.4 10.33\n15.4 15.11\n31.6 3.45\n36.5 15.09\n23.3 10.21\n19.4 6.89\n15.9 21.79\n54.7 6.81\n17.1 9.78\n14.0 1.61\n25.1 10.39\n0.125 0.25 0.5 1.0 2.0 4.0 8.0\nLL-test <= TMB <=\n19.0 17.06 30.12\n14.2 10.88 28.85\n9.0 15.66 34.1\n22.0 14.09 24.88\n25.3 15.66 30.39\n13.9 0.92 10.21\n26.8 3.45 28.87\n36.1 15.09 31.39\n22.1 10.21 34.1\n22.2 9.1 19.11\n13.5 0.96 15.09\n43.0 6.81 30.12\n13.0 9.78 19.67\n16.2 1.23 1.45\n20.6 10.39 33.08\nB\nC\nD\nE\nF\nFigure 1. Cutoff analysis with simulated data. 15 simulated survival datasets were generated for either a monotonic relationship\nwith TMB (A) or a non-monotonic relationship (D). B and E show the hazard ratios and associated log-likelihood ratio tests and\nassociated cutoffs of searching for an optimal cutoff with either the monotonic relationship or non-monotonic relationship,\nrespectively, while C and F show the search for the optimal 2 cutoffs of our proposed approach with a monotonic or non-monotonic\nrelationship, respectively. In A, B the TMB distribution is shown as a rug plot.\nThese results give a potential framework for identifying non-monotonic relationships of an input variable to survival—if\nthe single-cutoff approach has a better test statistic then the relationship is possibly monotonic, and if the two-cutoff\napproach has a better test statistic the relationship is possibly non-monotonic. However, while we generally saw this\nexpected pattern in the simulated data it was not always the case, and this heuristic doesn’t reveal the true underlying\nrelationship, such as a linear relationship versus a step function or others. Instead of comparing different transformation\nstrategies (one cutoff versus two), we can simply allow a fully connected neural network (FCN) to learn the relationship\nbetween predictor variable and outcome measures directly from the available data.\nWe looked at two of the simulated survival datasets for both the linear and non-monotonic relationships, and compared\nthe results of a fully connected network (FCN) to a Cox regression implemented with lifelines. To compare the model fits\nwe looked at the C-index and log-likelihood (a larger value is better in both cases) for data in the test folds of 10 stratified\nK-folds, and also compared these metrics to what would be obtained with the ground truth relationship (Table1). With\nregards to the linear data both the Cox model and the FCN had metrics nearly indistinguishable from the ground truth, but\nfor the non-monotonic data the FCN had noticeably better metrics than the Cox model. Notably, in the first dataset for the\nnon-monotonic data the two-cutoff approach had previously obtained a lower test statistic than the single-cutoff approach\nand would have incorrectly inferred the relationship while the relationship is clear with an FCN.\nWe also visualized the model fits of the FCNs, Cox regressions, and the true underlying relationships (Figure 2). Despite\nhaving a large number of parameters our FCN produced a very similar fit to a Cox regression for the linear data, with both\n3/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\n0 2 5 10 20 40 64\nTMB\nLog Partial Hazard\nNon-monotonic Data\nTrue Cox FCN\n0 2 5 10 20 40 64\nTMB\nLog Partial Hazard\nNon-monotonic Data\nTrue Cox FCN\n0 2 5 10 20 40 64\nTMB\nLog Partial Hazard\nLinear Data\nTrue Cox FCN\n0 2 5 10 20 40 64\nTMB\nLog Partial Hazard\nLinear Data\nTrue Cox FCN\nA\nB\nC\nD\nFigure 2. Simulated data model fits. Model fits from a Cox model and a neural net were averaged over 10 K-folds for two\nrepresentative simulated survival datasets for a linear and non-monotonic relationship. A, B monotonic data. C, D non-monotonic data.\nTMB distributions shown as rug plots.\nclosely following the true relationship (Figure 2A, B). Visually, the extra parameters of the FCN allowed it to correctly\nfollow the shape of the non-monotonic data (Figure 2C, D), while the Cox model was a poor fit to the data, which is\nconsistent with the model metrics.\nApplying Neural Networks to the TCGA\nThe simulated data gave us confidence that if the relationship strongly deviates from a monotonic relationship, then an\nFCN will be able to better model this and this would be reflected in the C-index and model log-likelihood metrics. Our\nnext question was whether this tool would allow us to find evidence for such “Goldilocks effects” in real-world datasets.\nUsing the mutation call and survival data for solid tumors in the TCGA dataset, we compared the model metrics of a Cox\nmodel and our FCN (Table 2).\nIn most cancer types there was limited difference between a Cox model and an FCN, suggesting the relationship likely\nis monotonic (or in some cases simply no relationship). However, in SKCM the FCN displayed a noticeably better log\nlikelihood and C-index. SKCM is a cancer type where immunotherapy is recommended and where TMB hase been\nsuggested as predictive biomarker. Looking at some of the model fits we see tumor types where the FCN predicted a\nperfectly linear relationship and followed the Cox predictions, cases where the FCN has slight deviations from the Cox\npredictions, and then SKCM where the FCN predicts a strong concave up relationship (Figure 3).\nApplying Neural Networks to Immunotherapy Datasets\nThe TCGA dataset contains patients receiving the standard of care at the time, and includes patients at different stages\nof disease. In this heterogeneous context, exploring the relationship of TMB with survival is more closely associated\nwith some form of prognostication rather than predicting response to a given therapeutic modality. In contrast, when\ninvestigating a cohort in which a specific treatment has been administered we generally are considering a biomarker that\nis predictive of response. It is in this context that TMB has often been explored as a biomarker, and many approaches\neffectively attempt to stratify the cohort into “TMB low” and “TMB high” and compare clinical outcome measures\n4/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\nCox-PH\nFCN\nCancer\nLL score C-index LL score C-index\nBLCA\n-1.337 .573 -1.338 .570\nCESC -.593 .512 -.600 .490\nESCA -.842 .556 -.842 .556\nGBM -1.967 .491 -1.967 .478\nHNSC -1.381 .532 -1.379 .540\nKIRC -.643 .616 -.643 .616\nKIRP -.372 .573 -.372 .573\nLGG -.630 .712 -.625 .712\nLIHC -.953 .572 -.955 .572\nLUAD/LUSC -1.488 .514 -1.488 .514\nOV -1.415 .569 -1.416 .569\nPAAD -1.177 .551 -1.193 .553\nCOAD/READ -.670 .580 -.670 .580\nSKCM -1.346 .578 -1.322 .617\nSTAD -1.203 .559 -1.204 .559\nUCEC -.515 .551 -.521 .562\nTable 2. TCGA data metrics. Log-likelihood scores and C-indexes for the test folds of different cancers in the TCGA for a Cox\nmodel and a neural net.\n0 2 5 10 20 40\nTMB\nLog Partial Hazard\nESCA\nCox FCN\n0 2 5 10 18\nTMB\nLog Partial Hazard\nHNSC\nCox FCN\n0 1 2 3 4 5\nTMB\nLog Partial Hazard\nKIRC\nCox FCN\n0 1 2 3 4 5\nTMB\nLog Partial Hazard\nKIRP\nCox FCN\n0 2 5 10 20 40 100\nTMB\nLog Partial Hazard\nSKCM\nCox FCN\n0 2 5 10 20 40 100\nTMB\nLog Partial Hazard\nSTAD\nCox FCN\n0 2 5 10 20 40\nTMB\nLog Partial Hazard\nBLCA\nCox FCN\n0 2 5 10 20 40 80 160 256\nTMB\nLog Partial Hazard\nCOAD/READ\nCox FCN\n0 2 5 10 20 40\nTMB\nLog Partial Hazard\nLUAD/LUSC\nCox FCN\nFigure 3. TCGA model fits. Nine selected cancer types are shown. Cox and neural net model fits were averaged over 10 K-folds.\nTMB distributions shown as rug plots.\n(response, disease-free survival, etc). Given our findings of a non-monotonic relationship of survival for SKCM in the\nTCGA dataset we wondered if we would identify non-monotonic relationships in other datasets, either in a prognostic or\npredictive context.\n5/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\nThe AACR Project GENIE Biopharma Collaborative (BPC) recently released clinical data for lung and colon cancer\npatients. Using the corresponding panel mutational data for these patients we applied our model to a lung cohort treated\nwithout immunotherapy (BPC NSCLC NonIO), a lung cohort treated with immunotherapy (BPC NSCLC IO), and a colon\ncohort treated without immunotherapy (BPC colorectal NonIO). Similar to the TCGA data our neural network did not\ndetect non-monotonic relationships in these cancer types (Table 3, Supplemental Figure 2).\nCox-PH\nFCN\nData\nLL score C-index LL score C-index\nBPC\nNSCLC NonIO -1.183 .568 -1.183 .552\nBPC NSCLC IO -1.486 .532 -1.486 .531\nBPC Colorectal NonIO -1.645 .511 -1.643 .477\nMSK SKCM IO (2019 cohort) -1.026 .565 -1.028 .566\nMSK NSCLC IO (2019 cohort) -1.883 .527 -1.871 .576\nMSK NSCLC IO (2021 cohort) -2.043 .514 -2.037 .551\nMSK NSCLC NonIO (2021 cohort) -1.297 .610 -1.294 .610\nMSK Colorectal NonIO (2021 cohort) -1.001 .484 -0.995 .565\nMSK Pancreatic NonIO (2021 cohort) -2.044 .577 -2.044 .577\nMSK Endometrial NonIO (2021 cohort) -0.375 .648 -0.379 .649\nTable 3. BPC and MSK data metrics. Log-likelihood scores and C-indexes for the test folds of different cancers in the BPC and\nMSK data for a Cox model and a neural net.\nMemorial Sloan Kettering (MSK) has released several large datasets of patients assayed with their gene panel and the\ncorresponding clinical information. Without some form of common identifier across these datasets it’s difficult to know\nhow much patient overlap exists between the data releases, but we applied our FCN to a 2019 dataset which only included\npatients treated with immunotherapy15, and a 2021 dataset that had both IO naive and IO treated patients18. While we had\nobserved TMB to have a non-monotonic relationship with survival in the TCGA dataset for SKCM, in the IO treated MSK\nmelanoma cohort we did not observe a difference between a Cox regression and our FCN (Table 3). This is consistent\nwith a monotonic relationship of TMB and clinical outcome in the context of melanoma treated with IO, and supports\na TMB-high vs TMB-low stratification in this context. The FCN model showed better performance over Cox modeling\nin the non-IO treated colorectal cancer cohort (MSK Colorectal NonIO), in which a concave down model was apparent\n(Supplemental Figure 3). Some improvement in performance was seen in the MSK non-small cell lung cancer cohorts\ntreated with IO, and the observed relationship was also concave down (Supplemental Figure 3).\nModel Application\nThe benefit of directly modeling the relationship with survival is that any deviations from a linear fit will be accounted for\nin the model predictions. This allows a researcher wanting to identify a cutoff for splitting patients to simply use a cutoff\nbased on model risks. Then, working backwards from the model risks the corresponding input variable cutoff(s) can be\nfound. If the relationship of the input variable to survival is non-monotonic then a single model risk will result in two input\nvariable values, while a monotonic relationship will result in a single input variable cutoff.\nWe can demonstrate what this might look like by splitting each cancer in the TCGA dataset by either the median TMB\nvalue or median FCN model risk value. When splitting by median TMB the higher TMB group may or may not have a\nhigher hazard, while when splitting by median model risk the higher risk group should have a higher hazard. Ignoring this\ndifference in sign, in most cases there is almost no difference in the test statistic since the FCN predicted a monotonic\nrelationship for most cancers (Figure 4A, B). However, for SKCM we see that a single cutoff of TMB is inappropriate. In\nfact, with a median cutoff the relationship of TMB to survival is barely significant while it is highly significant with a FCN\nmodel output-based risk cutoff (Figure 4C, D).\nDiscussion\nIdentifying consistent cutoffs for biomarkers has always been challenging22 since only relationships that resemble a step\nfunction result in stable optimal cutoff values23, a result recapitulated in our Figure 1 and Supplemental Figure 4. It is\npossible to identify a step function with a neural net (Supplemental Table 1, Supplemental Figure 4), but here we used\n6/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\n0.125 0.25 0.5 1.0 2.0 4.0 8.0\nBLCA\nCESC\nCOAD/READ\nESCA\nGBM\nHNSC\nKIRC\nKIRP\nLGG\nLIHC\nLUAD/LUSC\nOV\nPAAD\nSKCM\nSTAD\nUCEC\nLL-test\n14.8\n0.0\n7.5\n3.5\n1.4\n2.7\n4.9\n1.4\n32.1\n1.3\n0.0\n11.5\n1.6\n16.3\n2.1\n8.1\n0.125 0.25 0.5 1.0 2.0 4.0 8.0\nBLCA\nCESC\nCOAD/READ\nESCA\nGBM\nHNSC\nKIRC\nKIRP\nLGG\nLIHC\nLUAD/LUSC\nOV\nPAAD\nSKCM\nSTAD\nUCEC\nLL-test\n15.4\n0.1\n7.5\n3.5\n1.2\n2.5\n4.9\n1.4\n32.1\n1.3\n0.0\n11.5\n1.3\n4.4\n2.1\n5.5\n0 2000 4000 6000 8000 10000 12000\nDays\n0\n20\n40\n60\n80\n100% Surviving\nLogrank p-value=3.7E-02\nSKCM\nLow TMB\nHigh TMB\n0 2000 4000 6000 8000 10000 12000\nDays\n0\n20\n40\n60\n80\n100% Surviving\nLogrank p-value=4.9E-05\nSKCM\nLow Risk\nHigh Risk\nA B\nC D\nFigure 4. Model effect on a binary label. Patients were split by either median TMB (A, C), or median risk from a neural net (B, D).\nA, C show the hazard ratios and log-likelihood tests of each cancer type for these splits while B, D show the Kaplan curves for SKCM\nwith these splits along with the logrank p-value.\nneural nets to investigate yet another consideration when identifying potential biomarkers and cutoffs: the assumption\nof a monotonic relationship may not hold. If this scenario occurs it has several important implications. First, when no\nrelationship is found with a Cox regression a very strong relationship could still be present. Second, the predictions for\nvalues at the extremes, i.e. patients who would be predicted to have the best or worst survival according to a Cox model,\nwill contain the largest errors (see the fits for SKCM in Figure 3). Third, if a non-monotonic relationship is present then a\nsingle cutoff is inappropriate and two-cutoffs of the input variable will be needed to split patients into the appropriate two\ngroups.\nThe possibility of a non-monotonic relationship is not just theoretical. While approaches to correlate TMB to survival\nhave generally assumed a monotonic relationship, using a more flexible approach we have identified multiple datasets\nwhere the relationship between TMB and survival appears more complex. This study has a few important limitations,\nthe first of which is that we did not undertake a more complex multivariate modeling of clinical outcome using features\nsuch as age, stage, grade, etc. This was done on purpose as the intent of this work is to highlight how the relationship of\nTMB to clinical outcome can be modeled to account for potential non-monotonic relationships and not to comprehensively\napproach clinical outcome modeling across the relevant predictors. The second important limitation is related in that we\ndid not design these experiments such that we would be “validating” a specific TMB↔ clinical outcome correlation across\ntumor type and clinical setting. Again, this was intended given the modeling focus of the work. We believe what we\nhave developed and presented herein has important implications for studies that investigate TMB as a predictive and/or\nprognostic biomarker. Further, we show that fairly simple neural networks like we have presented here can help avoid the\npitfalls of ordinary Cox-PH based regression with respect to the potential of non-monotonic relationships.\n7/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\nMethods\nCode for reproducing the results in this manuscript is available at https://github.com/OmnesRes/tmb survival and has been\narchived at Zenodo: https://zenodo.org/doi/10.5281/zenodo.10520419.\nModeling\nCox-PH models were implemented with lifelines version 0.27.724 and the neural nets with TensorFlow 2.1225. TMB values\nwere log transformed before inclusion into the models. The neural nets were designed to have two dense layers of 128\nwith softplus activation and a dropout of .05, followed by a dense to 1 with no activation. A batch normalization layer\nwas utilized to keep the output values centered at 0. This output was interpreted as the log risk and used to calculate the\nnegative partial likelihood, mimicking the loss of a Cox model.\nWhen training with the real-world datasets the TMB values in the top 1% were discarded to avoid training with extreme\nvalues. Stratified K-fold training was performed with 10 train/test splits and stratified by whether the TMB value was in the\ntop 20th percentile to ensure a somewhat consistent range of TMB values across training splits. For each fold the ranks of\nthe test fold data were recorded in order to calculate the concordance indexes of the models. To ensure a comparable loss\ncalculation between the neural net and Cox models lifelines was used to calculate all losses, with the predicted risks from\nthe neural nets being passed into a Cox model from lifelines.\nData Processing\nSimulated survival data was generated by utilizing an exponential distribution26, and a uniform distribution was used for\ncensoring with approximately 30% of the data censored. For the simulated data only TMB values below 64 were used\nsince it was difficult to prevent extreme simulated risks with our quadratic equation for the non-monotonic relationship,\nwhich highlights a potential issue of using polynomials for modeling data with a long tail.\nThe TCGA somatic mutation calls27 were processed as previously described to calculate exomic TMB19. Correspond-\ning clinical data were obtained from Liu. et al 20. We used all available panels in the BPC data except for VICC-01-\nSOLIDTUMOR and UHN-48-V1 due to their small sizes. We used GENIE release 14.128 to obtain somatic mutations and\npanel coordinates, and defined TMB as nonsynonymous mutations per Mb of panel coding sequence. We only included\npatients whose pathology procedure occurred within 180 days of diagnosis.\nAUTHORS’ DISCLOSURES\nThe authors have nothing to disclose.\nACKNOWLEDGMENTS\nThe results here are in whole or part based upon data generated by the TCGA Research Network. The authors would like to\nacknowledge the American Association for Cancer Research and its financial and material support in the development of the\nAACR Project GENIE registry, as well as members of the consortium for their commitment to data sharing. Interpretations\nare the responsibility of study authors.\nFUNDING\nThis research was supported in part by the Leon Troper Professorship in Computational Pathology at Johns Hopkins.\nREFERENCES\n1.\nGentles, A. J. et al. The prognostic landscape of genes and infiltrating immune cells across human cancers. Nature\nmedicine 21, 938–945 (2015).\n2. Anaya, J. OncoLnc: linking TCGA survival data to mRNAs, miRNAs, and lncRNAs. PeerJ Computer Science 2, e67\n(2016).\n3. Smith, J. C. & Sheltzer, J. M. Genome-wide identification and analysis of prognostic features in human cancers. Cell\nreports 38, 110569 (2022).\n8/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\n4. Cox, D. R. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological) 34,\n187–202 (1972).\n5. Faraggi, D. & Simon, R. A neural network model for survival data. Statistics in medicine 14, 73–82 (1995).\n6. Ching, T., Zhu, X. & Garmire, L. X. Cox-nnet: an artificial neural network method for prognosis prediction of\nhigh-throughput omics data. PLoS computational biology 14, e1006076 (2018).\n7. Katzman, J. L. et al. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep\nneural network. BMC medical research methodology 18, 1–12 (2018).\n8. Wang, D., Jing, Z., He, K. & Garmire, L. X. Cox-nnet v2. 0: improved neural-network-based survival prediction\nextended to large-scale EMR data. Bioinformatics 37, 2772–2774 (2021).\n9. Zhan, Z. et al. Two-stage Cox-nnet: biologically interpretable neural-network model for prognosis prediction and its\napplication in liver cancer survival using histopathology and transcriptomic data. NAR genomics and bioinformatics\n3, lqab015 (2021).\n10. Snyder, A. et al. Genetic Basis for Clinical Response to CTLA-4 Blockade in Melanoma. New England Journal of\nMedicine 371, 2189–2199 (2014).\n11. Allen, E. M. V .et al. Genomic correlates of response to CTLA-4 blockade in metastatic melanoma. Science 350,\n207–211 (2015).\n12. Rizvi, N. A. et al. Mutational landscape determines sensitivity to PD-1 blockade in non–small cell lung cancer.\nScience 348, 124–128 (2015).\n13. Hellmann, M. D. et al. Nivolumab plus ipilimumab in lung cancer with a high tumor mutational burden.New England\nJournal of Medicine 378, 2093–2104 (2018).\n14. Hellmann, M. D. et al. Tumor Mutational Burden and Efficacy of Nivolumab Monotherapy and in Combination with\nIpilimumab in Small-Cell Lung Cancer. Cancer Cell 33, 853–861.e4 (2018).\n15. Samstein, R. M. et al. Tumor mutational load predicts survival after immunotherapy across multiple cancer types.\nNature Genetics 51, 202–206 (2019).\n16. Chowell, D. et al. Improved prediction of immune checkpoint blockade efficacy across multiple cancer types. Nature\nBiotechnology (2021).\n17. Wu, H.-X. et al. Tumor mutational and indel burden: a systematic pan-cancer evaluation as prognostic biomarkers.\nAnnals of translational medicine 7 (2019).\n18. Valero, C. et al. The association between tumor mutational burden and prognosis is dependent on treatment context.\nNature genetics 53, 11–15 (2021).\n19. Anaya, J., Sidhom, J.-W., Cummings, C. A. & and, A. S. B. Probabilistic Mixture Models Improve Calibration of\nPanel-derived Tumor Mutational Burden in the Context of both Tumor-normal and Tumor-only Sequencing.Cancer\nResearch Communications 3, 501–509. https://doi.org/10.1158/2767-9764.crc-22-0339 (Mar. 2023).\n20. Liu, J. et al. An integrated TCGA pan-cancer clinical data resource to drive high-quality survival outcome analytics.\nCell 173, 400–416 (2018).\n21. Goodman, A. M. et al. Tumor mutational burden as an independent predictor of response to immunotherapy in\ndiverse cancers. Molecular Cancer Therapeutics 16, 2598–2608 (2017).\n22. Altman, D. G., Lausen, B., Sauerbrei, W. & Schumacher, M. Dangers of Using ”Optimal” Cutpoints in the Evaluation\nof Prognostic Factors. JNCI Journal of the National Cancer Institute 86, 829–835. https://doi.org/10.1093/jnci/86.11.\n829 (June 1994).\n23. Giannoni, A. et al. Do Optimal Prognostic Thresholds in Continuous Physiological Variables Really Exist? Analysis\nof Origin of Apparent Thresholds, with Systematic Review for Peak Oxygen Consumption, Ejection Fraction and\nBNP. PLoS ONE 9 (ed Lionetti, V .) e81699.https://doi.org/10.1371/journal.pone.0081699 (Jan. 2014).\n9/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint \n\n24. Davidson-Pilon, C. lifelines: survival analysis in Python. Journal of Open Source Software 4, 1317. https://doi.org/10.\n21105/joss.01317 (2019).\n25. Abadi, M. et al. TensorFlow, Large-scale machine learning on heterogeneous systems Nov. 2015.\n26. Bender, R., Augustin, T. & Blettner, M. Generating survival times to simulate Cox proportional hazards models.\nStatistics in Medicine 24, 1713–1723. ISSN : 1097-0258. http://dx.doi.org/10.1002/sim.2059 (May 2005).\n27. Ellrott, K. et al. Scalable open science approach for mutation calling of tumor exomes using multiple genomic\npipelines. Cell Systems 6, 271–281 (2018).\n28. AACR Project GENIE Consortium and others. AACR Project GENIE: powering precision medicine through an\ninternational consortium. Cancer Discovery 7, 818–831 (2017).\n10/10\n.CC-BY 4.0 International licenseavailable under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made \nThe copyright holder for this preprintthis version posted January 20, 2024. ; https://doi.org/10.1101/2024.01.16.575937doi: bioRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}