{"paper_id":"44aa2ab5-2c78-4472-ae01-b4f0280b26b9","body_text":"Predicting cause of death from free-text health summaries: \ndevelopment of an interpretable machine learning tool \nChris McWilliams1¥, Eleanor I. Walsh2¥*, Avon Huxor3, Emma L. Turner2, Raul Santos-1 \nRodriguez1 2 \n1School of Computer Science, Electrical and Electronic Engineering, and Engineering Mathematics, 3 \nFaculty of Engineering, University of Bristol, Bristol, UK 4 \n2Bristol Medical School – Population Health Sciences, Faculty of Health Sciences, University of 5 \nBristol, Bristol, UK 6 \n3School of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK 7 \n¥ co-lead authors 8 \n* Correspondence:  9 \nCorresponding Author 10 \ne.walsh@bristol.ac.uk 11 \nKeywords: machine learning, free-text, accountability, transparency, prostate cancer, medical 12 \nrecords 13 \nStructured Abstract 14 \nPurpose: Accurately assigning cause of death is vital to understanding health outcomes in the 15 \npopulation and improving health care provision. Cancer-specific cause of death is a key outcome in 16 \nclinical trials, but assignment of cause of death from death certification is prone to misattribution, 17 \ntherefore can have an impact on cancer-specific trial mortality outcome measures.   18 \nMethods: We developed an interpretable machine learning classifier to predict prostate cancer death 19 \nfrom free-text summaries of medical history for prostate cancer patients (CAP). We developed 20 \nvisualisations to highlight the predictive elements of the free-text summaries. These were used by the 21 \nproject analysts to gain an insight of how the predictions were made. 22 \nResults: Compared to independent human expert assignment, the classifier showed >90% accuracy 23 \nin predicting prostate cancer death in test subset of the CAP dataset. Informal feedback suggested 24 \nthat these visualisations would require adaptation to be useful to clinical experts when assessing the 25 \nappropriateness of these ML predictions in a clinical setting. Notably, key features used by the 26 \nclassifier to predict prostate cancer death and emphasised in the visualisations, were considered to be 27 \nclinically important signs of progressing prostate cancer based on prior knowledge of the dataset. 28 \nConclusion: The results suggest that our interpretability approach improve analyst confidence in the 29 \ntool, and reveal how the approach could be developed to produce a decision-support tool that would 30 \nbe useful to health care reviewers. As such, we have published the code on GitHub to allow others to 31 \napply our methodology to their data (https://zenodo.org/badge/latestdoi/294910364). 32 \n 33 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \nNOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.\n\nFree-text cause of death predictions \n 2 This is a provisional file, not the final typeset article \nDeclarations 34 \nFunding: This work was funded by the Jean Golding Institute Seed Corn Fund 2019-2020.  The CAP 35 \ntrial was funded by grants from Cancer Research UK (C11043/A4286, C18281/A8145, 36 \nC18281/A11326, C18281/A15064, C18281/A24432). The UK Department of Health National 37 \nInstitute of Health Research provided partial funding. CM is funded by the HDR UK South West 38 \nBetter Care Partnership. RSR is supported by the UKRI Turing AI Fellowship EP/V024817/1. 39 \nConflict of Interest: The authors declare that the research was conducted in the absence of any 40 \ncommercial or financial relationships that could be construed as a potential conflict of interest. 41 \nAvailability of data: The dataset analysed in this studied cannot be shared publicly for reasons of 42 \ndata protection. 43 \nCode availability: Code is available at https://zenodo.org/badge/latestdoi/294910364 44 \nAuthor Contributions: CM and AH conducted data analysis and modelling. EW, AH, ET and RS 45 \ndeveloped the study concept and design. EW and ET contributed domain expertise and detailed 46 \nunderstanding of the dataset.  RS contributed expertise in interpretable machine learning. EW and 47 \nCM wrote the manuscript and all authors contributed edits and revisions.  48 \n 49 \nAcknowledgments: We acknowledge the contribution of the CAP trial group, including the 50 \nPrinciple Investigators Professor Richard Martin (Lead), Professor Jenny Donovan, David Neal and 51 \nFreddie Hamdy. A special thanks to the Cause of Death Evaluation Committee for reviewing the 52 \nsummaries written by the fieldworkers Naomi Williams, Siaw Yein Ng, Laura Hughes, Elizabeth 53 \nHill, Charlotte Davies, Liz Salter, Jainnee Mauree, Mari-Anne Rowlands, Leah Bowen, Sean 54 \nHarrison, Pete Holding and Lindsey Bell. And thank you to Tyler Seibert, Assistant Professor and 55 \nRadiologist at University California San Diego for his insights into the data visualisation outputs 56 \nfrom this project. The CAP trial recognises that this work uses data provided by patients and 57 \ncollected by the NHS as part of their care and support. We also acknowledge the work of the NHS 58 \nDigital Organization and Office of National Statistics for their assistance with this study.  59 \nEthics statement: This work was completed within the ethical approval already held CAP project 60 \n(Derby National Research Ethics Service Committee East Midlands & NHS Health Research 61 \nAuthority Confidential Advisory Group (CAG) approvals). In the case of the CAP trial, the 62 \nfieldworkers are highly trained to include minimal identifiable information (e.g. they exclude person 63 \nnames, hospital identifiers, full dates etc.). A derived extract of the, already pseudonymised, CAP 64 \nsummaries were then analysed. 65 \nConsent to participate: Not applicable. 66 \nConsent to publish: Not applicable. 67 \n  68 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n3 \n1. Introduction 69 \nFree-text electronic health records (hereafter, health records) contain information about a patient’s 70 \nmedical history. In health care, it is common for human experts to review health records to inform 71 \ndecision making and clinical practice. This review process can be time consuming and prone to error 72 \nand there is considerable potential for algorithmic methods to support human decision-making in this 73 \ncontext (1). One example of this process is retrospective auditing of health care practice, which often 74 \nrequires trained experts to manually review and assign outcomes or labels to the cases reviewed. The 75 \nhigh cost in person-hours can make such studies prohibitively expensive, including clinical trials, 76 \nwhich are essential for improving the delivery of patient care.   77 \nAs one of the most common cancers diagnosed in the UK, it is not surprising that prostate cancer is 78 \none of the cancers on which text mining of health records for research has focused (2). Much of this 79 \nwork has concentrated on predicting cancer detection (3), disease progression (4) and optimizing 80 \ntreatment (5). Knowledge of underlying cause of death is another key health outcome and machine 81 \nlearning techniques have made it possible to extract cancer mortality directly from Medical 82 \nCertificates of Cause of Death (MCCD)(6). But these methods have not taken into account the 83 \ninherent misattribution that exists within the death certificate. For example, prostate cancer deaths 84 \ncan be misattributed to other causes which result in underestimates of prostate cancer as a cause of 85 \ndeath. This is the case when deaths are attributed to the complications from investigations or 86 \ntreatment of prostate cancer, rather than the disease itself (e.g. infection from biopsy, post-surgery 87 \ncomplications). Deaths from other causes can also be attributed to prostate cancer which would cause 88 \nan over-estimate of prostate cancer as cause of death (7). This was demonstrated in the Cluster 89 \nrandomised trial of PSA testing for prostate cancer (CAP) where an independent committee assigned 90 \ncause of death, finding that death certification produced false positive prostate cancer deaths 9% of 91 \nthe time.  This increased to 23% if the individual had another cancer (not prostate cancer) diagnosed 92 \nduring their lifetime (8). It led to a recommendation that assignment of prostate cancer death, 93 \nespecially as an outcome in trials research, should be confirmed by an independent expert committee 94 \n(8).  In the CAP trial, semi-structured free-text summaries of a patient’s medical history from 95 \nhospital records are created by trained fieldworkers, which are then reviewed by an independent 96 \ncommittee of experts and assigned as either prostate cancer related death or not. The CAP dataset 97 \nthus provides a binary classification task – the identification of prostate cancer death – for which 98 \nmachine learning algorithms can be trained on the annotation of human experts. 99 \nThere have been significant advances in the field of text mining in recent years, with general purpose 100 \ndeep neural network models such as BERT (9) achieving state-of-the art performance on multiple 101 \nnatural language processing (NLP) tasks, such as question answering and named entity recognition. 102 \nApplying such NLP methodologies to clinical text data presents various challenges (10) and perhaps 103 \nthe most significant is the shift in word distributions as compared to the standard corpora on which 104 \nmodels are trained. This challenge can be overcome by training on general biomedical corpora (11) 105 \nand/or by developing task-specific models. For example, the state-of-the-art in the detection of 106 \nmedical concepts, such as ICD-10 codes, is to use task-specific recurrent neural network architectures 107 \n(12,13) which are pre-trained on a medical corpus such as MIMIC (14). However, on specific tasks, 108 \nclassical NLP approaches are still able to compete with deep learning methods (15–17).  109 \nThis work addresses the task of document classification.  An extensive review of this topic, in 110 \nrelation to clinical text data, is provided by Mujtaba et al (10). Specifically, we aim to train a binary 111 \nclassifier on human expert annotations in order to identify patients that died from prostate cancer 112 \nusing the CAP health records described above. Although optimal performance in such tasks is likely 113 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n 4 This is a provisional file, not the final typeset article \nto be achieved by deep learning models, we are more concerned here with the interpretability of the 114 \nclassifier in the context of decision-support. Interpretability is key to the safe and effective 115 \ndeployment of machine learning (ML) in contexts where there is a human-in-the-loop (18). The 116 \nrelated concepts of fairness (19), accountability (20) and transparency (21) have become key tenets 117 \nof ethical artificial intelligence and are nowhere more relevant than in the medical domain. Human 118 \nusers of an ML algorithm must be able to engage with and understand its predictions. Crucially, the 119 \nuser must have the confidence to either accept or reject the predictions based on their clinical 120 \nexpertise and a clear understanding of how the predictions were made. 121 \nIn this study we developed graphical methods to explain which textual elements contribute to the 122 \nclassification of a health records from CAP by a machine learning algorithm. We also used methods 123 \nfrom the FAT Forensics toolbox (22)  and the TreeInterpreter Python package (23) to quantify feature 124 \ncontributions. These contributions were then displayed to the user in an interpretable and visually 125 \nengaging format. The classifiers and interpretability methods were developed for the CAP dataset, 126 \nusing expert committee assignment of prostate cancer-related death to train the classifiers. However, 127 \nthe intention is that these methods will be developed into a decision-support tool that would help 128 \nhuman experts in the time-consuming task of classification of health records, both for this specific 129 \ntask and for similar tasks with different data sources.  130 \n2. Materials and Methods 131 \nThe machine learning classifier was developed using the CAP dataset, following which we 132 \ninvestigated a variety of visualization techniques to aid interpretation of the ML predictions. The 133 \nCAP dataset is outlined below (2.1). All code was written in Python and has been released publicly at 134 \nhttps://zenodo.org/badge/latestdoi/294910364 to facilitate re-use by other groups working in this 135 \narea.  136 \n2.1 CAP medical history summaries 137 \nThe CAP trial is a cluster-randomised control trial (RCT) that aims to investigate the effectiveness of 138 \nscreening for prostate cancer in the UK population, which has been running since 2001. The CAP 139 \ndataset used in this study contains 2,606 medical summaries of men who had a diagnosis of prostate 140 \ncancer, registered with the UK National Cancer Registrations Service in England and Wales, and 141 \nwho had died by 31st March 2016 (see Table 1 for characteristics). Deaths were notified by the Office 142 \nof National Statistics who provide death certification information via NHS Digital.  143 \nThe medical summaries were written by 13 trained fieldworkers who extracted prostate cancer and 144 \nend of life information directly from the medical records in hospitals. The semi-structured summaries 145 \nof the patients’ medical history include information about prostate cancer severity, treatments, and 146 \nprogression, as well as co-morbidities and competing causes of death. The full list of text fields 147 \npresent in this data are given in table ST1 in Online Resource 1. 148 \nThe medical summaries have been reviewed by an independent cause of death evaluation committee 149 \nmade up of clinical experts (i.e. urological surgeons, pathologists and oncologists) who decide 150 \nwhether the cause of death is due to prostate cancer for each case, or whether the patient died ‘with’ 151 \nprostate cancer and not ‘from’ prostate cancer. This human made decision was used as the ground 152 \ntruth against which to train the classifier.    153 \nTable 1. Characteristics of prostate cancer patients in the CAP dataset \nCharacteristic Prostate cancer dataset (CAP) \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n5 \nNo. of patients 2,606 \nSex M: 2,606 (100%); F: N/A \nAge at diagnosis (mean) 67.5 years (IQR: 64.1-71.3) \nAge at death (mean) 70.2 years (IQR: 66.5-74.5) \nYear of death (range) 2003 - 2016 \nPrimary outcome Prostate cancer related death \nYes: 1,095 (42%); No: 1,511 (58%) \nNote types Oncology and urology notes, inpatient notes, outpatient notes, \nradiology notes, multidisciplinary team meeting notes, lab reports, \noperative reports \n 154 \n2.2 Machine learning pipeline 155 \nWe used a bag-of-words representation of the health records which was produced as follows. The 156 \nsemi-structured text fields were concatenated into a single string and then lemmatized using the 157 \nWordNetLemmatizer from Python’s Natural Language Toolkit (NLTK). The full details of the 158 \nlemmatization are provided in Algorithm A1 in Online Resource 1. We then extracted features from 159 \nthe lemmatized text using CountVectorizer and TfidfTransformer from Scikit-learn. The 160 \nhyperparameters of CountVectorizer were optimized using GridSearchCV along with those of the 161 \nclassifier algorithm. The classifier was trained on 80% of the dataset with 20% held out for testing. 162 \nFollowing optimisation of the hyperparameters with 5-fold cross validation, the best classifier was 163 \nrefitted to the full training data.  Full details of the feature extraction and model training are provided 164 \nin Algorithm A2 in Online Resource 1. During model development we tested three classifier 165 \nalgorithms from scikit-learn. However, the code can easily be adapted to use any off-the-shelf or 166 \nbespoke classifier. We evaluated classifier performance using a suite of metrics, including accuracy 167 \nand AUC (area under the curve).  168 \n2.3 Interpretability methods 169 \nOn top of the classification pipeline, we developed interpretable outputs that allow users to engage 170 \nwith the classifier’s prediction and which are intended to act as a proof-of-concept for a future 171 \ndecision support tool based on this work. To achieve interpretability, we focused on the 172 \ncommunication of feature importance. We intended to convey an intuition for how the classifier 173 \nworks and to allow users to consider whether they agree with individual predictions based on how 174 \nthat prediction was made. In other words, we introduced the key elements of accountability (20) and 175 \ntrust (18) to the classification system. We used word clouds to visually represent the relative 176 \nimportance of the features (words and bigrams) by scaling their size. The colours of the words were 177 \nused to display the sign of the contribution of the feature towards the class prediction (where this 178 \ninformation was available). We extended this visual representation by producing augmented versions 179 \nof the original free-text summaries that displayed feature contributions in the context of the original 180 \nlanguage used.  181 \nTo quantify feature importance, we used four different approaches. Two of these methods are 182 \nspecific to the tree-based classifiers, while two are generally applicable and can be used with any 183 \nmachine learning classifier. The four methods are as follows: 184 \n1. Gini Importance – Is calculated as the normalised total reduction in the Gini impurity 185 \nbrought about by splits on a given feature across the ensemble of trees (24). This metric is 186 \npart of the scikit-learn implementation of the Random Forest classifier and only provides an 187 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n 6 This is a provisional file, not the final typeset article \naggregate measure of importance (i.e. at the level of the whole dataset). It also does not 188 \ncapture the sign of the contribution of the feature.      189 \n2. TreeInterpreter – This package (23) computes feature importance by decomposing a single 190 \nprediction into a sum over feature contributions. It captures both the sign and magnitude of 191 \neach feature contribution to a single prediction and can be averaged over many predictions to 192 \nproduce an aggregate measure of importance.   193 \n3. LIME – Local Interpretable Model-agnostic Explanations (25) are a standard tool of 194 \nexplainable AI and are implemented in (22). They provide a local measure of feature 195 \ncontribution that can be used with any machine learning classifier.  196 \n4. SHAP - SHapley Additive exPlanations (26) is a model agnostic explainer based on game 197 \ntheoretic methods that also provides a local measure of feature importance. 198 \n3. Results 199 \nA random forest classifier proved to be the best performing algorithm for the task of predicting 200 \nprostate cancer death according to accuracy and F1 score. This model achieved a classification 201 \naccuracy of 91.8% and an F1 of 0.931 on the test data. (Its cross-validation performance is illustrated 202 \nin figure S0 in Online Resource 1.) A support vector classifier and linear regression classifier 203 \nachieved comparable levels of performance (see Table 2 and Figure 1). We attempted recalibration of 204 \nthe random forest classifier output using both isotonic and sigmoid (27) methods (see Figure S1 in 205 \nOnline Resource 1) but both methods made only minor improvements to the probabilities output. As 206 \nsuch, the main results presented here are for the uncalibrated classifiers. We hypothesized that the 207 \ncases represented in the CAP dataset exhibited different degrees of classification difficulty, 208 \ncorresponding to the method with which the cases had been originally labelled with a cause of death. 209 \nThese different methods are collectively referred to as the “cause of death route” (COD) and are 210 \nexplained in Algorithm A3 in Online Resource 1. Using the COD route, we stratified the dataset into 211 \nhard and easy cases. We found that the classifier performance was worse for the hard cases than the 212 \neasy cases (Figure S2 in Online Resource 1) and that the probabilities were better calibrated for the 213 \neasy cases (Figure S3 in Online Resource 1). For hard cases, the classifiers tended to underestimate 214 \nthe probability of prostate cancer death towards the lower end of the range. This implies the existence 215 \nof patients who actually died of prostate cancer, but which look to the classifier like they did not. 216 \nThese results suggest that the stratification of cases based on COD route is meaningful and may have 217 \nsignificant implication for how a decision support tool could be used and evaluated in the future. 218 \nHowever, it should be noted that these groups are imbalanced with 2340 and 270 easy and hard cases 219 \nappearing in the dataset, respectively. We investigated the effect of authorship and found evidence of 220 \nclustering based on language style (Figure S4 in Online Resource 1) but this did not significantly 221 \naffect performance (see section SM1.1 in Online Resource 1). 222 \nTable 2. Performance summary of the three classifiers. SVC = Support Vector Classifier; LR = 223 \nLogistic Regression; RF = Random Forest. Bold face indicates the best performance for each metric.  224 \nClassifier AUC  Accuracy Precision Recall F1 Specificity \nSVC 0.9749 0.9157 0.9379 0.9199 0.9288 0.9095 \nLR 0.9733 0.9023 0.9196 0.9167 0.9181 0.8810 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n7 \nRF 0.9731 0.9176 0.9325 0.9295 0.9310 0.9000 \n 225 \nThe random forest was chosen to explore interpretability based on its performance. This choice also 226 \nallowed us to explore feature importance metrics that are only applicable to tree-based algorithms. 227 \nWe compared feature rankings obtained using the four metrics for feature importance using 228 \nSpearman’s rank correlation coefficient (see table ST2 in Online Resource 1). The rankings 229 \naccording to SHAP, LIME and Gini importance were all moderately to strongly correlated with the 230 \nTreeInterpreter rankings (ρ = 0.54 - 0.65). Here we present interpretability results using the 231 \nTreeInterpreter metric for feature importance, but the equivalent outputs can easily be produced using 232 \nthe alternative metrics (see for example, Figures S5 and S6 in Online Resource 1).    233 \nWe produced word clouds to illustrate the most important features that contributed to the classifier 234 \npredictions. These word clouds were shared with members of the CAP study team, who confirmed 235 \nthat the classifier was using clinically meaningful information to make predictions. Features with 236 \nlarge contributions (such as “bone scan”, “spine”, “hormone”, “androgen”) tend to be associated with 237 \nadvanced stage prostate cancer. These features can contribute positively or negatively to the 238 \nclassification depending on the frequency of occurrence of the term in the health record. For 239 \nexample, both “hormone” and “bone scan” contribute positively to the classification of prostate 240 \ncancer death when present in individual cases (see Figure 2(B)) but when averaged across the dataset 241 \nthey are indicative of non-prostate cancer death (see Figure 2(A)). 242 \nTherefore, we felt it was necessary to see these feature contributions in the context of the original 243 \ntext, to determine if the classifier is correctly identifying textual elements that indicate prostate 244 \ncancer death. For this reason, we sought a format that would allow users to engage with the classifier 245 \noutput and could potentially be used for decision support. The solution, arrived through dialogue with 246 \nmembers of the CAP project, was to produce augmented versions of the original health records which 247 \nwe refer to as interpretable vignettes. A partial view of one of the interpretable vignettes is shown in 248 \nfigure 3 and full examples are provided in Online Resources 2 and 3. The text uses the same 249 \nformatting as the word clouds to show feature contributions. Here the reader can see the context in 250 \nwhich the features appear and can therefore use their judgement to determine if the classifier is using 251 \nthe textual element in a meaningful way towards its prediction (28). A legend is provided so that 252 \nreaders can interpret the relative feature contribution sizes and summary information is provided in 253 \nthe header of the vignette.   254 \n4. Discussion 255 \nFor a machine learning tool to be useful, especially in the medical domain, it is essential for the user 256 \nto be able to interpret its output. As such, it is common for studies about clinical decision support or 257 \nprediction modelling to include a discussion of interpretability (29–31). However, as Lipton points 258 \nout, “interpretability is not a monolithic concept” (32) – it includes distinct yet intersecting ideas such 259 \nas comprehension, transparency and trust. It is also subjective, in that what one user may find 260 \ninterpretable another user may not. Most importantly, what the data scientist may consider to be 261 \ninterpretable is not necessarily of use to a clinical decision maker. In this study we used four standard 262 \ninterpretability metrics to produce measures of feature importance from a machine learning classifier 263 \nthat was trained to identify prostate cancer deaths from medical summaries. We then developed 264 \nvisual representations of these feature importances and presented them to one of the trained CAP 265 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n 8 This is a provisional file, not the final typeset article \nreviewers, who would be the intended users of a future decision support system based on this work. 266 \nTheir feedback is summarised below and used to highlight the strengths and weaknesses of our 267 \napproach, and to identify directions for future work.    268 \nThe word clouds of feature contributions were intended to provide a high-level overview of what the 269 \nclassifier learned from the data. The reviewer showed a preference for an alternative presentation of 270 \nthis information, stating that “they may be a reasonable way to display the model weights, but I 271 \nprobably would better understand a sorted list in tabular form with associated weight magnitudes”. 272 \nThe augmented vignettes (Online Resources 2 and 3) proved to be more useful. These allowed the 273 \nreviewer to engage with individual predictions, by highlighting features in the original text, using 274 \nfont colour and size to indicate the direction and magnitude of the contribution to the prediction. 275 \nSimilar visualisations have been used in other ML studies to provide interpretability (28,33–35). 276 \nBeing able to see the feature contributions in the semantic context of the original text enabled the 277 \nreviewer to determine where the classifier was correctly or incorrectly using features. In general, the 278 \nreviewer felt that the highlighted feature contributions were consistent with their clinical assessment: 279 \n“Many of the colors make sense here (in Online Resource 2). Metastases would be consistent 280 \nwith prostate cancer death…Meanwhile, mentions of “lungs”, “ascites”, “stomach”, and 281 \n“thorax” in the vignette suggest the patient has some non-prostate-cancer condition that is 282 \nworthy of attention—those are appropriately yellow.” 283 \n 284 \nThe features appearing in Online Resources 2 and 3 which were associated with  prostate cancer 285 \ndeath, aligned with the indicators of advancing disease (e.g. bony metastases, hormone treatment) 286 \nthat are used as clinical outcomes in prostate cancer trials (36). However, the feedback made it clear 287 \nthat the visualisations were less interpretable than intended: 288 \n“I am confused, again, that ‘bone scan’ and ‘hormone’ are blue here (in Online Resource 3) 289 \nbut were each yellow elsewhere (bone scan was yellow for Online Resource 2 and in Figure 290 \n2A; hormone was yellow in Figure 2A).” 291 \nHere the reviewer is referring to the ability of a feature (e.g. ‘bone scan’) to contribute positively to a 292 \nclassification of prostate cancer death when present in the text, but to contribute negatively to the 293 \nclassification when it is absent. This is an example of the potential for conflict, referred to by Lipton 294 \n(32), between what is a transparent and faithful representation of the mechanism of a classifier and 295 \nwhat is easily understandable by a human user. In this case, the problem might be overcome either by 296 \nincluding some indication of the actual feature value or by providing some training to the user to 297 \nresolve the apparent inconsistency.       298 \n 299 \nThe augmented vignettes let the user see if elements of text are missed or used incorrectly by the 300 \nclassifier, in which case they can exercise caution when considering the prediction. In this way the 301 \nreviewer determined: 302 \n“that some of the descriptions I pay most attention to (the rising PSA values and, to a lesser 303 \nextent, the high Gleason score) are gray—presumably because the algorithm is ignoring 304 \nthem.” 305 \nBoth PSA and Gleason score are numerical values which are often included in these medical 306 \nsummaries, but which are not captured by our current feature representation. This feedback suggests 307 \nthat an avenue for improved performance would be to incorporate prior clinical knowledge such as 308 \nthe importance of these two scores. Interestingly, it may also improve trust in the system if users 309 \ncould see that the classifier was making use of the elements of the medical summaries that they 310 \nconsider to be most important. The interpretable vignettes also revealed that classification of prostate 311 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n9 \ncancer death was problematic when negation appeared in the text. Our bag-of-words feature 312 \nrepresentation would not be expected to handle negation, so the application of methods to detect 313 \nnegation in clinical text data (37,38) would likely boost performance. Off-the-shelf classifiers 314 \nachieved good performance on the CAP dataset. For different health record datasets, additional effort 315 \nmay be required to achieve sufficient performance for a decision support tool to be useful. The 316 \nclustering of the health records based on authorship suggested that methods such as multiple-source 317 \ncross-validation (39) or domain adaptation (40) could be beneficial in dealing with differences in 318 \nwriting styles within other datasets. Other methods to boost performance would likely be task- or 319 \ndomain-specific and could include the addition of numeric clinical features extracted from structured 320 \ndata (41), or the use of state-of-the-art deep learning methods (12,37). Such methods would be 321 \ncompatible with our model-agnostic approach to interpretability.  322 \nThe CAP dataset contains a proxy for ‘difficulty’ of the cause of death assignment. Although we 323 \nwere not able to train a model to reliability predict hard cases, our cause-of-death-classifier did show 324 \nworse performance and calibration on the hard cases than on the easy cases. This suggests that the 325 \nstratification of cases according to difficulty is meaningful and is likely to have implications for the 326 \nfuture development and evaluation of a decision support tool. A systematic investigation of what 327 \nmakes the hard cases more difficult to classify, and which features are most predictive for different 328 \ntypes of cases, will help to inform more targeted data acquisition from hospital records. Named-entity 329 \nrecognition approaches could also be adapted to assist with this information retrieval (12,15). Such 330 \nknowledge could produce significant cost savings in data collection for CAP and similar projects. In 331 \npractice, the predictions for hard cases are less trustworthy and one way to address this would be to 332 \nproduce reliable estimates of uncertainty (42). In an applied setting it would be important for the 333 \ntransparency of the system to communicate to users the relative risks of both false positive and false 334 \nnegatives.        335 \nThe feedback of the CAP reviewer has given us confidence in the feasibility of these methods, and 336 \nthe next stage is to develop them into a usable decision support tool, following a user-centric design 337 \nprocess with members of the intended user group (43). Key to this will be to adapt the visualisations 338 \nto be appropriate for users in a clinical setting. We will need to test our classifiers on new CAP 339 \nreviews to determine how well they generalise to unseen data. Our bag-of-words approach is limited 340 \nby the size of the training data. There are 1360 words in the test data set that do not appear in our 341 \ntraining data (Figure S7 in Online Resource 1), and the CAP dataset has only limited overlap (Figure 342 \nS8 in Online Resource 1) with an example biomedical corpus (44). To optimise classifier 343 \nperformance in the future will likely require an adapted pre-trained deep learning model (11). We 344 \nhave identified benchmarking datasets (45) that would allow comparison of different classification 345 \napproaches to ensure that the best model can be selected. It is clear from our results that the different 346 \napproaches to quantifying feature importance produce distinct feature rankings. Choosing the best 347 \napproaches to ensure user trust and system transparency will be achieved using A/B testing across a 348 \nrange of users. The continued use of model-agnostic explainability methods will allow abstraction of 349 \nthe decision support interface from the underlying classifier and would allow the tool usable across a 350 \nrange of different tasks and datasets. For example, we plan to test our approach to interpretable 351 \ndocument classification on an intensive care dataset (46) that contains free-text medical notes that are 352 \nroutinely used by hospital staff to audit clinical practice. 353 \n5. Conclusion 354 \nAlgorithmic classification of health records, such as the identification of prostate cancer death in the 355 \nCAP dataset, could reduce the need for complex medical summaries to be reviewed by an 356 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n 10 This is a provisional file, not the final typeset article \nindependent committee. We have demonstrated use of visual methods to explain classifier 357 \npredictions to human users, which could be deployed in a future decision support tool to reduce the 358 \ncognitive burden on individual reviewers. Knowledge of the predictive features could also be used to 359 \ntarget data extraction from hospitals, reducing the workload and cost required in creating the free-text 360 \nsummaries. We encourage researchers to take a user-centric approach when developing interpretable 361 \nmachine learning tools, to ensure maximum trust and usability in the system. 362 \n  363 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n11 \nReferences 364 \n1.  Demner-Fushman D, Chapman WW, McDonald CJ. What can natural language processing do for 365 \nclinical decision support? J Biomed Inform. 2009 Oct 1;42(5):760–72.  366 \n2.  Spasić I, Livsey J, Keane JA, Nenadić G. Text mining of cancer-related information: Review of 367 \ncurrent status and future directions. Int J Med Inf. 2014 Sep 1;83(9):605–23.  368 \n3.  Strauss JA, Chao CR, Kwan ML, Ahmed SA, Schottinger JE, Quinn VP. Identifying primary and 369 \nrecurrent cancers using a SAS-based natural language processing algorithm. J Am Med Inform 370 \nAssoc. 2013 Mar 1;20(2):349–55.  371 \n4.  Datta MW, Hernandez AM, Schlicht MJ, Kahler AJ, DeGueme AM, Dhir R, et al. Perlecan, a 372 \ncandidate gene for the CAPB locus, regulates prostate cancer cell growth via the Sonic Hedgehog 373 \npathway. Mol Cancer. 2006 Mar 1;5(1):9.  374 \n5.  Banerjee I, Gensheimer MF, Wood DJ, Henry S, Aggarwal S, Chang DT, et al. Probabilistic 375 \nPrognostic Estimates of Survival in Metastatic Cancer Patients (PPES-Met) Utilizing Free-Text 376 \nClinical Narratives. Sci Rep. 2018 Jul 3;8(1):10037.  377 \n6.  Extracting Cancer Mortality Statistics from Free-text Death Certificates | Proceedings of the 23rd 378 \nAustralasian Document Computing Symposium [Internet]. [cited 2020 Aug 14]. Available from: 379 \nhttps://dl.acm.org/doi/abs/10.1145/3291992.3292003 380 \n7.  Black WC, Haggstrom DA, Gilbert Welch H. All-Cause Mortality in Randomized Trials of 381 \nCancer Screening. JNCI J Natl Cancer Inst. 2002 Feb 6;94(3):167–73.  382 \n8.  Turner EL, Metcalfe C, Donovan JL, Noble S, Sterne JAC, Lane JA, et al. Contemporary 383 \naccuracy of death certificates for coding prostate cancer as a cause of death: Is reliance on death 384 \ncertification good enough? A comparison with blinded review by an independent cause of death 385 \nevaluation committee. Br J Cancer. 2016 Jun;115(1):90–4.  386 \n9.  Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional 387 \nTransformers for Language Understanding. In: NAACL-HLT. 2019.  388 \n10.  Mujtaba G, Shuib L, Idris N, Hoo WL, Raj RG, Khowaja K, et al. Clinical text classification 389 \nresearch trends: Systematic literature review and open issues. Expert Syst Appl. 2019 Feb 390 \n1;116:494–520.  391 \n11.  J L, W Y, S K, D K, Ch S, J K. BioBERT: a pre-trained biomedical language representation 392 \nmodel for biomedical text mining. Bioinforma Oxf Engl. 2020 Feb 1;36(4):1234–40.  393 \n12.  Duarte F, Martins B, Pinto CS, Silva MJ. Deep neural models for ICD-10 coding of death 394 \ncertificates and autopsy reports in free-text. J Biomed Inform. 2018 Apr 1;80:64–77.  395 \n13.  Miftakhutdinov Z, Tutubalina E. KFU at CLEF eHealth 2017 Task 1: ICD-10 Coding of English 396 \nDeath Certificates with Recurrent Neural Networks. 2017.  397 \n14.  MIMIC-III, a freely accessible critical care database | Scientific Data [Internet]. [cited 2021 Jan 398 \n31]. Available from: https://www.nature.com/articles/sdata201635 399 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n 12 This is a provisional file, not the final typeset article \n15.  Mujtaba G, Shuib L, Raj RG, Rajandram R, Shaikh K. Automatic Text Classification of ICD-10 400 \nRelated CoD from Complex and Free Text Forensic Autopsy Reports. In: 2016 15th IEEE 401 \nInternational Conference on Machine Learning and Applications (ICMLA). 2016. p. 1055–8.  402 \n16.  Van Mulligen EM, Afzal Z, Akhondi S, Vo D, Kors J. Erasmus MC at CLEF eHealth 2016: 403 \nConcept recognition and coding in French texts. 2016 Jan 1 [cited 2021 Jan 31]; Available from: 404 \nhttps://repub.eur.nl/pub/100036/ 405 \n17.  Cabot C, Soualmia L, Dahamna B, Darmoni S. SIBM at CLEF eHealth Evaluation Lab 2016: 406 \nExtracting Concepts in French Medical Texts with ECMT and CIMIND. In 2016.  407 \n18.  Carvalho DV, Pereira EM, Cardoso JS. Machine Learning Interpretability: A Survey on Methods 408 \nand Metrics. Electronics. 2019 Aug;8(8):832.  409 \n19.  Binns R. Fairness in Machine Learning: Lessons from Political Philosophy. In: Conference on 410 \nFairness, Accountability and Transparency [Internet]. 2018 [cited 2020 Aug 16]. p. 149–59. 411 \nAvailable from: http://proceedings.mlr.press/v81/binns18a.html 412 \n20.  Diakopoulos N. Accountability in algorithmic decision making. Commun ACM. 2016 Jan 413 \n25;59(2):56–62.  414 \n21.  de Laat PB. Algorithmic Decision-Making Based on Machine Learning from Big Data: Can 415 \nTransparency Restore Accountability? Philos Technol. 2018 Dec;31(4):525–41.  416 \n22.  Sokol K, Hepburn A, Poyiadzi R, Clifford M, Santos-Rodriguez R, Flach P. FAT Forensics: A 417 \nPython Toolbox for Implementing andDeploying Fairness, Accountability and 418 \nTransparencyAlgorithms in Predictive Systems. J Open Source Softw. 2020 May 19;5(49):1904.  419 \n23.  treeinterpreter: Package for interpreting scikit-learn’s decision tree and random forest predictions. 420 \n[Internet]. [cited 2020 Sep 11]. Available from: https://github.com/andosa/treeinterpreter 421 \n24.  Archer KJ, Kimes RV. Empirical characterization of random forest variable importance 422 \nmeasures. Comput Stat Data Anal. 2008 Jan 10;52(4):2249–60.  423 \n25.  Ribeiro MT, Singh S, Guestrin C. “Why Should I Trust You?”: Explaining the Predictions of 424 \nAny Classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on 425 \nKnowledge Discovery and Data Mining [Internet]. New York, NY, USA: Association for 426 \nComputing Machinery; 2016 [cited 2020 Sep 11]. p. 1135–44. (KDD ’16). Available from: 427 \nhttps://doi.org/10.1145/2939672.2939778 428 \n26.  Lundberg SM, Lee S-I. A Unified Approach to Interpreting Model Predictions. Adv Neural Inf 429 \nProcess Syst. 2017;30:4765–74.  430 \n27.  Niculescu-Mizil A, Caruana R. Predicting good probabilities with supervised learning. In: 431 \nProceedings of the 22nd international conference on Machine learning [Internet]. New York, NY, 432 \nUSA: Association for Computing Machinery; 2005 [cited 2020 Sep 12]. p. 625–32. (ICML ’05). 433 \nAvailable from: https://doi.org/10.1145/1102351.1102430 434 \n28.  Amrit C, Paauw T, Aly R, Lavric M. Identifying child abuse through text mining and machine 435 \nlearning. Expert Syst Appl. 2017 Dec 1;88:402–18.  436 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n13 \n29.  Atutxa A, de Ilarraza AD, Gojenola K, Oronoz M, Perez-de-Viñaspre O. Interpretable deep 437 \nlearning to map diagnostic texts to ICD-10 codes. Int J Med Inf. 2019 Sep 1;129:49–59.  438 \n30.  Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. Interpretability of machine 439 \nlearning-based prediction models in healthcare. WIREs Data Min Knowl Discov. 440 \n2020;10(5):e1379.  441 \n31.  Berge GT, Granmo O, Tveit TO, Goodwin M, Jiao L, Matheussen BV. Using the Tsetlin 442 \nMachine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization With 443 \nMedical Applications. IEEE Access. 2019;7:115134–46.  444 \n32.  Lipton ZC. In machine learning, the concept of interpretability is both important and slippery. 445 \nMach Learn. :28.  446 \n33.  Chawla P, Hazarika S, Shen H-W. Token-wise sentiment decomposition for ConvNet: 447 \nVisualizing a sentiment classifier. Vis Inform. 2020 Jun 1;4(2):132–41.  448 \n34.  “What is relevant in a text document?”: An interpretable machine learning approach [Internet]. 449 \n[cited 2021 Mar 30]. Available from: 450 \nhttps://journals.plos.org/plosone/article?id=10.1371/journal.pone.0181142 451 \n35.  Ce P, Tie B. An Analysis Method for Interpretability of CNN Text Classification Model. Future 452 \nInternet. 2020 Dec;12(12):228.  453 \n36.  Hamdy FC, Donovan JL, Lane JA, Mason M, Metcalfe C, Holding P, et al. 10-Year Outcomes 454 \nafter Monitoring, Surgery, or Radiotherapy for Localized Prostate Cancer. N Engl J Med. 2016 455 \nOct 13;375(15):1415–24.  456 \n37.  Chen L. Attention-Based Deep Learning System for Negation and Assertion Detection in Clinical 457 \nNotes [Internet]. Rochester, NY: Social Science Research Network; 2019 Jan [cited 2020 Aug 458 \n18]. Report No.: ID 3342402. Available from: https://papers.ssrn.com/abstract=3342402 459 \n38.  Miller T, Bethard S, Amiri H, Savova G. Unsupervised Domain Adaptation for Clinical Negation 460 \nDetection. In: BioNLP 2017 [Internet]. Vancouver, Canada,: Association for Computational 461 \nLinguistics; 2017 [cited 2020 Aug 18]. p. 165–70. Available from: 462 \nhttps://www.aclweb.org/anthology/W17-2320 463 \n39.  McWilliams CJ, Lawson DJ, Santos-Rodriguez R, Gilchrist ID, Champneys A, Gould TH, et al. 464 \nTowards a decision support tool for intensive care discharge: machine learning algorithm 465 \ndevelopment using electronic healthcare data from MIMIC-III and Bristol, UK. BMJ Open. 2019 466 \nMar 1;9(3):e025925.  467 \n40.  Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial Discriminative Domain Adaptation. In 468 \n2017 [cited 2020 Aug 18]. p. 7167–76. Available from: 469 \nhttps://openaccess.thecvf.com/content_cvpr_2017/html/Tzeng_Adversarial_Discriminative_Dom470 \nain_CVPR_2017_paper.html 471 \n41.  Apostolova E, Wang T, Tschampel T, Koutroulis I, Velez T. Combining Structured and Free-text 472 \nElectronic Medical Record Data for Real-time Clinical Decision Support. In: Proceedings of the 473 \n18th BioNLP Workshop and Shared Task [Internet]. Florence, Italy: Association for 474 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n 14 This is a provisional file, not the final typeset article \nComputational Linguistics; 2019 [cited 2020 Aug 14]. p. 66–70. Available from: 475 \nhttps://www.aclweb.org/anthology/W19-5007 476 \n42.  Gal Y, Ghahramani Z. Dropout as a Bayesian Approximation:  Representing Model Uncertainty 477 \nin Deep Learning. :10.  478 \n43.  Bezemer T, Groot MCH de, Blasse E, Berg MJ ten, Kappen TH, Bredenoord AL, et al. A 479 \nHuman(e) Factor in Clinical Decision Support Systems. J Med Internet Res. 2019;21(3):e11732.  480 \n44.  Krallinger M, Akhondi S. Overview of the Chemical- Protein relation extraction track. :32.  481 \n45.  Peng Y, Yan S, Lu Z. Transfer Learning in Biomedical Natural Language Processing: An 482 \nEvaluation of BERT and ELMo on Ten Benchmarking Datasets. In: Proceedings of the 18th 483 \nBioNLP Workshop and Shared Task [Internet]. Florence, Italy: Association for Computational 484 \nLinguistics; 2019 [cited 2021 Feb 1]. p. 58–65. Available from: 485 \nhttps://www.aclweb.org/anthology/W19-5006 486 \n46.  McWilliams C, Inoue J, Wadey P, Palmer G, Santos-Rodriguez R, Bourdeaux C. Curation of an 487 \nintensive care research dataset from routinely collected patient data in an NHS trust. 488 \nF1000Research [Internet]. 2019 Aug 19 [cited 2021 Feb 1];8. Available from: 489 \nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6733376/ 490 \n 491 \n 492 \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\n  \n   \n \n \nFigure 1: Receiver operating characteristic (ROC) curve for three classifiers predicting cause of death from the CAP \ndataset, plotting the true positive rate (TPR) against the false positive rate (FPR). Performance curves are shown for: \nSVC = Support vector classifier; LR = logistic regression; RF = random forest \n \nFigure 2: Word clouds indicating the feature contributions towards random forest predictions of prostate cancer death \n(for the classifier depicted in Figure 1). The size of the word or bigram indicates its relative importanc e. Blue words or \nbigrams are indicative of prostate cancer death, while orange is indicative of not prostate cancer death. Feature \ncontributions determined using TreeInterpreter (see main text). (A) Average feature contributions over the CAP test \nset; (B) a single case that was correctly predicted prostate cancer death by the classifier (shown in Figure 3)  \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint \n\nFree-text cause of death predictions \n \n16 This is a provisional file, not the final typeset article \n \nFigure 3: Snapshot of an 'interpretable vignette' that allows users to engage with the prediction that is made by the \nclassifier. This case was correctly predicted to be a prostate cancer death by the classifier (cause of death code = 2). As \nin figure 2 the word (or bigram) size indicates the magnitude of the contribution of that feature to the prediction and \nthe colour indicates the sign of the contribution. Here the original format of the vignettes is retained, which is the \nformat in which the decision makers would normally engage with the document. Full interpretable vignette examples \nare provided in Online Resources 2 and 3     \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.15.21260082doi: medRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}