Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients

Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients Stefan Gustafsson, Daniel Gedon, Erik Lampa, Antônio H. Ribeiro, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1941398/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 10 You are reading this latest preprint version Abstract Myocardial infarction diagnosis is a common challenge in the emergency department. In managed settings, deep learning-based models and especially convolutional deep models have shown promise in electrocardiogram (ECG) classification, but there is a lack of high-performing models for the diagnosis of myocardial infarction in real-world scenarios. We aimed to train and validate a deep learning model using ECGs to predict myocardial infarction in real-world emergency department patients. We studied emergency department patients in the Stockholm region between 2007 and 2016 that had an ECG obtained because of their presenting complaint. We developed a deep neural network based on convolutional layers similar to a residual network. Inputs to the model were ECG tracing, age, and sex; and outputs were the probabilities of three mutually exclusive classes: non-ST-elevation myocardial infarction (NSTEMI), ST-elevation myocardial infarction (STEMI), and control status, as registered in the SWEDEHEART and other registries. We used an ensemble of five models. Among 492,226 ECGs in 214,250 patients, 5,416 were recorded with an NSTEMI, 1,818 a STEMI, and 485,207 without a myocardial infarction. In a random test set, our model could discriminate STEMIs/NSTEMIs from controls with a C-statistic of 0.991/0.832 and had a Brier score of 0.001/0.008. The model obtained a similar performance in a temporally separated test set, and achieved a C-statistic of 0.985 and a Brier score of 0.002 in discriminating STEMIs from controls in an external test set. We developed and validated a deep learning model with excellent performance in discriminating between control, STEMI, and NSTEMI on the presenting ECG of a real-world sample of the important population of all-comers to the emergency department. Hence, deep learning models for ECG decision support could be valuable in the emergency department. Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Background Emergency department care costs are high 1 and rising 2 in developed societies. Based on limited data in a chaotic environment, emergency medicine doctors must make quick decisions about patients’ probabilities for many diagnoses and risks. Diagnostic error is commonplace, 3 , 4 and there is a desperate need for emergency department decision support systems. 5 The emergency department handling of myocardial infarctions is especially precarious. In the United States, myocardial infarctions are missed in the range of 10–50,000 per year at emergency departments. 6 At the other end, less than half of those hospitalized for a suspected myocardial infarction are eventually diagnosed with the condition. 7 The electrocardiogram (ECG) can reveal the large ST-elevation myocardial infarctions (STEMIs), but the non-ST-elevation myocardial infarctions (NSTEMIs) are often inconspicuous to the human eye on the ECG, and rely on other means of diagnosis. Artificial intelligence (AI) has shown much recent promise in ECG classification, 8 for common ECG diagnoses 9 as well as for traits with unclear ECG diagnostic criteria or those not usually thought of as ECG diagnoses. 10 – 12 Even ECGs that appear normal to the human eye carries information useful for AI algorithms. 12 , 13 AI is very promising in the diagnosis of myocardial infarction, 14 , 15 but many studies have used limited 16 , 17 or managed 14 – 18 datasets. We postulate that AI ECG interpretation may be useful also in the most important population – all-comers to emergency departments. Using a large real-world sample of patients presenting at emergency departments, we developed and validated a deep learning model for diagnosis of NSTEMI and STEMI on the presenting ECG. Results Of the included total 492,226 ECGs from the emergency department visit, 5,416 (1.1%) were recorded with an NSTEMI, 1,818 (0.4%) with a STEMI and 484,992 (98.5%) without a myocardial infarction. Clinical characteristics of the study sample are presented in Table 1 and stratified for the data splits in Supplementary Table 2, and the patients’ age and admission date distributions are shown in Supplementary Figs. 1 and 2. Table 1 Clinical characteristics of the study sample Control NSTEMI STEMI Number of ECGs 484,992 5,416 1,818 Clinical characteristics at ED visit Age 65.0 (47.0,78.0) 71.0 (62.0,81.0) 66.0 (57.0,77.0) Male 47.3 65.4 73.7 Year 2013 (2010,2015) 2013 (2011,2015) 2013 (2011,2015) Presenting complaint Chest pain 21.5 71.3 70.1 Difficulty breathing 14.5 12.3 5.9 Dizziness 7.1 0.7 1.2 Heart problems 2.0 1.3 1.9 Circulatory arrest 0.1 1.1 3.7 Cardiovascular diagnoses prior to ED visit* Myocardial infarction 8.4 26.1 16.2 Unstable angina 4.2 11.2 6.4 Ischemic heart disease 20.1 43.1 23.8 Stroke 9.1 11.2 7.4 Peripheral artery disease 7.0 12.4 7.2 Heart failure 15.8 20.6 9.6 Atrial fibrillation 19.9 15.8 8.7 Cardiovascular disease 59.0 70.8 52.2 Drugs with > = 1 dispensation within one year prior to ED visit Renin-angiotensin system inhibitors 33.0 51.8 36.9 Calcium channel blockers 17.9 29.3 21.1 Beta-receptor blockers 36.3 50.6 33.9 Mineralocorticoid receptor antagonists 6.3 6.2 3.0 Diuretics 27.6 34.2 19.4 Anti-arrhythmic drugs 1.7 0.4 0.6 Statins 23.7 41.9 24.3 Anticoagulants 12.8 8.8 5.6 Antiplatelets 27.8 48.8 28.9 Cardiac enzymes within ED visit or coronary care unit hospitalization** Troponin I measured 1.1 8.2 13.8 Max troponin I (ng/L) 29.7 (29.7,40.0) 2700.0 (610.0,10150.0) 18100.0 (3100.0,47150.0) Troponin T measured 38.2 87.2 87.6 Max troponin T (ng/L) 9.9 (5.0,19.0) 266.0 (93.0,814.0) 1900.0 (519.0,4680.0) NTproBNP measured 8.7 21.5 17.1 Max NTproBNP (ng/L) 1340 (286,4300) 2940 (784,8720) 3120 (772,9220) Main diagnoses at admission *** Myocardial infarction 0.0 94.1 97.2 Unstable angina 0.2 4.1 1.6 Ischemic heart disease 1.3 94.6 97.2 Stroke 2.0 1.1 0.7 Peripheral artery disease 0.4 0.3 0.3 Heart failure 3.3 2.1 0.9 Atrial fibrillation 5.9 0.8 0.3 Cardiovascular disease 18.4 96.2 98.4 Mortality after coronary care unit admission 30-day all-cause death 3.5 6.4 10.6 In-hospital all-cause death 2.6 5.5 9.2 Patient characteristics of coronary care unit admissions included in the study, by control/NSTEMI/STEMI outcome. Data are medians (quartiles) or percent. *Prevalent disease based on any diagnosis position, inpatient and outpatient specialist care combined. **Combining troponin and high-sensitive troponin laboratory measurements from regional laboratory databases and the SWEDEHEART database. Maximum of all available measurements within the time window reported. Done separately for troponin I and T. ***Primary diagnosis from inpatient specialist care. STEMI, ST-elevation myocardial infarction; NSTEMI, non-ST-elevation myocardial infarction; ED, emergency department; NTproBNP, N-terminal pro-B-type natriuretic peptide. We conducted a preliminary analysis with a subgroup of patients that were admitted the CCU only. From a preliminary analysis we identified that all model and training modifications to the original baseline model 9 were effective in some metrics and overall contributed to the performance of our model. We identified two changes which were most important: the extension of our original dataset with the repeated recordings as data augmentation in the training data set, and the use of an ensemble-based model. The performance of our model in the two test datasets and a publicly available dataset, PTB-XL, is described in Table 2 including model uncertainty over ten different initialization seeds. Additionally in Supplementary Table 4, we show bootstrapped data uncertainty with similar performance, details for this are in the Supplementary Methods. In the random test set, STEMIs could be discriminated with a C-statistic of 0.991 and the model had a Brier score of 0.001. For NSTEMIs, the model had a C-statistic of 0.832 and a Brier score of 0.008, with lower precision than STEMIs. The temporal test set, which contained data from emergency department visits that did not overlap in time of the development set, resulted in a C-statistic of 0.985 for STEMI and 0.867 for NSTEMI. Figure 5 illustrates receiver operating characteristics and precision-recall curves for multiple independently trained models on the two test splits. Calibration was acceptable (calibration plots in Supplementary Fig. 3, predicted probabilities in Supplementary Fig. 4). Table 2 Performance of the model in random, temporal and external test sets Random Temporal PTB-XL N (%) Control 88,742 (98.9) 27,561 (98.7) 200 (72.7) STEMI 193 (0.2) 108 (0.4) 25 (27.3) NSTEMI 820 (0.9) 263 (0.9) - MI 1,013 (1.1) 371 (1.3) - C-statistic (↑) Control 0.863 (0.860–0.870) 0.903 (0.896–0.909) 0.962 (0.953–0.97) STEMI 0.991 (0.988–0.994) 0.985 (0.983–0.987) 0.932 (0.904–0.959) NSTEMI 0.832 (0.828–0.841) 0.867 (0.859–0.876) - MI 0.863 (0.860–0.870) 0.903 (0.896–0.909) - AP (↑) Control 0.998 (0.998–0.998) 0.998 (0.998–0.998) 0.955 (0.934–0.972) STEMI 0.692 (0.641–0.727) 0.744 (0.716–0.773) 0.954 (0.935–0.971) NSTEMI 0.160 (0.134–0.168) 0.184 (0.144–0.214) - MI 0.330 (0.307–0.347) 0.466 (0.42–0.484) - Brier (↓) Control 0.009 (0.009–0.009) 0.009 (0.009–0.010) 0.145 (0.126–0.158) STEMI 0.001 (0.001–0.001) 0.002 (0.002–0.002) 0.184 (0.167–0.196) NSTEMI 0.008 (0.008–0.008) 0.008 (0.008–0.009) - Multiclass 0.018 (0.018–0.018) 0.019 (0.019–0.020) - ECE (↓) Multiclass 0.417 (0.416–0.418) 0.415 (0.415–0.417) 0.277 (0.257–0.297) Results of the model in the two test sets and the publicly available PTB-XL dataset as comparison in the rightmost column. Sample size is given by each testset and outcome label together with proportion out of the given testset. Performance metrics are given as median (minimum-maximum) over ten trained models initiated with different seeds; each of the ten models is an ensemble consisting of five model members. Arrows indicate direction of better performance. We compute the metrics as class vs. all. Note that the class MI merges the classes STEMI and NSTEMI in one class. ECE is for multi-class calibration instead of class-wise calibration. STEMI, ST-elevation myocardial infarction; NSTEMI, non-ST-elevation myocardial infarction; AP, Average Precision or equivalently Area Under the Precision-Recall curve; ECE, Expected Calibration Error. To ensure that our model did not use any proxies for its predictions, we stratified the test datasets according to different possible confounders. Supplementary Figs. 5 (C-statistic) illustrate results stratified according to test set records 1) without any filter applied based on ST-elevation label noise, 2) ST-elevation filter corresponding to results in main table (used in all following test set subsets); 3) stricter ST-elevation where “possible STEMI” (see Supplementary Methods) were removed from both STEMI and NSTEMI cases, 3) age tertiles, 4) sex, 5) ECG collected at the same day as the admission or not, 6) emergency department at Karolinska Hospital (main source of data) or another emergency department in the Stockholm region, 7) patients attending the CCU only, 8) ECGs recorded using the most common machine type (MAC55) or not, 9) ECGs recorded using the most common software (v237) or not. Apart from analyses restricting the test set controls to CCU controls only, we observed no major changes to the model performances, indicating that the predictions are not driven by these potential confounders. As an additional test, we trained a model with only ECG traces as input, omitting age and sex. The results were almost identical to our main results indicating that the explicit addition of age and sex to the model was not crucial for our results. Furthermore, we compare our results with a model trained on data from patients at the coronary care unit alone. This dataset has 16,628 ECGs, i.e. a subset of 3.4% of our current dataset, omitting most of the control patients which were not admitted to the coronary care unit. The results from this model show that the larger dataset including all controls is important for our performance. Inspecting Grad-CAM plots yielded new insights. Figure 6 illustrates four STEMIs correctly classified with high probability. In panels A and B the model had focused on the ST-segment, where a human would look. In panels C and D, the model also used the down-sloping part of the T-wave, where a human would not focus when diagnosing a STEMI. Figure 7 illustrates four correctly classified NSTEMIs. In all panels, the model had focused on the ST-segment, but had also used the last part of the T-wave, which a human would not. Characteristics of hospitalizations with misclassified ECGs are described in Supplementary Table 3. Among the misclassified ECGs, those misclassified as STEMI more often had perimyocarditis, valvular disease or cardiomyopathy; those misclassified as NSTEMI more often had valvular or congenital heart disease, pulmonary edema, gastric ulcer or dental traits. Discussion In a very large sample of all-comers to emergency departments, we developed and validated an AI model that can classify STEMI and NSTEMI versus non-myocardial infarctions using routine 10-second ECGs. The model achieved excellent performance for both STEMI and NSTEMIs. Doctors’ ECG interpretation is often imprecise, with a reported accuracy of 0.69 overall for practicing physicians and 0.75 for cardiologists in controlled test settings, 27 with similar numbers reported for STEMIs. 28 , 29 Performance of doctors outside of such standardized settings is unknown but not likely better. Diagnosis by humans of NSTEMIs from ECGs is almost by definition futile and has seen very little research. Importantly, we do not observe a worse performance in the temporal test set when compared to the random test set. This suggests that our model works on data outside of the training data. In addition, it performs well in the discrimination of STEMIs in the external PTB-XL database test set. This study is clinically important as it uses the most relevant sample possible. These consecutive all-comers represent the real-world ECG experience for emergency doctors, with ECGs - especially among the non-infarctions - that are far from the very clear specimens in managed online databases or heavily curated samples. Notably, in Sweden today and during the study period, pre-hospital ECGs are sent to coronary care units for immediate diagnosis, so the obvious STEMI cases usually bypass the emergency department and transfer straight to the coronary intervention lab upon arrival to hospital, rendering the STEMIs in the present study the less obvious cases and the walk-ins. Hence, these are cases in great need of decision support. Further, we did not exclude difficult cases, comorbidities, or previous myocardial infarctions (except for technical reasons, we removed potentially linked hospitalizations for the same myocardial infarction, and LBBBs, which cannot per se identify an acute myocardial infarction from a single ECG, but need a prior ECG for comparison). Other studies using deep learning models have also shown good performance but lower than our model’s performance when tested in representative settings, 14 with reports of very good and similar to our model’s performance in managed settings. 14 , 15 Our study differs from the previous literature in that it is a multicenter study using real-world data of consecutive patients with very few exclusions, and with output labeling by many doctors. Descriptions of the clinical setting and the controls are sometimes unclear. 15 Past studies have seldom investigated NSTEMIs. Our model performed slightly better in younger than older patients for NSTEMI classification, and slightly better for STEMI classification in men than in women. Younger NSTEMI patients might have fewer underlying diseases, potentially making the prediction simpler. Men contribute to around 2/3 of the myocardial infarctions, potentially making male infarctions easier to learn given more data. A stricter filter for label noise based on a mismatch between original labels and updated ST-elevation annotations shows that a stricter filter improved the result for STEMI whereas the opposite was seen for NSTEMI, but the differences were minor. Overall, the results were comparable across test set subsets and some differences in prediction results might be due to pure chance. A notable difference in the analyses restricting the test set controls to CCU controls only may be due to more underlying heart conditions in those controls, and low power. The Grad-CAM plots in Figs. 6 and 7 provide important insights. The model recognizes the same ST-segment features that humans would. But the model also finds features that are novel, or that are imperceptible to the human eye. This shows an interesting way forward. We give the AI ECGs and the label, then the AI teaches us novel ways to read the ECGs. Variants of such model evaluation can likely give useful clinical and pathophysiological clues in many medical fields. Our model’s misclassifications as STEMI follows known clinical and machine learning patterns, with perimyocarditis as an important impostor. 30 The conditions over-represented in those misclassified as NSTEMI were logical to some extent, such as valvular or congenital heart disease and pulmonary edema; the gastric ulcer and dental traits more surprising. Some important limitations are worth mentioning. Our dataset contains some label noise. The label was determined at discharge from the coronary care unit or emergency room when the whole care episode could be summarized. The ECGs in the test sets of this study may hence not always be the ones guiding the final diagnosis. We mitigate that to some extent by using multiple ECGs if available within the day before admission in the training set, but not in the test sets. On the other hand, the hindsight allows for more stable labels for the episode as a whole, which is the ultimate goal for the classification. More information about the handling of label noise is provided in the supplementary materials. Another important limitation is the lack of an external validation sample. We did hold out the 10% of the patients with their first admission in 2016 as a temporal test set; many circumstances in that set would be similar to those in the earlier training set, but a restructuring of the Stockholm region emergency department logistics which is our main data source during the data collection period did change the composition of the sample. Furthermore, we make use of the publicly available PTB-XL dataset which does include data containing STEMI but not NSTEMI. In this dataset, our model achieves good discriminative performance. No publicly available data repositories contain ECGs with NSTEMI labels to test this externally. While the calibration of our model was better than that of comparable models, there is still room for improvement; calibration is indeed an underappreciated property in general. We did not consider transferring learned features, only model architecture, from a previous study. 9 An exploration of potential improvements in model convergence speed and final performance boost by pre-training on a different ECG classification task with a dataset in a different context may be useful, but may also introduce model biases from the other dataset. Lastly, we did not compare the performance of this ECG model to troponin-based or other methods of diagnosing myocardial infarction. Ultimately, clinical usefulness must be evaluated in a randomized trial. In conclusion, we developed and validated a deep learning model with excellent performance in discriminating between NSTEMI, STEMI and controls on the presenting ECG of a large real-world sample of general emergency department patients. Considering the high and rising emergency department care costs and the high numbers of missed myocardial infarctions at emergency departments, our model could be of clinical value for ECG decision support, with promise of further performance development. Methods Sample We utilized a consecutive sample of adult all-comer patients attending emergency departments in the Stockholm region between 2007 and 2016, for whom a routine ECG was obtained upon their presenting complaint. Details of the data sources are described in the Supplementary Methods. The procedure and criteria used to define the study sample are described in Fig. 1 , with available exposure and outcome data described below and in Supplementary Methods. In total, 217,667 patients had at least one registered emergency department within the study period with at least one valid ECG recording within 1 day of visit. The emergency department visits were either followed by a coronary care unit (CCU) admission (NSTEMI/STEMI/control) or had no subsequent CCU admission (control only). After applying the sequence of filters described in Fig. 1 to ensure inclusion of at-event before-treatment ECGs, as well as confirming the outcome label, 214,250 patients with 492,226 ECGs were available for analysis, representing a total of 12,328 CCU admissions and 412,980 non-CCU visits. Out of these ECGs, 67,137 exams were repeated recordings, i.e., the same patient had multiple ECG exams in the same visit (used in training the model only). The study was approved by the Swedish Ethical Review Agency, application number 2020 − 01654. Informed consent was waived in this study by all applicable ethics review boards, i.e. the Region Stockholm Ethics Review Board and the National Ethics Review Authority. All methods were performed in accordance with the relevant guidelines and regulations. Exposures and outcomes High-quality data on the exposures and outcomes were available for all included patients from discharge records from the emergency departments, electronic health records, from linked hospitalizations, and from the SWEDEHEART registry with patient records joined on the personal identifier number of the patient. Data sources and definitions used are described in Supplementary Methods and Supplementary Table 1. As exposures, we used digital ECG data, age and sex, as in a previous study. 15 Standard 10-second 12-lead ECG recordings sampled at 250 to 500Hz were used; 8 leads were used in the present study as 4 of the standard leads are linear combinations of these 8 and are hence redundant. For the mutually exclusive outcome labels NSTEMI/STEMI/control status, trained in a single model, we used the high-quality SWEDEHEART registry (CCU admissions). In addition, diagnoses from the Swedish in-patient and cause-of-death registries were used to confirm a myocardial infarction (I21) for cases, or absence of a myocardial infarction for controls. The SWEDEHEART Riks-HIA labels are the decision of a discharging physician that followed the entire patient journey during the hospitalization; this physician had access to other exams besides the ECG, such as coronary angiograms in some patients, and blood testing in all patients. This label is the accepted gold standard for all research using SWEDEHEART data; efforts to further minimize label noise are described in Supplementary Methods. We only included cases with complete data on the exposures and outcomes. Machine learning methods Training and validation datasets The patients fulfilling the inclusion criteria were divided in 70%/30% splits with records from the same patient always put in the same split. The 70% split was used for training and developing the model and the 30% split for testing the model performance. The 30% test split was further divided into two splits containing 20% and 10% of the complete data, to allow us to test the model in two different scenarios. The 10% split contains patients with a first recorded admission date after 2016-01-01 or later, hence temporally separated from the training data set which included patients with a first recorded admission before this date. This way the 10% test split can be used to assess the model susceptibility to shifts and trends that change with time. We denote this split the temporal test split . The other 20% split was sampled at random from entries with an admission date before 2016-01-01, which is the same period the 70% training split is sampled from. We denote this split the random test split . The process is illustrated in Fig. 2 and patient characteristics in the sets are presented in Supplementary Table 2. Our temporal test set can be considered the best possible external validation available for the NSTEMI cases since there exists no other publicly available data set available containing ECGs and NSTEMI annotations. An additional external validation dataset with 75 acute myocardial infarction cases with ST-elevation and 200 randomly selected controls was manually curated from the PTB-XL database. 19 , 20 The PTB-XL is a publicly available database of 21,837 10-second 12-lead ECGs annotated with 71 different ECG statements, including cases of myocardial infarction. The inclusion criteria of the PTB-XL test set used in this study is described in the Supplementary Methods. Data pre-processing Data pre-processing steps are described in Fig. 3 . In addition to the ECG tracings, we limited ourselves to adding age and sex, to make the model as transportable and unbiased as possible. The output was the probabilities of the three mutually exclusive outcome classes: NSTEMI/STEMI/control. Model architecture Our Deep Neural Network (DNN) model architecture is an extension of a previous model, 9 for which the DNN was trained to detect six types of ECG abnormalities. 21 We used a neural network based on convolutional layers similar to a residual network (ResNet) that is commonly used in image classification, but adapted here to unidimensional signals. This architecture allows DNNs to be efficiently trained by including skip connections. We adapt a modification of layer arrangements within the residual block and a skip connection which is shown to be more effective. 22 The model architecture is depicted and described in Fig. 4 . Age and sex were passed through a fully connected layer and concatenated with the flattened output of the residual blocks. The resulting features were used in the final linear classification layer which outputs the model prediction. The output of the trained model was the probabilities of the three mutually exclusive outcome classes NSTEMI/STEMI/control. Ensembles of neural network models improve predictive performance, 23 and lead to better calibrated models. Therefore, we expanded our model as an ensemble of five model members. Each of the five individual model members was trained from scratch from different parameter initialization but with the same training data to obtain different output logits. These logits were averaged to obtain the final prediction. A detailed description of all model hyperparameters is provided in the Supplementary Methods. Model training and validation The model was trained by minimizing the cross-entropy loss for 100 epochs. Details about the training hyperparameter and regularization terms are described in the Supplementary Methods. In addition to the training dataset, we made use of the repeated ECG exams of patients who had multiple exams at an emergency department visit for training but not for validation and testing. We consider this as a form of data augmentation since these exams have the same label but are recorded at different times and therefore with a slightly different state of the patient and possibly different placement of the ECG leads. For validation and testing, only the first recorded ECG of an admission was used. Details are in the Supplementary Methods. We set aside a total of 10% of the training data for validation. Note that we improve the training procedure and model architecture from a previous study 9 with established methods. 24 The details for the model and training improvements are described in Supplementary Methods. Hyperparameter search We conducted a search over 6 hyperparameters and their respective values. This gives a total of 2025 possible combinations. Due to the high number of possible combinations, we ran a random search for 345 combinations. The remaining model and training hyperparameters were fixed in this search since they have been proven efficient in the baseline architecture. 9 We evaluated the tested hyperparameters in the validation dataset. Note that during the search we only train a single model and not an ensemble of multiple models. Details of the hyperparameter search are outlined in the Supplementary Methods. Model calibration Performance of prediction models is often quantified using the C-statistic that measures discrimination, i.e. the model’s ability to assign higher risk estimates/probabilities to patients who will experience the event than patients who will not. While discrimination is important, it tells us nothing about the reliability of the probability estimates. Calibration, i.e. if the model’s probability estimates reflect the ground truth empirical class frequencies, is a more important property of the model for clinical use. One of our main concerns was model calibration, since modern DNN architectures are generally poorly calibrated. 25 As metrics of model calibration, we focused on the Expected Calibration Error (ECE) and the Brier score estimated in the validation set. The ECE is the weighted absolute difference between the class membership and the estimated probability for that class averaged over 15 bins, while the Brier score measures the average squared error on the probability scale. Both metrics are applicable to both binary and multiclass problems. We visualized the calibration of our model using calibration plots, also called reliability diagrams. Model evaluation We investigated possible patterns for our models correct and incorrect classifications of STEMIs and NSTEMIs. First, we used Grad-CAM plots to highlight the parts of the ECG that the model focuses on - meaning the sections of the input ECG where the model puts its most weight on - to make its predictions. 26 This method generates the visualization in two steps: In a forward pass step, it computes the activations of the neural network in a given intermediary layer (in our case, the first convolutional layer). And in a backward step, it computes the gradients corresponding to these activations. The gradients are averaged to get the proportional importance of each channel, which is then used to compute a proportional mean of the activations. Positive values obtained were plotted as purple disks overlaid on top of the ECG, with size proportional to its magnitude, to generate the visualization. One senior cardiologist (JS) inspected the Grad-CAM plots from the ten cases with the highest estimated probability for each myocardial infarction class and selected four illustrative plots per class. We tested the over/underrepresentation of inpatient and specialized outpatient care diagnoses at the time of the visit among correctly classified versus misclassified patients with a predicted probability > 0.5 in pairwise independent tests to find diagnoses where the model often made incorrect classifications. Declarations Funding The Kjell and Märta Beijer Foundation, Anders Wiklöf, the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, and Uppsala University. Acknowledgements Non-author contributions We thank David Widmann for the discussions and his input towards analyzing and improving the calibration of our model. Role of the funding source The study was funded by The Kjell and Märta Beijer Foundation, Anders Wiklöf, The Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by Knut and Alice Wallenberg Foundation, and Uppsala University. The computations were enabled by resources in project sens2020005 and sens2020598 provided by the Swedish National Infrastructure for Computing (SNIC) at UPPMAX, partially funded by the Swedish Research Council through grant agreement no. 2018-05973. The funders had no role in in study design; collection, analysis, and interpretation of data; writing of the report; or decision to submit the paper for publication. Data Availability The data that support the findings of this study are available from the Swedish Board of Health and Welfare and the included healthcare regions, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from corresponding author (J.S.) upon reasonable request and with permission of the Swedish Board of Health and Welfare and the included healthcare regions. All code together with the parameter/hyperparameter estimates of the presented models is available upon request. Author Contributions All authors participated in the study design, interpretation of the data, and critically reviewing the paper. S.G, D.G, E.L, A.H.R, and J.S wrote the first draft, and M.J.H, and T.B.S. contributed to the writing of subsequent versions. Data was acquired and prepared for analysis by M.J.H, S.G, and A.H.R. Statistical analyses and preparation of tables and figures was performed by S.G, D.G, E.L, and A.H.R with support from J.S and T.B.S. M.J.H, J.S, T.B.S. were responsible for funding acquisition, project administration, and supervision. All authors had access to and could verify the underlying data. Competing Interests J.S. reports stock ownership in companies providing services to Itrim, Amgen, Janssen, Novo Nordisk, Pfizer, Takeda, AstraZeneca and Vifor Pharma, outside the submitted work. S.G. is employed at Sence Research AB. The other authors report no conflicts of interest. References Galarraga JE and Pines JM. Costs of ED episodes of care in the United States. Am J Emerg Med. 2016;34:357–65. Lane BH, Mallow PJ, Hooker MB and Hooker E. Trends in United States emergency department visits and associated charges from 2010 to 2016. Am J Emerg Med. 2020;38:1576–1581. Moonen PJ, Mercelina L, Boer W and Fret T. Diagnostic error in the Emergency Department: follow up of patients with minor trauma in the outpatient clinic. Scand J Trauma Resusc Emerg Med. 2017;25:13. Medford-Davis L, Park E, Shlamovitz G, Suliburk J, Meyer AN and Singh H. Diagnostic errors related to acute abdominal pain in the emergency department. Emerg Med J. 2016;33:253–9. Wright B, Faulkner N, Bragge P and Graber M. What interventions could reduce diagnostic error in emergency departments? A review of evidence, practice and consumer perspectives. Diagnosis (Berl). 2019;6:325–334. Sharp AL, Baecker A, Nassery N, Park S, Hassoon A, Lee MS, Peterson S, Pitts S, Wang Z, Zhu Y and Newman-Toker DE. Missed acute myocardial infarction in the emergency department-standardizing measurement of misdiagnosis-related harms using the SPADE method. Diagnosis (Berl). 2020;8:177–186. Caulfield CA and Stephens JR. Things We Do for No Reason: Hospitalization for the Evaluation of Patients with Low-Risk Chest Pain. J Hosp Med. 2018;13:277–279. Siontis KC, Noseworthy PA, Attia ZI and Friedman PA. Artificial intelligence-enhanced electrocardiography in cardiovascular disease management. Nature reviews Cardiology. 2021. Ribeiro AH, Ribeiro MH, Paixao GMM, Oliveira DM, Gomes PR, Canazart JA, Ferreira MPS, Andersson CR, Macfarlane PW, Meira W, Jr., Schon TB and Ribeiro ALP. Automatic diagnosis of the 12-lead ECG using a deep neural network. Nat Commun. 2020;11:1760. Tison GH, Zhang J, Delling FN and Deo RC. Automated and Interpretable Patient ECG Profiles for Disease Detection, Tracking, and Discovery. Circ Cardiovasc Qual Outcomes. 2019;12:e005289. Cohen-Shelly M, Attia ZI, Friedman PA, Ito S, Essayagh BA, Ko WY, Murphree DH, Michelena HI, Enriquez-Sarano M, Carter RE, Johnson PW, Noseworthy PA, Lopez-Jimenez F and Oh JK. Electrocardiogram screening for aortic valve stenosis using artificial intelligence. Eur Heart J. 2021. Raghunath S, Ulloa Cerna AE, Jing L, vanMaanen DP, Stough J, Hartzel DN, Leader JB, Kirchner HL, Stumpe MC, Hafez A, Nemani A, Carbonati T, Johnson KW, Young K, Good CW, Pfeifer JM, Patel AA, Delisle BP, Alsaid A, Beer D, Haggerty CM and Fornwalt BK. Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network. Nature medicine. 2020;26:886–891. Attia ZI, Noseworthy PA, Lopez-Jimenez F, Asirvatham SJ, Deshmukh AJ, Gersh BJ, Carter RE, Yao X, Rabinstein AA, Erickson BJ, Kapa S and Friedman PA. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. Lancet. 2019;394:861–867. Liu WC, Lin CS, Tsai CS, Tsao TP, Cheng CC, Liou JT, Lin WS, Cheng SM, Lou YS, Lee CC and Lin C. A Deep-Learning Algorithm for Detecting Acute Myocardial Infarction. EuroIntervention . 2021. Cho Y, Kwon JM, Kim KH, Medina-Inojosa JR, Jeon KH, Cho S, Lee SY, Park J and Oh BH. Artificial intelligence algorithm for detecting myocardial infarction using six-lead electrocardiography. Sci Rep. 2020;10:20495. Makimoto H, Hockmann M, Lin T, Glockner D, Gerguri S, Clasen L, Schmidt J, Assadi-Schmidt A, Bejinariu A, Muller P, Angendohr S, Babady M, Brinkmeyer C, Makimoto A and Kelm M. Performance of a convolutional neural network derived from an ECG database in recognizing myocardial infarction. Sci Rep. 2020;10:8445. Al-Zaiti S, Besomi L, Bouzid Z, Faramand Z, Frisch S, Martin-Gill C, Gregg R, Saba S, Callaway C and Sejdic E. Machine learning-based prediction of acute coronary syndrome using only the pre-hospital 12-lead electrocardiogram. Nat Commun. 2020;11:3966. Zhao Y, Xiong J, Hou Y, Zhu M, Lu Y, Xu Y, Teliewubai J, Liu W, Xu X, Li X, Liu Z, Peng W, Zhao X, Zhang Y and Xu Y. Early detection of ST-segment elevated myocardial infarction by artificial intelligence with 12-lead electrocardiogram. Int J Cardiol. 2020;317:223–230. Wagner P, Strodthoff N, Bousseljot RD, Kreiseler D, Lunze FI, Samek W and Schaeffter T. PTB-XL, a large publicly available electrocardiography dataset. Sci Data. 2020;7:154. Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng CK and Stanley HE. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. 2000;101:E215-20. Alkmim MB, Figueira RM, Marcolino MS, Cardoso CS, Pena de Abreu M, Cunha LR, da Cunha DF, Antunes AP, Resende AG, Resende ES and Ribeiro AL. Improving patient access to specialized health care: the Telehealth Network of Minas Gerais, Brazil. Bull World Health Organ. 2012;90:373–8. He K, Zhang X, Ren S and Sun J. Identity Mappings in Deep Residual Networks. Computer Vision – ECCV 2016. 2016:630–645. Hansen LK and Salamon P. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1990;12:993–1001. Bello I, Fedus W, Du X, Cubuk ED, Srinivas A, Lin T-Y, Shlens J and Zoph B. Revisiting resnets: Improved training and scaling strategies. Advances in Neural Information Processing Systems 34 . 2021. Guo C, Pleiss G, Sun Y and Weinberger KQ. On calibration of modern neural networks. International Conference on Machine Learning . 2017:1321–1330. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D and Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE international conference on computer vision . 2017:618–626. Cook DA, Oh SY and Pusic MV. Accuracy of Physicians' Electrocardiogram Interpretations: A Systematic Review and Meta-analysis. JAMA internal medicine. 2020;180:1461–1471. McCabe JM, Armstrong EJ, Ku I, Kulkarni A, Hoffmayer KS, Bhave PD, Waldo SW, Hsue P, Stein JC, Marcus GM, Kinlay S and Ganz P. Physician accuracy in interpreting potential ST-segment elevation myocardial infarction electrocardiograms. Journal of the American Heart Association. 2013;2:e000268. Soares WE, 3rd, Price LL, Prast B, Tarbox E, Mader TJ and Blanchard R. Accuracy Screening for ST Elevation Myocardial Infarction in a Task-switching Simulation. West J Emerg Med. 2019;20:177–184. Tanguay A, Lebon J, Brassard E, Hebert D and Begin F. Diagnostic accuracy of prehospital electrocardiograms interpreted remotely by emergency physicians in myocardial infarction patients. Am J Emerg Med. 2019;37:1242–1247. Additional Declarations Competing interest reported. J.S. reports stock ownership in companies providing services to Itrim, Amgen, Janssen, Novo Nordisk, Pfizer, Takeda, AstraZeneca and Vifor Pharma, outside the submitted work. S.G. is employed at Sence Research AB. The other authors report no conflicts of interest. Supplementary Files AI4ECGSTEMINSTEMIsupplementupdated20220816.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Major revision 23 Sep, 2022 Reviews received at journal 10 Sep, 2022 Reviews received at journal 10 Sep, 2022 Reviewers agreed at journal 31 Aug, 2022 Reviewers agreed at journal 31 Aug, 2022 Reviewers invited by journal 30 Aug, 2022 Editor assigned by journal 30 Aug, 2022 Editor invited by journal 23 Aug, 2022 Submission checks completed at journal 23 Aug, 2022 First submitted to journal 08 Aug, 2022 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1941398","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":131105075,"identity":"dd46687f-1168-4255-a421-6a96871ceb94","order_by":0,"name":"Stefan Gustafsson","email":"","orcid":"","institution":"Uppsala University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Stefan","middleName":"","lastName":"Gustafsson","suffix":""},{"id":131105077,"identity":"916a7cb3-ec50-49f9-9803-92504e3b665e","order_by":1,"name":"Daniel Gedon","email":"","orcid":"","institution":"Uppsala University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Daniel","middleName":"","lastName":"Gedon","suffix":""},{"id":131105079,"identity":"2d35ac61-0c58-490f-b3b7-adceeba8ee8d","order_by":2,"name":"Erik Lampa","email":"","orcid":"","institution":"Uppsala University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Erik","middleName":"","lastName":"Lampa","suffix":""},{"id":131105080,"identity":"4f4f538a-5dfe-4314-8305-c60d25eb891c","order_by":3,"name":"Antônio H. Ribeiro","email":"","orcid":"","institution":"Uppsala University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Antônio","middleName":"H.","lastName":"Ribeiro","suffix":""},{"id":131105082,"identity":"bcd94c36-13d5-4bd7-8eff-ced439070895","order_by":4,"name":"Martin J. Holzmann","email":"","orcid":"","institution":"Karolinska University Hospital, Karolinska Institutet","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Martin","middleName":"J.","lastName":"Holzmann","suffix":""},{"id":131105084,"identity":"72f80f0a-f089-4821-bd91-eeaacc3c2b8b","order_by":5,"name":"Thomas B. Schön","email":"","orcid":"","institution":"Uppsala University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Thomas","middleName":"B.","lastName":"Schön","suffix":""},{"id":131105085,"identity":"e2d601dc-8f65-4c42-979d-8076b76c71c8","order_by":6,"name":"Johan Sundström","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYBACxgYeGDMBTMqRpIWxAUgaE2EPmpbEBkIamNt7D34uYKhL3M6e/vxxRcW29A03Epg/fMDnsJ5zydIzGA4n7ux5Y9h45sztXKAWNskZ+LTMyDGQ5mE4kLjhRg5jY2MbRAszD34txr95gA7bcCP9IUhLugHQYZ//4NdiBrSFGaglwRCkJQGohUEan/cZe86YWfMYHDbecOaN4cyGM7cNZ5552CbZg0eLYXuP8W2eijrZDcfTH3xsqLgtz3c8+fCHH/i0NIBIA1SbG/C5i0Eer+woGAWjYBSMAhAAAOXGVTZqLrcLAAAAAElFTkSuQmCC","orcid":"","institution":"Uppsala University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Johan","middleName":"","lastName":"Sundström","suffix":""}],"badges":[],"createdAt":"2022-08-08 13:14:30","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1941398/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1941398/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":25677077,"identity":"686c87ba-ece4-40ae-9990-e1b2797f60d8","added_by":"auto","created_at":"2022-08-25 18:23:14","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":262488,"visible":true,"origin":"","legend":"\u003cp\u003eDerivation of the study sample\u003c/p\u003e\u003cp\u003eInclusion/exclusion criteria applied to define the study sample. SWEDEHEART, Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies; RIKS-HIA, Register of Information and Knowledge About Swedish Heart Intensive Care Admissions; CCU, coronary care unit; ED, emergency department; ED-CCU, ED visit with CCU admission with outcome label and ECG within 0-1 days prior to CCU admission and ED date within 0-1 days prior to CCU admission; ED-only, ED visit without any CCU admission within +/-30 days of ECG recording; NSTEMI, non-ST-elevation myocardial infarction; STEMI, ST-elevation myocardial infarction.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/ef92f5103771919efff47f00.png"},{"id":25677446,"identity":"1c0bc5b8-4b47-4f3a-8833-0ef38aa865b5","added_by":"auto","created_at":"2022-08-25 18:28:14","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":84215,"visible":true,"origin":"","legend":"\u003cp\u003eData splitting\u003c/p\u003e\u003cp\u003eThe dataset was split in a training set consisting of 70% and two test sets consisting of 10% and 20% of the complete data each. The last row in the figure lists the total number of ECGs in each data split. Repeated ECG recordings at the same visit of a patient were added to the training set only.\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/a8295c65b1f606a7b479f958.png"},{"id":25677071,"identity":"278e9b47-5aec-423f-9aed-2a0eccc6efbe","added_by":"auto","created_at":"2022-08-25 18:23:14","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":37754,"visible":true,"origin":"","legend":"\u003cp\u003eData pre-processing\u003c/p\u003e\u003cp\u003eFrom the raw input ECG we removed the baseline by filtering the tracing with a high-pass filter to remove biases and low-frequency trends. The filter is an elliptic filter with a cut-off frequency of 0.8 Hz and an attenuation of 40 dB, applied to the forward and reverse direction to obtain zero-phase distortion. We then resampled all ECGs to 400 Hz and zero-padded to a fixed length of 4096 samples, since the convolution-based model requires a fixed input size. For duplicated ECGs with identical data and collection time, the first copy was kept. ECGs where one or more required leads were missing or contained all-zero entries were removed. We used one-hot encoding for sex and normalized the age with the mean and standard deviation of the training dataset.\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/b8271f48d327f3ff58fb973e.png"},{"id":25677447,"identity":"eb6e1f31-cafc-49db-842a-581e41f7f982","added_by":"auto","created_at":"2022-08-25 18:28:14","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":114896,"visible":true,"origin":"","legend":"\u003cp\u003eDeep neural network model architecture\u003c/p\u003e\u003cp\u003eThe left panel is a high-level model architecture for ECG classification consisting of one part to extract features from the ECG exam and one for features from phenotypes age and sex. The light green block contains a convolutional layer followed by a batch normalization for rescaling the output and a ReLU activation function. This layer is followed by four sets of each three residual blocks in light blue, i.e. 12 residual blocks in total. The name of the block indicates the filter size of the convolutions, the number of filters and the downsampling factor (if applicable). Note that we downsample the signal by a factor of 1/2 in the beginning and ending of each set of residual blocks. The right panel illustrates the content of each residual block. Dropout is used after each nonlinear activation function as regularization. Only the first residual block does not contain the first batch normalization, ReLU and dropout layer since these layers are already applied after the initial convolution layer. We rearrange layers from the initial architecture\u003c/p\u003e\u003cp\u003e\u003csup\u003e9\u003c/sup\u003e and extend it with Squeeze and Excite (SE) blocks. This operation helps to weight the channel-wise information. Downsampling residual blocks consist in the residual skip connection (dashed lines) of a Max Pooling operation followed by a convolutional layer with filter length 1 to match the dimensions with the main branch for the summation. The remaining skip connections (full lines) do not contain any operations since input and output dimensions of the residual block are equal.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/213899bdbd9ef3594f64f861.png"},{"id":25677450,"identity":"d6678739-2435-4ffa-89ad-0757efc72055","added_by":"auto","created_at":"2022-08-25 18:28:14","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":261228,"visible":true,"origin":"","legend":"\u003cp\u003eReceiver operating characteristics and precision-recall curves\u003c/p\u003e\u003cp\u003eTop row: Temporal test split. Bottom row: Random test split. Left column: Receiver operating characteristics curve. Right Column: Precision-recall curve. We show the curve with the median C-statistic or AP respectively as a solid line over-trained models with 10 seeds of our ensemble-based model. The shaded area shows the min and max values for all 10 models. The dashed lines indicate the curve corresponding to a random guess for the class in the given color according the legend. For the Precision-Recall curve this is a horizontal line at the value (number of positive examples)/(number of all examples). Note that the worst-case curves are overlapping for NSTEMI, STEMI and MI in the precision recall curves due to the small fraction of positives (see dashed line close to the bottom). In the Precision-Recall curves the grey curves depict iso-F1 curves. Each curve is class \u003cem\u003evs\u003c/em\u003e all, where MI specifically is STEMI and NSTEMI \u003cem\u003evs\u003c/em\u003e Control.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/3b3ebe20643e4568518ebedc.png"},{"id":25677612,"identity":"61eea8cb-ee27-4c49-b7c1-6467a287d572","added_by":"auto","created_at":"2022-08-25 18:33:14","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":841090,"visible":true,"origin":"","legend":"\u003cp\u003eFour representative STEMIs correctly classified with high probability\u003c/p\u003e\u003cp\u003eFour Grad-CAM plots of STEMIs correctly classified with high probability, highlighting the parts in the ECG that the model focuses on for its prediction. Gradients corresponding to the activations in the first convolutional layer of the neural network are averaged to get the proportional importance of each channel, which is then used to compute a proportional mean of the activations. Positive values obtained were plotted as purple disks overlaid on top of the ECG, with size proportional to its magnitude.\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"6.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/5cb1573d0c2187d2685852cd.png"},{"id":25677449,"identity":"7c40e3ef-6917-47d1-9a3c-6a15acc4efd6","added_by":"auto","created_at":"2022-08-25 18:28:14","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":952298,"visible":true,"origin":"","legend":"\u003cp\u003eFour representative NSTEMIs correctly classified with high probability\u003c/p\u003e\u003cp\u003eFour Grad-CAM plots of NSTEMIs correctly classified with high probability, highlighting the parts in the ECG that the model focuses on for its prediction. Gradients corresponding to the activations in the first convolutional layer of the neural network are averaged to get the proportional importance of each channel, which is then used to compute a proportional mean of the activations. Positive values obtained were plotted as purple disks overlaid on top of the ECG, with size proportional to its magnitude.\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/0ebd56b891f48223d4e1fe0f.png"},{"id":25677613,"identity":"b2f81d67-2237-45bf-acbf-c28a64f9abac","added_by":"auto","created_at":"2022-08-25 18:33:17","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":560275,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/58fef3df-73ee-423a-b54a-bd1bc6e3b0f6.pdf"},{"id":25677075,"identity":"b39902ad-7322-49fc-b6b9-782358662bd4","added_by":"auto","created_at":"2022-08-25 18:23:14","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":564675,"visible":true,"origin":"","legend":"","description":"","filename":"AI4ECGSTEMINSTEMIsupplementupdated20220816.docx","url":"https://assets-eu.researchsquare.com/files/rs-1941398/v1/fd1bb0fc2a1de09f0e88cf0d.docx"}],"financialInterests":"Competing interest reported. J.S. reports stock ownership in companies providing services to Itrim, Amgen, Janssen, Novo Nordisk, Pfizer, Takeda, AstraZeneca and Vifor Pharma, outside the submitted work. S.G. is employed at Sence Research AB. The other authors report no conflicts of interest.","formattedTitle":"Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients","fulltext":[{"header":"Background","content":"\u003cp\u003eEmergency department care costs are high\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e and rising\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e in developed societies. Based on limited data in a chaotic environment, emergency medicine doctors must make quick decisions about patients\u0026rsquo; probabilities for many diagnoses and risks. Diagnostic error is commonplace,\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e and there is a desperate need for emergency department decision support systems.\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eThe emergency department handling of myocardial infarctions is especially precarious. In the United States, myocardial infarctions are missed in the range of 10\u0026ndash;50,000 per year at emergency departments.\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e At the other end, less than half of those hospitalized for a suspected myocardial infarction are eventually diagnosed with the condition.\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e The electrocardiogram (ECG) can reveal the large ST-elevation myocardial infarctions (STEMIs), but the non-ST-elevation myocardial infarctions (NSTEMIs) are often inconspicuous to the human eye on the ECG, and rely on other means of diagnosis.\u003c/p\u003e \u003cp\u003eArtificial intelligence (AI) has shown much recent promise in ECG classification,\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e for common ECG diagnoses\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e as well as for traits with unclear ECG diagnostic criteria or those not usually thought of as ECG diagnoses.\u003csup\u003e\u003cspan additionalcitationids=\"CR11\" citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e Even ECGs that appear normal to the human eye carries information useful for AI algorithms.\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e AI is very promising in the diagnosis of myocardial infarction,\u003csup\u003e\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e but many studies have used limited\u003csup\u003e\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e or managed\u003csup\u003e\u003cspan additionalcitationids=\"CR15 CR16 CR17\" citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e datasets. We postulate that AI ECG interpretation may be useful also in the most important population \u0026ndash; all-comers to emergency departments.\u003c/p\u003e \u003cp\u003eUsing a large real-world sample of patients presenting at emergency departments, we developed and validated a deep learning model for diagnosis of NSTEMI and STEMI on the presenting ECG.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eOf the included total 492,226 ECGs from the emergency department visit, 5,416 (1.1%) were recorded with an NSTEMI, 1,818 (0.4%) with a STEMI and 484,992 (98.5%) without a myocardial infarction. Clinical characteristics of the study sample are presented in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and stratified for the data splits in Supplementary Table\u0026nbsp;2, and the patients\u0026rsquo; age and admission date distributions are shown in Supplementary Figs.\u0026nbsp;1 and 2.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eClinical characteristics of the study sample\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eControl\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eNSTEMI\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eSTEMI\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNumber of ECGs\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e484,992\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5,416\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1,818\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eClinical characteristics at ED visit\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAge\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e65.0 (47.0,78.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e71.0 (62.0,81.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e66.0 (57.0,77.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMale\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e47.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e65.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e73.7\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eYear\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2013 (2010,2015)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2013 (2011,2015)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2013 (2011,2015)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003ePresenting complaint\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eChest pain\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e21.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e71.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e70.1\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDifficulty breathing\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e14.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e12.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5.9\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDizziness\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e7.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHeart problems\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.9\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCirculatory arrest\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3.7\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eCardiovascular diagnoses prior to ED visit*\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMyocardial infarction\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e8.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e26.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e16.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnstable angina\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e11.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6.4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eIschemic heart disease\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e20.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e43.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e23.8\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eStroke\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e9.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e11.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e7.4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003ePeripheral artery disease\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e7.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e12.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e7.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHeart failure\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e20.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e9.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAtrial fibrillation\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e19.9\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e8.7\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCardiovascular disease\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e59.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e70.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e52.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eDrugs with \u0026gt;\u0026thinsp;=\u0026thinsp;1 dispensation within one year prior to ED visit\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eRenin-angiotensin system inhibitors\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e33.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e51.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e36.9\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCalcium channel blockers\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e17.9\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e29.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e21.1\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBeta-receptor blockers\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e36.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e50.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e33.9\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMineralocorticoid receptor antagonists\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3.0\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDiuretics\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e27.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e34.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e19.4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAnti-arrhythmic drugs\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eStatins\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e23.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e41.9\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e24.3\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAnticoagulants\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e12.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e8.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAntiplatelets\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e27.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e48.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e28.9\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eCardiac enzymes within ED visit or coronary care unit hospitalization**\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTroponin I measured\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e8.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e13.8\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMax troponin I (ng/L)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e29.7 (29.7,40.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2700.0 (610.0,10150.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e18100.0 (3100.0,47150.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTroponin T measured\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e38.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e87.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e87.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMax troponin T (ng/L)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e9.9 (5.0,19.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e266.0 (93.0,814.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1900.0 (519.0,4680.0)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNTproBNP measured\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e8.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e21.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e17.1\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMax NTproBNP (ng/L)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1340 (286,4300)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2940 (784,8720)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3120 (772,9220)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eMain diagnoses at admission ***\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMyocardial infarction\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e94.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e97.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnstable angina\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eIschemic heart disease\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e94.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e97.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eStroke\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.7\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003ePeripheral artery disease\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.3\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHeart failure\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.9\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAtrial fibrillation\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5.9\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.3\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCardiovascular disease\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e18.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e96.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e98.4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eMortality after coronary care unit admission\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e30-day all-cause death\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e10.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eIn-hospital all-cause death\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e9.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003ctfoot\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\"\u003ePatient characteristics of coronary care unit admissions included in the study, by control/NSTEMI/STEMI outcome. Data are medians (quartiles) or percent.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\"\u003e*Prevalent disease based on any diagnosis position, inpatient and outpatient specialist care combined.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\"\u003e**Combining troponin and high-sensitive troponin laboratory measurements from regional laboratory databases and the SWEDEHEART database. Maximum of all available measurements within the time window reported. Done separately for troponin I and T.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"4\"\u003e***Primary diagnosis from inpatient specialist care. STEMI, ST-elevation myocardial infarction; NSTEMI, non-ST-elevation myocardial infarction; ED, emergency department; NTproBNP, N-terminal pro-B-type natriuretic peptide.\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tfoot\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eWe conducted a preliminary analysis with a subgroup of patients that were admitted the CCU only. From a preliminary analysis we identified that all model and training modifications to the original baseline model\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e were effective in some metrics and overall contributed to the performance of our model. We identified two changes which were most important: the extension of our original dataset with the repeated recordings as data augmentation in the training data set, and the use of an ensemble-based model.\u003c/p\u003e\n\u003cp\u003eThe performance of our model in the two test datasets and a publicly available dataset, PTB-XL, is described in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e including model uncertainty over ten different initialization seeds. Additionally in Supplementary Table\u0026nbsp;4, we show bootstrapped data uncertainty with similar performance, details for this are in the Supplementary Methods. In the random test set, STEMIs could be discriminated with a C-statistic of 0.991 and the model had a Brier score of 0.001. For NSTEMIs, the model had a C-statistic of 0.832 and a Brier score of 0.008, with lower precision than STEMIs. The temporal test set, which contained data from emergency department visits that did not overlap in time of the development set, resulted in a C-statistic of 0.985 for STEMI and 0.867 for NSTEMI. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e illustrates receiver operating characteristics and precision-recall curves for multiple independently trained models on the two test splits. Calibration was acceptable (calibration plots in Supplementary Fig.\u0026nbsp;3, predicted probabilities in Supplementary Fig.\u0026nbsp;4).\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003cdiv class=\"colspec\" align=\"char\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003ctable id=\"Tab2\" style=\"width: 1025px;\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003ePerformance of the model in random, temporal and external test sets\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003cth style=\"height: 35px; width: 159.438px;\" align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n\u003cth style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n\u003cth style=\"height: 35px; width: 231px;\" align=\"left\"\u003e\n\u003cp\u003eRandom\u003c/p\u003e\n\u003c/th\u003e\n\u003cth style=\"height: 35px; width: 238px;\" align=\"left\"\u003e\n\u003cp\u003eTemporal\u003c/p\u003e\n\u003c/th\u003e\n\u003cth style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003ePTB-XL\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 140px; width: 159.438px;\" rowspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eN (%)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eControl\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e88,742 (98.9)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e27,561 (98.7)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e200 (72.7)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e193 (0.2)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e108 (0.4)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e25 (27.3)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eNSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e820 (0.9)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e263 (0.9)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e1,013 (1.1)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e371 (1.3)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 140px; width: 159.438px;\" rowspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eC-statistic (\u0026uarr;)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eControl\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.863 (0.860\u0026ndash;0.870)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.903 (0.896\u0026ndash;0.909)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.962 (0.953\u0026ndash;0.97)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.991 (0.988\u0026ndash;0.994)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.985 (0.983\u0026ndash;0.987)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.932 (0.904\u0026ndash;0.959)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eNSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.832 (0.828\u0026ndash;0.841)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.867 (0.859\u0026ndash;0.876)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.863 (0.860\u0026ndash;0.870)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.903 (0.896\u0026ndash;0.909)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 140px; width: 159.438px;\" rowspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eAP (\u0026uarr;)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eControl\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.998 (0.998\u0026ndash;0.998)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.998 (0.998\u0026ndash;0.998)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.955 (0.934\u0026ndash;0.972)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.692 (0.641\u0026ndash;0.727)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.744 (0.716\u0026ndash;0.773)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.954 (0.935\u0026ndash;0.971)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eNSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.160 (0.134\u0026ndash;0.168)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.184 (0.144\u0026ndash;0.214)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.330 (0.307\u0026ndash;0.347)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.466 (0.42\u0026ndash;0.484)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 140px; width: 159.438px;\" rowspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eBrier (\u0026darr;)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eControl\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.009 (0.009\u0026ndash;0.009)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.009 (0.009\u0026ndash;0.010)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.145 (0.126\u0026ndash;0.158)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.001 (0.001\u0026ndash;0.001)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.002 (0.002\u0026ndash;0.002)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.184 (0.167\u0026ndash;0.196)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eNSTEMI\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.008 (0.008\u0026ndash;0.008)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.008 (0.008\u0026ndash;0.009)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eMulticlass\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.018 (0.018\u0026ndash;0.018)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.019 (0.019\u0026ndash;0.020)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 35px;\"\u003e\n\u003ctd style=\"height: 35px; width: 159.438px;\" align=\"left\"\u003e\n\u003cp\u003eECE (\u0026darr;)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 135.562px;\" align=\"left\"\u003e\n\u003cp\u003eMulticlass\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 231px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.417 (0.416\u0026ndash;0.418)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 238px;\" align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.415 (0.415\u0026ndash;0.417)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 35px; width: 225px;\" align=\"left\"\u003e\n\u003cp\u003e0.277 (0.257\u0026ndash;0.297)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003ctfoot\u003e\n\u003ctr style=\"height: 65.6562px;\"\u003e\n\u003ctd style=\"height: 65.6562px; width: 989px;\" colspan=\"5\"\u003eResults of the model in the two test sets and the publicly available PTB-XL dataset as comparison in the rightmost column. Sample size is given by each testset and outcome label together with proportion out of the given testset. Performance metrics are given as median (minimum-maximum) over ten trained models initiated with different seeds; each of the ten models is an ensemble consisting of five model members. Arrows indicate direction of better performance. We compute the metrics as class \u003cem\u003evs.\u003c/em\u003e all. Note that the class MI merges the classes STEMI and NSTEMI in one class. ECE is for multi-class calibration instead of class-wise calibration. STEMI, ST-elevation myocardial infarction; NSTEMI, non-ST-elevation myocardial infarction; AP, Average Precision or equivalently Area Under the Precision-Recall curve; ECE, Expected Calibration Error.\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tfoot\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo ensure that our model did not use any proxies for its predictions, we stratified the test datasets according to different possible confounders. Supplementary Figs.\u0026nbsp;5 (C-statistic) illustrate results stratified according to test set records 1) without any filter applied based on ST-elevation label noise, 2) ST-elevation filter corresponding to results in main table (used in all following test set subsets); 3) stricter ST-elevation where \u0026ldquo;possible STEMI\u0026rdquo; (see Supplementary Methods) were removed from both STEMI and NSTEMI cases, 3) age tertiles, 4) sex, 5) ECG collected at the same day as the admission or not, 6) emergency department at Karolinska Hospital (main source of data) or another emergency department in the Stockholm region, 7) patients attending the CCU only, 8) ECGs recorded using the most common machine type (MAC55) or not, 9) ECGs recorded using the most common software (v237) or not. Apart from analyses restricting the test set controls to CCU controls only, we observed no major changes to the model performances, indicating that the predictions are not driven by these potential confounders.\u003c/p\u003e\n\u003cp\u003eAs an additional test, we trained a model with only ECG traces as input, omitting age and sex. The results were almost identical to our main results indicating that the explicit addition of age and sex to the model was not crucial for our results. Furthermore, we compare our results with a model trained on data from patients at the coronary care unit alone. This dataset has 16,628 ECGs, i.e. a subset of 3.4% of our current dataset, omitting most of the control patients which were not admitted to the coronary care unit. The results from this model show that the larger dataset including all controls is important for our performance.\u003c/p\u003e\n\u003cp\u003eInspecting Grad-CAM plots yielded new insights. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e illustrates four STEMIs correctly classified with high probability. In panels A and B the model had focused on the ST-segment, where a human would look. In panels C and D, the model also used the down-sloping part of the T-wave, where a human would not focus when diagnosing a STEMI. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e illustrates four correctly classified NSTEMIs. In all panels, the model had focused on the ST-segment, but had also used the last part of the T-wave, which a human would not.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCharacteristics of hospitalizations with misclassified ECGs are described in Supplementary Table\u0026nbsp;3. Among the misclassified ECGs, those misclassified as STEMI more often had perimyocarditis, valvular disease or cardiomyopathy; those misclassified as NSTEMI more often had valvular or congenital heart disease, pulmonary edema, gastric ulcer or dental traits.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn a very large sample of all-comers to emergency departments, we developed and validated an AI model that can classify STEMI and NSTEMI \u003cem\u003eversus\u003c/em\u003e non-myocardial infarctions using routine 10-second ECGs.\u003c/p\u003e \u003cp\u003eThe model achieved excellent performance for both STEMI and NSTEMIs. Doctors\u0026rsquo; ECG interpretation is often imprecise, with a reported accuracy of 0.69 overall for practicing physicians and 0.75 for cardiologists in controlled test settings,\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e with similar numbers reported for STEMIs.\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u003c/sup\u003e Performance of doctors outside of such standardized settings is unknown but not likely better. Diagnosis by humans of NSTEMIs from ECGs is almost by definition futile and has seen very little research. Importantly, we do not observe a worse performance in the temporal test set when compared to the random test set. This suggests that our model works on data outside of the training data. In addition, it performs well in the discrimination of STEMIs in the external PTB-XL database test set.\u003c/p\u003e \u003cp\u003eThis study is clinically important as it uses the most relevant sample possible. These consecutive all-comers represent the real-world ECG experience for emergency doctors, with ECGs - especially among the non-infarctions - that are far from the very clear specimens in managed online databases or heavily curated samples. Notably, in Sweden today and during the study period, pre-hospital ECGs are sent to coronary care units for immediate diagnosis, so the obvious STEMI cases usually bypass the emergency department and transfer straight to the coronary intervention lab upon arrival to hospital, rendering the STEMIs in the present study the less obvious cases and the walk-ins. Hence, these are cases in great need of decision support. Further, we did not exclude difficult cases, comorbidities, or previous myocardial infarctions (except for technical reasons, we removed potentially linked hospitalizations for the same myocardial infarction, and LBBBs, which cannot \u003cem\u003eper se\u003c/em\u003e identify an acute myocardial infarction from a single ECG, but need a prior ECG for comparison).\u003c/p\u003e \u003cp\u003eOther studies using deep learning models have also shown good performance but lower than our model\u0026rsquo;s performance when tested in representative settings,\u003csup\u003e\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e with reports of very good and similar to our model\u0026rsquo;s performance in managed settings.\u003csup\u003e\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e Our study differs from the previous literature in that it is a multicenter study using real-world data of consecutive patients with very few exclusions, and with output labeling by many doctors. Descriptions of the clinical setting and the controls are sometimes unclear.\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e Past studies have seldom investigated NSTEMIs.\u003c/p\u003e \u003cp\u003eOur model performed slightly better in younger than older patients for NSTEMI classification, and slightly better for STEMI classification in men than in women. Younger NSTEMI patients might have fewer underlying diseases, potentially making the prediction simpler. Men contribute to around 2/3 of the myocardial infarctions, potentially making male infarctions easier to learn given more data. A stricter filter for label noise based on a mismatch between original labels and updated ST-elevation annotations shows that a stricter filter improved the result for STEMI whereas the opposite was seen for NSTEMI, but the differences were minor. Overall, the results were comparable across test set subsets and some differences in prediction results might be due to pure chance. A notable difference in the analyses restricting the test set controls to CCU controls only may be due to more underlying heart conditions in those controls, and low power.\u003c/p\u003e \u003cp\u003eThe Grad-CAM plots in Figs.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e6\u003c/span\u003e and \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e7\u003c/span\u003e provide important insights. The model recognizes the same ST-segment features that humans would. But the model also finds features that are novel, or that are imperceptible to the human eye. This shows an interesting way forward. We give the AI ECGs and the label, then the AI teaches us novel ways to read the ECGs. Variants of such model evaluation can likely give useful clinical and pathophysiological clues in many medical fields.\u003c/p\u003e \u003cp\u003eOur model\u0026rsquo;s misclassifications as STEMI follows known clinical and machine learning patterns, with perimyocarditis as an important impostor.\u003csup\u003e\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e The conditions over-represented in those misclassified as NSTEMI were logical to some extent, such as valvular or congenital heart disease and pulmonary edema; the gastric ulcer and dental traits more surprising.\u003c/p\u003e \u003cp\u003eSome important limitations are worth mentioning. Our dataset contains some label noise. The label was determined at discharge from the coronary care unit or emergency room when the whole care episode could be summarized. The ECGs in the test sets of this study may hence not always be the ones guiding the final diagnosis. We mitigate that to some extent by using multiple ECGs if available within the day before admission in the training set, but not in the test sets. On the other hand, the hindsight allows for more stable labels for the episode as a whole, which is the ultimate goal for the classification. More information about the handling of label noise is provided in the supplementary materials. Another important limitation is the lack of an external validation sample. We did hold out the 10% of the patients with their first admission in 2016 as a temporal test set; many circumstances in that set would be similar to those in the earlier training set, but a restructuring of the Stockholm region emergency department logistics which is our main data source during the data collection period did change the composition of the sample. Furthermore, we make use of the publicly available PTB-XL dataset which does include data containing STEMI but not NSTEMI. In this dataset, our model achieves good discriminative performance. No publicly available data repositories contain ECGs with NSTEMI labels to test this externally. While the calibration of our model was better than that of comparable models, there is still room for improvement; calibration is indeed an underappreciated property in general. We did not consider transferring learned features, only model architecture, from a previous study.\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e An exploration of potential improvements in model convergence speed and final performance boost by pre-training on a different ECG classification task with a dataset in a different context may be useful, but may also introduce model biases from the other dataset. Lastly, we did not compare the performance of this ECG model to troponin-based or other methods of diagnosing myocardial infarction. Ultimately, clinical usefulness must be evaluated in a randomized trial.\u003c/p\u003e \u003cp\u003eIn conclusion, we developed and validated a deep learning model with excellent performance in discriminating between NSTEMI, STEMI and controls on the presenting ECG of a large real-world sample of general emergency department patients. Considering the high and rising emergency department care costs and the high numbers of missed myocardial infarctions at emergency departments, our model could be of clinical value for ECG decision support, with promise of further performance development.\u003c/p\u003e "},{"header":"Methods","content":"\u003cp\u003eSample\u003c/p\u003e \u003cp\u003e We utilized a consecutive sample of adult all-comer patients attending emergency departments in the Stockholm region between 2007 and 2016, for whom a routine ECG was obtained upon their presenting complaint. Details of the data sources are described in the Supplementary Methods. The procedure and criteria used to define the study sample are described in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e1\u003c/span\u003e, with available exposure and outcome data described below and in Supplementary Methods. In total, 217,667 patients had at least one registered emergency department within the study period with at least one valid ECG recording within 1 day of visit. The emergency department visits were either followed by a coronary care unit (CCU) admission (NSTEMI/STEMI/control) or had no subsequent CCU admission (control only). After applying the sequence of filters described in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e1\u003c/span\u003e to ensure inclusion of at-event before-treatment ECGs, as well as confirming the outcome label, 214,250 patients with 492,226 ECGs were available for analysis, representing a total of 12,328 CCU admissions and 412,980 non-CCU visits. Out of these ECGs, 67,137 exams were repeated recordings, i.e., the same patient had multiple ECG exams in the same visit (used in training the model only). The study was approved by the Swedish Ethical Review Agency, application number 2020\u0026thinsp;\u0026minus;\u0026thinsp;01654. Informed consent was waived in this study by all applicable ethics review boards, i.e. the Region Stockholm Ethics Review Board and the National Ethics Review Authority. All methods were performed in accordance with the relevant guidelines and regulations.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eExposures and outcomes\u003c/p\u003e \u003cp\u003eHigh-quality data on the exposures and outcomes were available for all included patients from discharge records from the emergency departments, electronic health records, from linked hospitalizations, and from the SWEDEHEART registry with patient records joined on the personal identifier number of the patient. Data sources and definitions used are described in Supplementary Methods and Supplementary Table\u0026nbsp;1. As exposures, we used digital ECG data, age and sex, as in a previous study.\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e Standard 10-second 12-lead ECG recordings sampled at 250 to 500Hz were used; 8 leads were used in the present study as 4 of the standard leads are linear combinations of these 8 and are hence redundant.\u003c/p\u003e \u003cp\u003eFor the mutually exclusive outcome labels NSTEMI/STEMI/control status, trained in a single model, we used the high-quality SWEDEHEART registry (CCU admissions). In addition, diagnoses from the Swedish in-patient and cause-of-death registries were used to confirm a myocardial infarction (I21) for cases, or absence of a myocardial infarction for controls. The SWEDEHEART Riks-HIA labels are the decision of a discharging physician that followed the entire patient journey during the hospitalization; this physician had access to other exams besides the ECG, such as coronary angiograms in some patients, and blood testing in all patients. This label is the accepted gold standard for all research using SWEDEHEART data; efforts to further minimize label noise are described in Supplementary Methods. We only included cases with complete data on the exposures and outcomes.\u003c/p\u003e \u003cp\u003eMachine learning methods\u003c/p\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eTraining and validation datasets\u003c/h2\u003e \u003cp\u003eThe patients fulfilling the inclusion criteria were divided in 70%/30% splits with records from the same patient always put in the same split. The 70% split was used for training and developing the model and the 30% split for testing the model performance. The 30% test split was further divided into two splits containing 20% and 10% of the complete data, to allow us to test the model in two different scenarios. The 10% split contains patients with a first recorded admission date after 2016-01-01 or later, hence temporally separated from the training data set which included patients with a first recorded admission before this date. This way the 10% test split can be used to assess the model susceptibility to shifts and trends that change with time. We denote this split the \u003cem\u003etemporal test split\u003c/em\u003e. The other 20% split was sampled at random from entries with an admission date before 2016-01-01, which is the same period the 70% training split is sampled from. We denote this split the \u003cem\u003erandom test split\u003c/em\u003e. The process is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003e and patient characteristics in the sets are presented in Supplementary Table\u0026nbsp;2. Our temporal test set can be considered the best possible external validation available for the NSTEMI cases since there exists no other publicly available data set available containing ECGs and NSTEMI annotations.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAn additional external validation dataset with 75 acute myocardial infarction cases with ST-elevation and 200 randomly selected controls was manually curated from the PTB-XL database.\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e The PTB-XL is a publicly available database of 21,837 10-second 12-lead ECGs annotated with 71 different ECG statements, including cases of myocardial infarction. The inclusion criteria of the PTB-XL test set used in this study is described in the Supplementary Methods.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eData pre-processing\u003c/h2\u003e \u003cp\u003eData pre-processing steps are described in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e. In addition to the ECG tracings, we limited ourselves to adding age and sex, to make the model as transportable and unbiased as possible. The output was the probabilities of the three mutually exclusive outcome classes: NSTEMI/STEMI/control.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eModel architecture\u003c/h2\u003e \u003cp\u003eOur Deep Neural Network (DNN) model architecture is an extension of a previous model,\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e for which the DNN was trained to detect six types of ECG abnormalities.\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e We used a neural network based on convolutional layers similar to a residual network (ResNet) that is commonly used in image classification, but adapted here to unidimensional signals. This architecture allows DNNs to be efficiently trained by including skip connections. We adapt a modification of layer arrangements within the residual block and a skip connection which is shown to be more effective.\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e The model architecture is depicted and described in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAge and sex were passed through a fully connected layer and concatenated with the flattened output of the residual blocks. The resulting features were used in the final linear classification layer which outputs the model prediction. The output of the trained model was the probabilities of the three mutually exclusive outcome classes NSTEMI/STEMI/control.\u003c/p\u003e \u003cp\u003eEnsembles of neural network models improve predictive performance,\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e and lead to better calibrated models. Therefore, we expanded our model as an ensemble of five model members. Each of the five individual model members was trained from scratch from different parameter initialization but with the same training data to obtain different output logits. These logits were averaged to obtain the final prediction. A detailed description of all model hyperparameters is provided in the Supplementary Methods.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eModel training and validation\u003c/h2\u003e \u003cp\u003eThe model was trained by minimizing the cross-entropy loss for 100 epochs. Details about the training hyperparameter and regularization terms are described in the Supplementary Methods. In addition to the training dataset, we made use of the repeated ECG exams of patients who had multiple exams at an emergency department visit for training but not for validation and testing. We consider this as a form of data augmentation since these exams have the same label but are recorded at different times and therefore with a slightly different state of the patient and possibly different placement of the ECG leads. For validation and testing, only the first recorded ECG of an admission was used. Details are in the Supplementary Methods. We set aside a total of 10% of the training data for validation. Note that we improve the training procedure and model architecture from a previous study\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e with established methods.\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e The details for the model and training improvements are described in Supplementary Methods.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eHyperparameter search\u003c/h2\u003e \u003cp\u003eWe conducted a search over 6 hyperparameters and their respective values. This gives a total of 2025 possible combinations. Due to the high number of possible combinations, we ran a random search for 345 combinations. The remaining model and training hyperparameters were fixed in this search since they have been proven efficient in the baseline architecture.\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e We evaluated the tested hyperparameters in the validation dataset. Note that during the search we only train a single model and not an ensemble of multiple models. Details of the hyperparameter search are outlined in the Supplementary Methods.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003eModel calibration\u003c/h2\u003e \u003cp\u003ePerformance of prediction models is often quantified using the C-statistic that measures discrimination, i.e. the model\u0026rsquo;s ability to assign higher risk estimates/probabilities to patients who will experience the event than patients who will not. While discrimination is important, it tells us nothing about the reliability of the probability estimates. Calibration, i.e. if the model\u0026rsquo;s probability estimates reflect the ground truth empirical class frequencies, is a more important property of the model for clinical use. One of our main concerns was model calibration, since modern DNN architectures are generally poorly calibrated.\u003csup\u003e\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e As metrics of model calibration, we focused on the Expected Calibration Error (ECE) and the Brier score estimated in the validation set. The ECE is the weighted absolute difference between the class membership and the estimated probability for that class averaged over 15 bins, while the Brier score measures the average squared error on the probability scale. Both metrics are applicable to both binary and multiclass problems. We visualized the calibration of our model using calibration plots, also called reliability diagrams.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eModel evaluation\u003c/h2\u003e \u003cp\u003eWe investigated possible patterns for our models correct and incorrect classifications of STEMIs and NSTEMIs. First, we used Grad-CAM plots to highlight the parts of the ECG that the model focuses on - meaning the sections of the input ECG where the model puts its most weight on - to make its predictions.\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e This method generates the visualization in two steps: In a forward pass step, it computes the activations of the neural network in a given intermediary layer (in our case, the first convolutional layer). And in a backward step, it computes the gradients corresponding to these activations. The gradients are averaged to get the proportional importance of each channel, which is then used to compute a proportional mean of the activations. Positive values obtained were plotted as purple disks overlaid on top of the ECG, with size proportional to its magnitude, to generate the visualization. One senior cardiologist (JS) inspected the Grad-CAM plots from the ten cases with the highest estimated probability for each myocardial infarction class and selected four illustrative plots per class.\u003c/p\u003e \u003cp\u003eWe tested the over/underrepresentation of inpatient and specialized outpatient care diagnoses at the time of the visit among correctly classified \u003cem\u003eversus\u003c/em\u003e misclassified patients with a predicted probability\u0026thinsp;\u0026gt;\u0026thinsp;0.5 in pairwise independent tests to find diagnoses where the model often made incorrect classifications.\u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eFunding\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eThe Kjell and M\u0026auml;rta Beijer Foundation, Anders Wikl\u0026ouml;f, the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, and Uppsala University. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eAcknowledgements\u003c/h2\u003e\n\u003ch3\u003eNon-author contributions\u003c/h3\u003e\n\u003cp\u003eWe thank David Widmann for the discussions and his input towards analyzing and improving the calibration of our model.\u003c/p\u003e\n\u003ch3\u003eRole of the funding source\u003c/h3\u003e\n\u003cp\u003eThe study was funded by The Kjell and M\u0026auml;rta Beijer Foundation, Anders Wikl\u0026ouml;f, The \u003cem\u003eWallenberg AI, Autonomous Systems and Software Program (WASP)\u003c/em\u003e funded by Knut and Alice Wallenberg Foundation, and Uppsala University. The computations were enabled by resources in project sens2020005 and sens2020598 provided by the Swedish National Infrastructure for Computing (SNIC) at UPPMAX, partially funded by the Swedish Research Council through grant agreement no. 2018-05973. The funders had no role in in study design; collection, analysis, and interpretation of data; writing of the report; or decision to submit the paper for publication.\u003c/p\u003e\n\u003ch3\u003eData Availability\u003c/h3\u003e\n\u003cp\u003eThe data that support the findings of this study are available from the Swedish Board of Health and Welfare and the included healthcare regions, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from corresponding author (J.S.) upon reasonable request and with permission of the Swedish Board of Health and Welfare and the included healthcare regions. All code together with the parameter/hyperparameter estimates of the presented models is available upon request.\u003c/p\u003e\n\u003ch3\u003eAuthor Contributions\u003c/h3\u003e\n\u003cp\u003eAll authors participated in the study design, interpretation of the data, and critically reviewing the paper. S.G, D.G, E.L, A.H.R, and J.S wrote the first draft, and M.J.H, and T.B.S. contributed to the writing of subsequent versions. Data was acquired and prepared for analysis by M.J.H, S.G, and A.H.R. Statistical analyses and preparation of tables and figures was performed by S.G, D.G, E.L, and A.H.R with support from J.S and T.B.S. M.J.H, J.S, T.B.S. were responsible for funding acquisition, project administration, and supervision. All authors had access to and could verify the underlying data.\u003c/p\u003e\n\u003ch3\u003eCompeting Interests\u003c/h3\u003e\n\u003cp\u003eJ.S. reports stock ownership in companies providing services to Itrim, Amgen, Janssen, Novo Nordisk, Pfizer, Takeda, AstraZeneca and Vifor Pharma, outside the submitted work. S.G. is employed at Sence Research AB. The other authors report no conflicts of interest.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eGalarraga JE and Pines JM. Costs of ED episodes of care in the United States. Am J Emerg Med. 2016;34:357\u0026ndash;65.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLane BH, Mallow PJ, Hooker MB and Hooker E. Trends in United States emergency department visits and associated charges from 2010 to 2016. Am J Emerg Med. 2020;38:1576\u0026ndash;1581.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMoonen PJ, Mercelina L, Boer W and Fret T. Diagnostic error in the Emergency Department: follow up of patients with minor trauma in the outpatient clinic. Scand J Trauma Resusc Emerg Med. 2017;25:13.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMedford-Davis L, Park E, Shlamovitz G, Suliburk J, Meyer AN and Singh H. Diagnostic errors related to acute abdominal pain in the emergency department. Emerg Med J. 2016;33:253\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWright B, Faulkner N, Bragge P and Graber M. What interventions could reduce diagnostic error in emergency departments? A review of evidence, practice and consumer perspectives. Diagnosis (Berl). 2019;6:325\u0026ndash;334.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSharp AL, Baecker A, Nassery N, Park S, Hassoon A, Lee MS, Peterson S, Pitts S, Wang Z, Zhu Y and Newman-Toker DE. Missed acute myocardial infarction in the emergency department-standardizing measurement of misdiagnosis-related harms using the SPADE method. Diagnosis (Berl). 2020;8:177\u0026ndash;186.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCaulfield CA and Stephens JR. Things We Do for No Reason: Hospitalization for the Evaluation of Patients with Low-Risk Chest Pain. J Hosp Med. 2018;13:277\u0026ndash;279.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSiontis KC, Noseworthy PA, Attia ZI and Friedman PA. Artificial intelligence-enhanced electrocardiography in cardiovascular disease management. Nature reviews Cardiology. 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRibeiro AH, Ribeiro MH, Paixao GMM, Oliveira DM, Gomes PR, Canazart JA, Ferreira MPS, Andersson CR, Macfarlane PW, Meira W, Jr., Schon TB and Ribeiro ALP. Automatic diagnosis of the 12-lead ECG using a deep neural network. Nat Commun. 2020;11:1760.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTison GH, Zhang J, Delling FN and Deo RC. Automated and Interpretable Patient ECG Profiles for Disease Detection, Tracking, and Discovery. Circ Cardiovasc Qual Outcomes. 2019;12:e005289.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCohen-Shelly M, Attia ZI, Friedman PA, Ito S, Essayagh BA, Ko WY, Murphree DH, Michelena HI, Enriquez-Sarano M, Carter RE, Johnson PW, Noseworthy PA, Lopez-Jimenez F and Oh JK. Electrocardiogram screening for aortic valve stenosis using artificial intelligence. Eur Heart J. 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaghunath S, Ulloa Cerna AE, Jing L, vanMaanen DP, Stough J, Hartzel DN, Leader JB, Kirchner HL, Stumpe MC, Hafez A, Nemani A, Carbonati T, Johnson KW, Young K, Good CW, Pfeifer JM, Patel AA, Delisle BP, Alsaid A, Beer D, Haggerty CM and Fornwalt BK. Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network. Nature medicine. 2020;26:886\u0026ndash;891.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAttia ZI, Noseworthy PA, Lopez-Jimenez F, Asirvatham SJ, Deshmukh AJ, Gersh BJ, Carter RE, Yao X, Rabinstein AA, Erickson BJ, Kapa S and Friedman PA. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. Lancet. 2019;394:861\u0026ndash;867.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu WC, Lin CS, Tsai CS, Tsao TP, Cheng CC, Liou JT, Lin WS, Cheng SM, Lou YS, Lee CC and Lin C. A Deep-Learning Algorithm for Detecting Acute Myocardial Infarction. \u003cem\u003eEuroIntervention\u003c/em\u003e. 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCho Y, Kwon JM, Kim KH, Medina-Inojosa JR, Jeon KH, Cho S, Lee SY, Park J and Oh BH. Artificial intelligence algorithm for detecting myocardial infarction using six-lead electrocardiography. Sci Rep. 2020;10:20495.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMakimoto H, Hockmann M, Lin T, Glockner D, Gerguri S, Clasen L, Schmidt J, Assadi-Schmidt A, Bejinariu A, Muller P, Angendohr S, Babady M, Brinkmeyer C, Makimoto A and Kelm M. Performance of a convolutional neural network derived from an ECG database in recognizing myocardial infarction. Sci Rep. 2020;10:8445.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAl-Zaiti S, Besomi L, Bouzid Z, Faramand Z, Frisch S, Martin-Gill C, Gregg R, Saba S, Callaway C and Sejdic E. Machine learning-based prediction of acute coronary syndrome using only the pre-hospital 12-lead electrocardiogram. Nat Commun. 2020;11:3966.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao Y, Xiong J, Hou Y, Zhu M, Lu Y, Xu Y, Teliewubai J, Liu W, Xu X, Li X, Liu Z, Peng W, Zhao X, Zhang Y and Xu Y. Early detection of ST-segment elevated myocardial infarction by artificial intelligence with 12-lead electrocardiogram. Int J Cardiol. 2020;317:223\u0026ndash;230.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWagner P, Strodthoff N, Bousseljot RD, Kreiseler D, Lunze FI, Samek W and Schaeffter T. PTB-XL, a large publicly available electrocardiography dataset. Sci Data. 2020;7:154.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng CK and Stanley HE. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. 2000;101:E215-20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlkmim MB, Figueira RM, Marcolino MS, Cardoso CS, Pena de Abreu M, Cunha LR, da Cunha DF, Antunes AP, Resende AG, Resende ES and Ribeiro AL. Improving patient access to specialized health care: the Telehealth Network of Minas Gerais, Brazil. Bull World Health Organ. 2012;90:373\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe K, Zhang X, Ren S and Sun J. Identity Mappings in Deep Residual Networks. Computer Vision \u0026ndash; ECCV 2016. 2016:630\u0026ndash;645.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHansen LK and Salamon P. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1990;12:993\u0026ndash;1001.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBello I, Fedus W, Du X, Cubuk ED, Srinivas A, Lin T-Y, Shlens J and Zoph B. Revisiting resnets: Improved training and scaling strategies. Advances in Neural Information Processing Systems \u003cem\u003e34\u003c/em\u003e. 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuo C, Pleiss G, Sun Y and Weinberger KQ. On calibration of modern neural networks. \u003cem\u003eInternational Conference on Machine Learning\u003c/em\u003e. 2017:1321\u0026ndash;1330.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSelvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D and Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. \u003cem\u003eProceedings of the IEEE international conference on computer vision\u003c/em\u003e. 2017:618\u0026ndash;626.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCook DA, Oh SY and Pusic MV. Accuracy of Physicians' Electrocardiogram Interpretations: A Systematic Review and Meta-analysis. JAMA internal medicine. 2020;180:1461\u0026ndash;1471.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcCabe JM, Armstrong EJ, Ku I, Kulkarni A, Hoffmayer KS, Bhave PD, Waldo SW, Hsue P, Stein JC, Marcus GM, Kinlay S and Ganz P. Physician accuracy in interpreting potential ST-segment elevation myocardial infarction electrocardiograms. Journal of the American Heart Association. 2013;2:e000268.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSoares WE, 3rd, Price LL, Prast B, Tarbox E, Mader TJ and Blanchard R. Accuracy Screening for ST Elevation Myocardial Infarction in a Task-switching Simulation. West J Emerg Med. 2019;20:177\u0026ndash;184.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTanguay A, Lebon J, Brassard E, Hebert D and Begin F. Diagnostic accuracy of prehospital electrocardiograms interpreted remotely by emergency physicians in myocardial infarction patients. Am J Emerg Med. 2019;37:1242\u0026ndash;1247.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-1941398/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1941398/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eMyocardial infarction diagnosis is a common challenge in the emergency department. In managed settings, deep learning-based models and especially convolutional deep models have shown promise in electrocardiogram (ECG) classification, but there is a lack of high-performing models for the diagnosis of myocardial infarction in real-world scenarios. We aimed to train and validate a deep learning model using ECGs to predict myocardial infarction in real-world emergency department patients.\u003c/p\u003e \u003cp\u003eWe studied emergency department patients in the Stockholm region between 2007 and 2016 that had an ECG obtained because of their presenting complaint. We developed a deep neural network based on convolutional layers similar to a residual network. Inputs to the model were ECG tracing, age, and sex; and outputs were the probabilities of three mutually exclusive classes: non-ST-elevation myocardial infarction (NSTEMI), ST-elevation myocardial infarction (STEMI), and control status, as registered in the SWEDEHEART and other registries. We used an ensemble of five models.\u003c/p\u003e \u003cp\u003eAmong 492,226 ECGs in 214,250 patients, 5,416 were recorded with an NSTEMI, 1,818 a STEMI, and 485,207 without a myocardial infarction. In a random test set, our model could discriminate STEMIs/NSTEMIs from controls with a C-statistic of 0.991/0.832 and had a Brier score of 0.001/0.008. The model obtained a similar performance in a temporally separated test set, and achieved a C-statistic of 0.985 and a Brier score of 0.002 in discriminating STEMIs from controls in an external test set.\u003c/p\u003e \u003cp\u003eWe developed and validated a deep learning model with excellent performance in discriminating between control, STEMI, and NSTEMI on the presenting ECG of a real-world sample of the important population of all-comers to the emergency department. Hence, deep learning models for ECG decision support could be valuable in the emergency department.\u003c/p\u003e","manuscriptTitle":"Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-08-25 18:23:12","doi":"10.21203/rs.3.rs-1941398/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2022-09-23T16:29:31+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-09-10T09:18:18+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-09-10T08:00:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"9db7bfc5-3c9a-4d46-8877-2475d059b7d6","date":"2022-08-31T10:47:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"9ae3a135-d8cb-4dc0-b439-cce736564c37","date":"2022-08-31T05:10:02+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2022-08-31T02:51:27+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2022-08-31T02:45:59+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2022-08-23T15:40:54+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2022-08-23T15:36:47+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2022-08-08T13:08:05+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"12c4bd38-80f8-4c61-9ed5-cd018fa0796f","owner":[],"postedDate":"August 25th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2022-11-11T17:59:23+00:00","versionOfRecord":[],"versionCreatedAt":"2022-08-25 18:23:12","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1941398","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1941398","identity":"rs-1941398","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}