The Virtual Cardiologist: Three Deep Learning Pipelines in an Inexpensive Portable Device and Web/Mobile Application for Rapid Cardiovascular Diagnosis and Clinical Decision-Making

Background Heart disease is the foremost cause of death worldwide, and specialty care such as cardiologists and expensive Doppler echocardiograms are scarce in rural areas. There is significant variability among physicians in interpreting heart sounds and 2D echocardiograms from more basic, widely available tools like stethoscopes and point-of-care ultrasound devices (POCUS), leading to treatment delays. Methods 26 Deep-learning models were tested to optimally [a] stratify severity of aortic stenosis from POCUS 2D echocardiogram images [b] estimate cardiac ejection fraction (heart failure) from POCUS 2D echocardiogram videos and [c] identify pathologic heart murmurs using digital stethoscope recordings. The top performing model for each was integrated into a portable, inexpensive Raspberry-Pi device (Virtual Cardiologist), mobile application, and website. Each model was tested on two independent datasets to ensure accuracy. The heart murmur algorithm was also tested on 323 patients by recording their heart sounds via a custom and digital stethoscope in an outpatient setting for clinical validation. Results The three top-performing models outperformed cardiologists and previous AI methods. The aortic stenosis model (AUC=0.89) utilized EfficientNet, a CNN with compound scaling built on a pre-trained computational backbone. The ejection fraction model (EchoSwin) incorporated Microsoft Swin transformers modified for video input (MAE=5.6). The heart murmur model employed an LSTM Autoencoder algorithm (AUC=0.92). Conclusion The Virtual Cardiologist can serve as an accurate tool for patient assessment and triage for referral to tertiary care in remote areas with limited specialty care. This is the first tool of its kind for detection of multiple cardiovascular morbidities.