Detection of dementia on raw voice recordings using deep learning: A Framingham Heart Study

Background Identification of reliable, affordable, and easy-to-use strategies for detection of dementia are sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available. Methods and findings We used 1264 digital voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 minutes in duration, on average, and contained at least two speakers (participant and clinician). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the raw audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that were non-demented (NDE (NC+MCI)) and DE. Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740±0.017, mean balanced accuracy of 0.647±0.027, and mean weighted F1-score of 0.596±0.047 in predicting cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805±0.027, mean balanced accuracy of 0.743±0.015, and mean weighted F1-score of 0.742±0.033 in predicting cases with DE from those with NC. For the task related to classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734±0.014, mean balanced accuracy of 0.675±0.013, and mean weighted F1-score of 0.671±0.015. The CNN model achieved a mean AUC of 0.746±0.021, mean balanced accuracy of 0.652±0.020, and mean weighted F1-score of 0.635±0.031 in predicting cases with DE from those who were NDE. Conclusion This proof-of-concept study demonstrates the potential that raw audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.