Street images classification according to COVID-19 risk in Lima, Peru: A convolutional neural networks analysis

ABSTRACT Background During the COVID-19 pandemic, convolutional neural networks (CNNs) have been used in clinical medicine (e.g., to classify chest X-rays for COVID-19 diagnosis). Whether CNNs could also inform the epidemiology of COVID-19 analysing street images has been understudied, though it could identify high-risk places and relevant features of the built environment. We trained CNNs to classify bus stops (Lima, Peru) into moderate or extreme COVID-19 risk. Methods We used five images per bus stop. The outcome label (moderate or extreme) for each bus stop was extracted from the local transport authority. We used transfer learning and updated the output layer of five CNNs: NASNetLarge, InceptionResNetV2, Xception, ResNet152V2, and ResNet101V2. We chose the best performing network which was further tuned to increase performance. Results There were 1,788 bus stops (1,173 moderate and 615 extreme), totalling 8,940 images. NASNetLarge outperformed the other CNNs except in the recall metric for the extreme label: 57% versus 59% in NASNetLarge and ResNet152V2, respectively. NASNetLarge was further tuned and reached: training loss of 0.50; training accuracy of 75%; precision, recall and F1 score for the moderate label of 80%, 83% and 82%, respectively; these metrics for the extreme label were 65%, 51% and 63%. Conclusions CNNs has the potential to accurately classify street images into levels of COVID-19 risk. In addition to applications in clinical medicine, CNNs and street images could also advance the epidemiology of COVID-19 at the population level.