Deep Learning Analysis of in Vivo Hyperspectral Images for Automated Intraoperative Nerves Detection

Abstract Nerves are difficult to recognize during surgery and inadvertent injuries may occur, bringing catastrophic consequences for the patient. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing biological tissue property quantification. We show for the first time that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in-vivo at the pixel level. An animal model is used comprising eight anesthetized pigs with a neck midline incision, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models have been trained to recognize these tissue types in HSI data. The best model is a Convolutional Neural Network (CNN), achieving an overall average sensitivity of 0.91 and specificity of 0.99, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieves an average sensitivity of 0.76 and specificity of 1.0. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.