Artificial intelligence directed development of a digital twin to measure soft tissue shift during head and neck surgery

Introduction Digital twins derived from 3D scanning data were developed to measure soft tissue deformation in head and neck surgery by an artificial intelligence approach. This framework was applied suggesting feasibility of soft tissue shift detection as a hitherto unsolved problem. Methods In a pig head cadaver model 104 soft tissue resection had been performed. The surface of the removed soft tissue (RTP) and the corresponding resection cavity (RC) was scanned (N=416) to train an artificial intelligence (AI) with two different 3D object detectors (HoloLens 2; ArtecEva). An artificial tissue shift (TS) was created by changing the tissue temperature from 7,91±4,1°C to 36,37±1,28°C. Results Digital twins of RTP and RC in cold and warm conditions had been generated and volumes were calculated based on 3D surface meshes. Significant differences in number of vertices created by the different 3D scanners (HoloLens2 51313 vs. ArtecEva 21694, p<0.0001) hence result in differences in volume measurement of the RTC (p=0.0015). A significant TS could be induced by changing the temperature of the tissue of RC (p=0.0027) and RTP (p=<0.0001). RC showed more correlation in TS by heating than RTP with a volume increase of 3.1 μl or 9.09% (p=0.449). Conclusions Cadaver models are suitable for training a machine learning model for deformable registration through creation of a digital twin. Despite different point cloud densities, HoloLens and ArtecEva provide only slightly different estimates of volume. This means that both devices can be used for the task.TS can be simulated and measured by temperature change, in which RC and RTP react differently. This corresponds to the clinical behaviour of tumour and resection cavity during surgeries, which could be used for frozen section management and a range of other clinical applications.