Feasibility and Accuracy of a Real-Time Depth-Based Markerless Navigation Method for Hologram-Guided Surgery

Background: Two-dimensional (2D) medical visualization techniques are often insufficient for displaying complex, three-dimensional (3D) anatomical structures. Moreover, the visualization of medical data on a 2D screen during surgery is undesirable, because it requires a surgeon to continuously switch focus. This switching focus problem also results in extensive use of perioperative radiation to gain additional insights for a 3D configuration. The use of augmented reality (AR) has the potential to overcome these problems, for instance by using markers on target points that are aligned with the AR solution. However, placing markers for a precise holographic overlay are time-costly, always have to be visible within the field of view and disrupt the surgical workflow. In this study, we developed ARCUS, a depth-based, markerless holographic navigation system, which overlays 3D holograms onto target body parts to overcome the limitations of 2D medical visualization techniques. Methods and results In a phantom study, our markerless ARCUS system was evaluated for accuracy and precision by comparing it to a Quick Response (QR) code-based holographic registration method. The evaluation involved measuring the Euclidean distance between target points on a 3D-printed face and their corresponding points on the overlayed hologram using a robotic arm for precise measurements. Correlations between the measuring points provided by the our markerless system and the actual measuring points on the 3D-print were high, with promising consistent Euclidean distances between the 3D points and the holographic points generated by both our markerless system and the Vuforia QR Code system. Conclusion The markerless holographic navigation system holds strong potential as a 3D visualization method in clinical settings. While both ARCUS and the Vuforia QR code-based method fell short of meeting the surgical threshold of a 2 mm offset, our markerless system demonstrated promising features such as instant registration, markerless operation, and potential compatibility with non-rigid structures. Its automated hologram overlay onto target body parts offers significant advantages, paving the way for investigations into future clinical use.