BrainPixGAN: Generating Intraoperative MR with Mask-Based Generative Networks

Abstract In recent years, there has been a significant focus on enhancing precision in brain tumor surgical treatments by utilizing artificial intelligence (AI) techniques. This involves integrating AI algorithms with medical imaging. In this study, Generative Adversarial Networks (GANs) were used to establish a relationship between preoperative MRI and resection cavity segmentation masks obtained from intraoperative ultrasound, aiming to generate a new intraoperative MR image free from the tumor. U-Net and U-Net with transfer learning were employed for resection cavity segmentations. The most successful model, U-Net + EfficientNetB7, achieved high scores. The resection cavity mask was applied to preoperative MRI using Pix2Pix, SPADE GAN, and BrainPixGAN. BrainPixGAN, incorporating transfer learning, outperformed the others. This innovative approach represents a pioneering effort in generating GAN models for intraoperative MR images, leveraging intraoperative ultrasound data, despite the challenges in setup and cost associated with intraoperative MR imaging.