Text-Controlled GAN Inversion for Image Editing

Abstract This paper presents an optimization-based approach that allows for text-based image editing using Generative Adversarial Networks (GANs) with preserving specific details of the image. While recent Diffusion Models can generate images using text by training with text and image pairs, GANs are trained only with images, making it challenging to control the latent space according to text prompts. Therefore, this paper introduces two-step pivotal mechanisms that enable text instructions for StyleGAN in the latent space while maintaining the original image information during editing. This approach simplifies the editing process and eliminates the need for separate training to get the particular direction in the latent space. The paper demonstrates the effectiveness of this method with diverse input images and written text prompts, showing compelling editing results.