An Evaluation of the Reasoning Effectiveness of ChatGPT to Scrutinize and Quash Untrue Social Media Memes

Abstract Evaluating the effectiveness of ChatGPT in scrutinizing and debunking misinformation embedded within social media memes reveals significant potential for AI-driven fact-checking tools. Thirty memes from Facebook, Instagram, and TikTok were analyzed, showcasing the model's strengths and limitations in handling diverse content types. High accuracy and reasoning capabilities were observed, particularly with clear textual claims, while challenges remained with visually-oriented and contextually sparse content. The study underscores the necessity for platform-specific optimizations and multimodal approaches to enhance performance. Implications for social media fact-checking and AI development are discussed, alongside the study's limitations and suggestions for future research. By assisting human fact-checkers, AI models like ChatGPT can enhance the reliability of information disseminated online, contributing to a more informed and discerning public.