Comparative Analysis of ChatGPT-4 and Google Gemini for Spam Detection on the SpamAssassin Public Mail Corpus

Abstract This study addresses the critical challenge of spam detection in the realm of cybersecurity, motivated by the escalating sophistication of spamming techniques and their significant implications for communication security. With the advent of advanced artificial intelligence (AI) models, this research compares the efficacy of two leading models, ChatGPT-4 by OpenAI and Google Gemini, in identifying spam within the widely recognized SpamAssassin public mail corpus. Through a meticulous methodology that includes preprocessing of the dataset, application of standardized evaluation metrics (accuracy, precision, recall, and F1-score), and a detailed performance analysis, this study unveils the distinct capabilities of each model in spam detection. ChatGPT-4 demonstrates a balanced performance with high precision and recall, making it suitable for general spam detection tasks. In contrast, Google Gemini excels in recall, highlighting its potential in scenarios where capturing the maximum number of spam emails is paramount, despite a slightly higher tendency to misclassify legitimate emails as spam. These findings contribute valuable insights into the comparative strengths and application contexts of ChatGPT-4 and Google Gemini, offering a nuanced understanding of their roles in enhancing spam detection mechanisms. The study underscores the significance of selecting AI models based on specific spam detection needs and sets a foundation for future research aimed at advancing AI-driven spam detection solutions.