Training Against Human and Generative-AI Social Engineering Attacks with Cognitive Models

Social engineering attacks are commonly used by cybercriminals to gain valuable and sensitive data. Although the concern of attackers using AI-generated content is serious, training against social engineering attacks is typically based on simple human-designed emails. Our research introduces an experimental paradigm to determine whether there is a difference in the detection of human-generated and AI-generated emails. The behavioral results show that emails written by humans and stylized by Generative-AI models are more challenging for end-users. Alongside this novel experiment, we propose a cognitive model that can be used to predict end-user behavior during training, with the potential to improve the quality of examples used during training and the training feedback. Overall, the contributions of this work are, first, the outline of some limitations to current social engineering training methods and, second, pinpointing a potential solution to these limitations through the use of a cognitive model to improve learning outcomes.