Robust Hashing for Improved CNN Performance in Image-Based Malware Detection

This paper presents a comparative study on the impact of robust hashing in enhancing image-based malware classification. While convolutional neural networks (CNNs) have shown promise when working with image-based malware samples, their performance degrades significantly when obfuscation techniques are taken in consideration to hamper the malware classification or detection. To address this, we apply a robust hashing technique that generates invariant visual representations of malware samples, enabling improved generalization under obfuscation implemented as image salting. Using a custom obfuscation method to simulate polymorphic variants, we evaluate MobileNet, ResNet, and DenseNet architectures across five salting conditions (0% to 40%). The results demonstrate that robust hashing substantially boosts classification accuracy, with DenseNet achieving 89.50% on unsalted data, compared to only 68.00% without hashing. Across all salting levels, models consistently performed better when robust hashing was applied, confirming its effectiveness in preserving structural features and mitigating adversarial noise. These findings position robust hashing as a powerful preprocessing strategy for resilient malware detection. Keywords: Malware Detection, Convolutional Neural Networks, Robust Hashing, Dataset Obfuscation, MobileNet, ResNet, DenseNet, Adversarial Attacks, Cybersecurity.