Opcode-Based Ransomware Detection Using Random Forest and Long Short-Term Memory Neural Networks: An Automated Machine Learning Approach

Abstract Ransomware has evolved into one of the most destructive forms of cyberattacks, causing severe financial losses and compromising critical data across various sectors. Addressing the dynamic and evasive nature of ransomware demands novel detection methods that are capable of identifying sophisticated attacks before significant damage occurs. A new approach is introduced that combines opcode sequence analysis with machine learning, specifically using Random Forest for feature selection and Long Short-Term Memory networks for capturing temporal dependencies in opcode streams. This hybrid model demonstrates high detection accuracy and precision, offering a more resilient solution against ransomware variants, including those employing obfuscation techniques. The experimental evaluation shows that the model effectively reduces false positives while maintaining computational efficiency, making it suitable for real-time ransomware detection. The integration of opcode-based feature selection and sequence analysis creates a robust detection framework capable of addressing modern ransomware challenges.