Automated Ransomware Detection Using Windows File System Activity Monitoring and a Novel Machine Learning Approach

Ransomware has become a significant cybersecurity threat, targeting users and organizations through the encryption of critical files and demanding ransom payments for their recovery. Traditional detection methods, relying heavily on static signatures, often fail to identify novel ransomware variants, necessitating more adaptive and behavior-based approaches. In this work, a hybrid detection system combining Isolation Forest for anomaly detection and XGBoost for classification is proposed, enabling real-time monitoring of file system activities to detect ransomware with high precision. This method leverages the unique behaviors of ransomware, such as abnormal file modifications and encryption, allowing it to identify threats before significant damage occurs. The results demonstrate the system's ability to detect a wide range of ransomware variants, even those previously unseen, while maintaining low false positive rates, making it suitable for deployment in both enterprise and smaller-scale environments. Additionally, the system's architecture ensures scalability and efficient performance, enabling its integration into diverse operational settings where real-time detection is critical.