A Kernel-Based Solution for Detecting and Preventing Fileless Malware on Linux

The first appearance of viruses can date back to the late last century. As an effective form of malware, viruses reside in the permanent storage of target hosts. Before a virus can execute, that must load into memory from the persistent storage included in the associated file. Due to the reliable destructive power of viruses, many mechanisms have been developed to defend computer systems against these hazardous threats. Antivirus software is one of the most famous and popular among these mechanisms. Most antivirus software uses static analysis (signature-based) technology on files stored in permanent storage, such as hard disks or USB flashes, to detect viruses hidden in files. Fileless malware was developed to enhance the survivability of malware by circumventing detection. Fileless malware only exists in the target hosts’ memory, not files. Antivirus software cannot even access the fileless malware code, much less analyze it, since it may be performed in memory directly without needing to load it from a disk. As a result, it is difficult for an antivirus engine to defend a system against fileless malware attacks. This paper proposes a kernel-based solution called Check-on-Execution (CoE) to detect fileless malware on a Linux system. When a program is going to execute a piece of code in a writable and executable memory area of a process, CoE suspends the code execution first. Coe retrieves the code from memory, packs the code with an ELF header to create an ELF file, and uses VirusTotal to check the file to prevent a Linux system from executing fileless malware. Experimental results show that CoE noticeably enhances the ability of a Linux system to defend itself again fileless malware. CoE is also suitable for protecting a system from shell code injection attacks, such as buffer and heap overflow attacks. It is capable of handling even packed malware. But in this paper, we only focus on fileless malware.