An Enhanced Encrypted Traffic Classifier via Combination of Deep Learning and Automata Learning

Abstract Characterizing the network traffic and identifying running applications play an instrumental role in several network administration tasks such as protecting against malicious behavior, firewalling, and balancing bandwidth usage. This classification, due to its complexity, has brought about several difficulties for researchers. Moreover, recent advances in the Internet, such as encryption-based security protocols, Tor networks, and virtual private networks. have made the classification even more complicated. Although many studies have been conducted to tackle these issues independently, few of them have partially been successful, addressing only one issue. We propose an approach called DeNeTLang, a general traffic classifier that addresses all the above-mentioned challenges. We utilize the automata learning technique to derive the behavioral packet-based model of each application in terms of a k-testable language. As some packets always appear together, we apply machine learning techniques to automatically identify those packets with similar timing and statistical features to increase the granularity of alphabets of the learned languages. This leads to smaller models yet still precise to characterize applications. To classify by a deep neural network, we convert the learned languages to feature vectors. The results of applying DeNeTLang on real traffic indicate that our approach outperforms the stateof-the-art methods both in application identification and traffic characterization tasks. Also, the proposed approach is resilient to noise stem from simultaneous execution of multiple applications.