MiniSIEM: A Log Analysis & Security Monitoring System

MiniSIEM: A Log Analysis & Security Monitoring System | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article MiniSIEM: A Log Analysis & Security Monitoring System Dhanakoti V, Charan Raj K, Akash D, Hamsahaasan G This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9249355/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Cybercriminals now have access to a much larger attack surface due to the ever-increasing reliance on internet-connected devices. This has resulted in an increase in network-based attacks such as port scanning, brute-force logins, distributed denial-of-service attacks, and attempts to gain unauthorized access [ 1 ]. Firewalls usually serve as a first line of defence for most networks, however, they generate a lot of log data which is not often properly analysed due to difficulties with manual analysis [ 6 ]. The objective of this study is to design and implement a Firewall Security Analytics System that will collect and analyse firewall logs to identify potentially malicious network activity, classify the attacks, and provide real-time monitoring of security events. The general design includes a client-server architecture using a Flask backend and a React-based Security Operations Center (SOC) dashboard. The firewall logs are collected from an Ubuntu server using SSH-based secure log ingestion into a structured database format. The classification of attack patterns is accomplished by an analysis engine that utilizes rules to determine whether or not an event is suspicious (for example: port scanning, brute-force attempts, and abnormal connection activity). If a security threshold is exceeded, an alerts engine will generate an alert to notify the administrator. In addition, the solution includes a dashboard where real-time analytics can be viewed to visualize current attacks on the network. Ultimately, the results indicate that the proposed platform is able to convert raw data from firewalls into action-based intelligence about the state of your network, giving the administrator a better understanding of the current status of their network, allowing for faster detection of potential threats. Cybersecurity Firewall Monitoring Security Analytics Intrusion Detection Flask React Attack Classification Network Security Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 Figure 13 Figure 14 I. Introduction As a result of the rapid growth of digital infrastructure, organizations relying primarily on the internet have become increasingly vulnerable to cyberattacks. An increasing number and types of critical services available through these networks provides attackers with significantly more opportunities to exploit system/application weaknesses supporting these services. Organization's use firewalls as their first layer of protection from both authorized/unattributed access to their protected resources by filtering incoming/outgoing network traffic through network traffic filtering techniques. While firewalls effectively block incoming malicious network traffic, firewalls generate large amounts of logs that contain detailed timestamps, connection attempts, blocked/non-blocked packet statistics, source/destination addresses, and port numbers. Due to the high volume of firewall logs generated, manually analyzing them is exceedingly difficult for networking staff. Firewalls have traditionally provided very little analysis of network activity beyond filtering traffic, so some of the threat patterns could go undetected until they cause extensive damage to the network infrastructure. As a result, there is a very real and urgent need for intelligent monitoring systems that can continuously monitor firewall log data and identify suspicious activity in real time. The System firewall Security Analytics solution automates how you collect, analyze and visualize the firewall log as part of a methodology to enhance your ability to manage your organization’s network activity and respond to potential threats in a timely manner by collecting, analyzing and visualizing firewall logs from your organization’s monitored workstations so that administrators can monitor their organization’s network activity and alert them to possible suspicious activity quickly. II. Related Work The Firewall Security Analytics System’s Architecture is built using many of today’s current principles in the development of systems for Network Security Monitoring (NSM), Intrusion Detection Systems (IDS), and Log Analytical Platform [ 6 ]. This section will discuss previously published research related to firewall monitoring, attack detection methods, security visualization utilities etc. A. Log Analysis and Parsing Techniques Since distributed computing environments can create a large number of different types of logs, performing log analysis has become an essential part of modern cybersecurity monitoring systems. Logs contain important information about the behaviour of the system, any operational problems that may have occurred, and security events that have occurred. A systematic mapping study by Partovian et al. identifies log analysis methods used for smart troubleshooting within Industry 4.0 environments. They found that automated log analysis methods (especially when using machine learning) are most frequently used to find anomalies and diagnose system failures in very large cyber-physical systems [ 1 ]. Log parsing transforms unstructured log data into structured representations, which are necessary to support effective analysis. Automated log parsing frameworks have been developed by Marlaithong et al. specifically for the ALICE O2 computing infrastructure supporting the analysis of large volumes of runtime log data produced by distributed scientific computation environments [ 2 ]. The authors propose that using TF–IDF-based feature extraction and genetic programming allows for the automatic generation of log templates and improved accuracy of log parsing. These and other effective methods for log parsing allow unstructured log entries to be converted into structured events that can be used in analysing for anomalous activity or monitoring of systems. B. Log Anomaly Detection and AI-Based Security Analytics As distributed systems have become increasingly complex, researchers have begun applying machine learning and deep learning techniques so that they can detect anomalies in their corresponding system logs. Through their research, Raeiszadeh et al. developed ALogSCAN, which incorporates a self-supervised dual-network architecture that can adaptively detect anomalies within cloud computing environments. In doing this, the authors utilize dynamic frequency-based log filtering to be able to filter out infrequent but critical log events while also reducing the dependency on labeled training data [ 3 ]. Horv́áth et al. developed and tested several methods for detecting anomalies in log files by using real-time analyses of log data, including deep models based upon neural networks, clustering algorithms, and standard statistical techniques. Their evaluation study found that the more advanced neural networks were equally effective at detecting anomalies as the simpler statistical methods, depending upon the complexity of the system and the operational constraints under which it was maintained [4]. Hybrid deep learning models have recently been examined to boost the effectiveness of anomaly detection systems. Alzamil et al. introduced DualBERT, a novel hybrid approach for anomaly detection based on using time-based features through the use of a Long Short Term Memory (LSTM) regression model combined with a transformer-based analysis of logs using DualBERT itself to analyse event sequences [ 5 ]. The incorporation of symbolic event sequences into the temporal analysis of events leads to a substantial improvement in the accuracy of anomaly detection and the percentage of false positives associated with these systems [ 5 ]. Similarly, in relation to the analysis of firewall logs, Aboudrar et al. introduced a new AI-based security information and event management (or SIEM) system that utilize deep-learning models capable of analysing information from firewall logs and detecting malicious activity on the network [ 6 ]. The proposed system integrates both event correlation and event classification based on neural networks to provide improved threat detection capabilities and a reduction in the number of false positives experienced with cybersecurity monitoring systems [ 6 ]. C. Firewall Monitoring, Visualization and Security Optimization In addition to using techniques to detect anomalies, many studies have been performed using firewall performance analysis as well as monitoring security events. Schufrin et al. (2010), developed an interactive visual log analysis system for performing firewall log analysis through visual analytics techniques. The system utilizes a number of different visual analytic interfaces to provide security analysts with access to similar functionality, including both an overview of the network security events as well as a detailed analysis of the network security events represented by firewall logs [ 7 ]. Lee et al. (2013), analysed firewalls event logs for high-performance computing networks and proposed a filtering mechanism to limit the amount of processing performed by the firewall. The results showed that the identification of firewall traffic patterns can be used to improve the performance of a firewall and provide the same (or better) security guarantee as compared to a similar amount of analysis of the same event logs without any performance consideration [ 8 ]. Another significant area that many researchers focus on in the field of cybersecurity is developing new datasets used to evaluate intrusion detection systems. For example, Black et al. created the Firewall Attack Detection and Extraction (FADE) dataset, which contains millions of labeled, benign, and malicious network requests to aid in developing and testing firewall attack detection systems [ 10 ]. Lastly, work is being done by Durante et al. to establish a formal model for distributing packet filtering rules across multiple firewalls within a network. The proposed rule redistribution algorithm will reduce the processing overhead related to firewalls by effectively balancing the filtering workload among all of the cascade firewalls in a given environment; thus, increasing the performance of packet processing within large, network-based infrastructures [ 9 ]. III. Proposed System Architecture and Design The Firewall Security Analytics System uses a Modular Design by integrating multiple components of collecting logs from Firewalls, Processing Events, Detecting Attacks & Visualizing Events. Thus, allowing for capabilities of Scalability, Reliability and Efficiency when analyzing Network Events. Fig: 3.1 Architecture Diagram A. Design Principles The architecture of the system is directed by 3 fundamental design principles. Modularity and Scalability The framework consists of 4 distinct modules providing functions for gathering logs, processing data, classifying attacks/attempts, and providing a means to visualize security events. The modular nature of the solution makes it possible to add new security features to the platform without having to modify already-existing modules. Performance and Efficiency To support efficient processing of large volumes of log data, network monitoring systems require an appropriate architecture at the backend. This architecture is constructed to allow for continuous log ingestion and classification while maintaining the responsiveness of the dashboard interface. Security and Reliability Given that any information transmitted through your systems could potentially include some form of sensitive network traffic, secure communications have been established between the logging service and back-end servers to provide confidentiality. Logging also provides authentication services to ensure that only authorized individuals can view the monitoring dashboard views. B. System Components The proposed system consists of three major layers that work together to deliver the platform’s functionality. Fig: 3.2 System Component Specification Log Collection Layer Firewall logs generated by the Ubuntu Uncomplicated Firewall are collected using secure SSH connections. These logs contain information such as source IP addresses, destination ports, protocol types, and timestamps. Fig: 3.3 Firewall Event Log Fields & Description Backend Processing Layer The backend of the application was designed and implemented using the Flask web framework, which parses firewall log files, inserts records of events into an SQLite database, and executes an algorithm that classifies attacks against a network. The backend also exposes a number of REST APIs which provide processed data to the front-end user interface (for example, the dashboard). Visualization Layer The front-end dashboard application has been developed using React, and provides visual representations of network activity in nearly real-time. The dashboard displays statistics related to attacks, timelines accompanying logs, and displays geographic maps where possible attacks may occur to assist network/system administrators with monitoring for potential attacks. Fig: 3.4 Comparison of Traditional & Proposed System IV. Implementation & Module Functionality The MiniSIEM: A Log Analysis & Security Monitoring System was implemented as a modular cybersecurity analytics platform with firewall logs provides real-time threat intelligence by processing the firewall logs into various analytical modules. These analytical modules process the firewall logs collected from any Ubuntu-based firewall, providing insight into threats through multiple analytical modules. Each analytical module contributes to identifying malicious activity and to identifying attack patterns that are helpful in assisting security analysts in identifying the threats on a network. Fig: 4.1 Analytics Features & Description Fig: 4.2 Real-Time Overview Attack Timeline The Attack Timeline Module displays a chronological distribution of the time when security events were detected through the use of timestamps from firewalls whereby security analysts can see the frequency of when attacks occur and can see any abnormal spike in malicious activity at a point in time in order to identify peak attack times. Severity Distribution The Severity Distribution Module breaks down the detected events based upon their threat level. Detects will fall under a certain severity based upon the attack method used and the potential impact it may have on a network (low, medium or high). This will produce graphical representations that will aid security analysts in assessing the security health of their network. Attack Type Distribution The Attack Type Distribution Module provides insight to the types of attacks that were detected within a monitored environment. By utilizing a rule-based classification engine, the module is able to categorize various types of attacks (port scanning, brute-force attempts and connection bursts) and produce a graphical representation of the frequency of each attack. This will allow analysts to identify their highest, most prevalent threat types. Top Attacker Ips Through usage of the Top Attacker IPs module, malicious activity from external IP Addresses can be identified and ranked according to the amount of detected events found within the firewall logs. By using this information, analyst can prioritize their efforts in investigating and mitigating attacks from the most active external IPs. Fig: 4.3 Dashboard Charts Attack Map The Attack Map module shows you visually where attacks are originating from geographically; meaning you can see where cyber attacks originated by looking at the sources IP Address mapped to its approximate geographical location (using available IP geolocation services). This information allows you to see where attacks originated, as well as display how malicious activity is globally distributed within the network that you are monitoring. Fig: 4.4 Attack Map Live Event Feed The Live Event Feed module provides live feeds of detected security events (threats), with events being displayed on the dashboard immediately after they have been detected (from the backend processing of the firewall logs). This module provides a near-real time view of threats as they happen, similar to a SOC (Security Operations Center) monitoring view. Fig: 4.5 Live Event Feed Attack Replay The Attack Replay module allows analyst to look back, at attack sequences, historically; replaying attacks based on one or more attackers OR based on a specific time window. This feature allows for re-creation of an attack timeline, enabling security teams to gain insight into attacker behaviour and to then use this information to improve their security postures against such attacks. Event Table The Event Table module offers a well-defined structure to present a list of all events relating to firewalls stored in the system's DB (database). Each entry contains attributes like the time/date stamp, source IP address, destination Port, protocol used, severity level and the type of detected attack. The table allows for easy inspection and filtering of security events. Suspicious IP Reputation The Suspicious IP Reputation module measures and rates the level of trust that an IP address has on the network. At the same time, the system tracks IPs that have been consistently causing high numbers of malicious events and then assigns these IPs with a flag marking them as suspicious. This information helps analysts to identify any potential repeating threat actors. Attacker Persistence In the proposed system, the Attacker Persistence module tracks the number of repeated connection attempt from a single IP source. Through measuring how often and how long an attack is made, the system can determine whether that source IP is a persistent attacker against the network. IP Intelligence The IP Intelligence module collects contextually relevant data on the attacking IP address. This can include information about where the attacking IP is physically located, which network provider is supplying the service to the attacker's IP, and the perceived threat level associated with that IP address. This contextual data improves situational awareness for the incident response team and helps them make more informed decisions during their response to an incident. Fig: 4.6 Threat Intelligence Card Targeted Port Analysis The Targeted Port Analysis module finds the most attacked network ports. By analysing firewall logs' destination port field, the module can determine which services attackers may be trying to test or attack. Protocol Distribution The Protocol Distribution module looks at the communication protocols used during events on the network. By reviewing the firewall logs, the module can verify whether or not the events were TCP, UD, or ICMP. Understanding protocol distribution can help identify abnormal traffic patterns. Service Exposure Analysis The Service Exposure Analysis module determines which services are most exposed to attempts at external access. By connecting targeted ports to service types, the module can identify those services that may require additional security hardening. Fig: 4.7 Network Exposure Cards Attack Trend The Attack Trend module tracks long-term changes in attack activity. It will analyse historical event data to determine whether the number of attacks over a defined period of time has increased, decreased or remained constant. Attack Growth The Attack Growth module calculates the rate of increase in the level of attack activity over a defined period of time. This provides security analysts with a metric for detecting the sudden acceleration of attack campaigns. Attack Heatmap The Attack Heatmap Module records times and places (over time) where high volumes of attacks are occurring using a color-coded display. Attack Velocity The Attack Velocity Module is used to rate the number of attacks that occur in very short periods of time and can be used to identify an attack that is happening due to an automated attack campaign or a bot. Fig: 4.8 Temporal Analysis Attack Intensity The Attack Intensity Module is used to determine how many attacks are concentrated on a particular system during a defined period of time, with very high attack intensity values indicating either large-scale or coordinated attempts to attack. Attack Correlation The Attack Correlation Module identifies how different security events relate to one another. Through the analysis of event attributes such as their source (IP), port (target), and the time (timestamp), this correlations module can correlate and identify events that come from the same attack campaign or individual attack. Fig: 4.9 Attack Behaviour Anomaly Detection The Anomaly Detection Module identifies unusual patterns of traffic at the firewall. The anomaly detection module takes the current activity and compares it to historical activity, using the historical activity to detect any differences that could indicate an undetected or new form of attack. Fig: 4.10 Security Monitoring V. Experimental Results and Discussion A controlled laboratory environment was used to evaluate the MiniSIEM: A Log Analysis and Security monitoring System for firewall log monitoring in real-time, attack detection, and security analytics visualisation. The experimental setup included the following: An attacker machine generating simulated malicious traffic, a firewall machine running Ubuntu OS configured with Uncomplicated Firewall (UFW) and a host machine managing both the collecting of logs and processing of analytics. Several attack scenarios were created to assess the system's functionality, performance and reliability. A. Functionality and Correctness Functional testing was performed in order to establish if the solution would accurately retrieve, store and analyse firewall logs in real-time. The log collector retrieved firewalls logs from the Ubuntu Server using secure SSH communications by forwarding them to the backend processing engine. The log parser created structured security events from the firewall logs by extracting key attributes including source IP address, destination port, protocol type, timestamp and action performed by the firewall. Multiple attack scenarios (i.e., port scans, repeated connection attempts and abnormal bursts of requests) were simulated as part of the evaluation of the detection capability. The rule classification engine identified each of these attacks correctly based upon the defined detection rules. The classification engine in the proposed system identifies sequential connection attempts from a single IP source across multiple ports as port scanning, and brute-force where repeated attempts to access the same service as potential behaviour of it. The system generated alerts based upon the detected suspicious activity and displayed the alerts via a real-time dashboard. The support of the visualization modules (e.g., attack timeline, severity distribution and attack types) displayed the processed events with high accuracy. It is clear from these results that the system can interpret firewall logs with accuracy and provide actionable cybersecurity insight based upon them. B. Performance and Scalability The evaluation of how well the system processes immense amounts of firewall logs placed priority upon whether or not it does so while responding in real-time. The backend processing engine was that it accomplishes this task by being subjected to continuous logging conditions, along with simulated attack traffic, as opposed to logs simply being processed at certain discrete intervals. The evaluation indicated all log ingestion and processing could occur with almost no delay. The log parsing function and the event classification function performed well when processing multiple logs/events at the same time. The backend of this application is developed with a lightweight architecture utilizing Flask and SQLite allowing for high performance and low system resource utilization. The dashboard user interface is also created with React, and displays security events (visually) at "near real-time" updates that refresh every five seconds. Because of its modular architecture, this solution can be expanded and adapted to support new types of detection modules and/or analytic modules for use with firewalls and firewall logs (shown below). The design of the system allows for these new modules to be added without affecting the core functionality of the system as a whole, therefore, making it easy to use in high volume, continuously generated, real-time firewall log monitoring environments. C. Security and Reliability The objective was to conduct a security evaluation to assess both the soundness of the authentication mechanisms utilized by the system and the reliability of the alert generation process. There is a JSON web token (JWT)-based authentication system in place that prevents unauthorized users from accessing the analytics dashboard. Based on test results, unauthorized access attempts were blocked successfully. Communication channels between the application and server were appropriately secured so that data could be considered safe. An alert generation system was designed to be capable of accurately detecting and reporting suspicious activity. For example, anytime the classification engine detected a possible attack pattern, the alert generation system produced an alert and created an entry in the event record. All alerts were shown immediately on both the live event feed and in alert panels within the analytics dashboard. In addition to the real-time alert generation capabilities, the event record provides a historical record of security-related events that are available for use by security analysts as reference material when conducting investigations into past incidents or for evaluating patterns associated with cyber-attacks over an extended period of time. The reliable storage and retrieval of event records ensure that security-related data can be used as evidence in forensic investigations and incident response efforts. VI. Future Work The proposed MiniSIEM: A Log Analysis & Security Monitoring System provides a solid foundation for real-time firewall monitoring, attack detection, and security analytics; nonetheless, there are many possible improvements that could make the system work better and allow it to do more. For example, one area of potential future development would be to integrate machine learning (ML) and artificial intelligence (AI) techniques into the attack classification engine, which would allow for intelligent detection of complex and unknown patterns of attacks. Through the use of historical log data collected from firewalls, ML models could discover abnormal behaviour and patterns of behaviour that traditional, rule-based detection systems would not have found. This would improve accuracy of detections and decrease false positive detections. In addition, the system could potentially support third-party threat intelligence sources (feeds) and third-party (IP) reputation services to provide additional contextual information about potentially malicious IP addresses, resulting in more accurate threat assessments. Future versions of the platform could also support automated incident response processes (e.g., updating firewall rules to block malicious IP addresses in real-time or temporarily restricting (black-listing) traffic from an IP address if that IP address has shown abnormal behaviour) and allowing automated incident response actions. Lastly, additional functionality could be added to the system to support multiple firewall vendors, distributed log collection from large enterprise networks, and providing additional forms of advanced analytics (e.g., predictive modelling of threats and long-term trends of attacks). The enhancements proposed for establishing an extensive, intelligent cybersecurity monitoring framework would enable the implementation of a much larger and more dynamic monitoring platform that would adequately fulfill today's security operations centers growing requirements for a robust technology base to support their operational needs. VII. Conclusion The proposed Firewall Security Analytics System is aimed at providing improved methods of enhancing the monitoring of network security through real-time analysis of firewall logs along with smart rendering. Automated log collection; structured event processing; attack classification by rules; event-trigger alerts; and support for interactive analytics dashboards are all features that provide a means to convert general firewall log data into high-quality cybersecurity intelligence. The established architecture provides for continuous monitoring of network activities, while also providing the security administrator with an overview of attack patterns, threat levels, and potential unsafe behaviours. Through the use of the analytical modules of attack timeline, severity distribution, attacker IP analysis, and geographic location based risk visualization of attacks, the administrator is able to quickly identify anomalies and respond to potential cybersecurity incidents. Both the performance based evaluation and test results show that the system is effective at processing firewall logs to provide high degrees of accuracy for identifying suspicious activities, ultimately resulting in actionable information being provided through real-time dashboards. The proposed modular/scalable architecture will allow it to be adaptable as the requirements associated with cybersecurity continue to evolve and to support additional analytical functionality over a more extended period of time. This proposed solution offers an inexpensive, practical framework for enhancing the effectiveness of firewall-based threat detection, thereby enhancing the effectiveness of proactive cybersecurity defence mechanisms for today’s networks. Declarations Author Contribution C.R.K., A.D., and H.G. contributed to the conception, design, and development of the system, including implementation, testing, and validation. C.R.K. performed the data analysis and drafted the manuscript. A.D. prepared the figures, and H.G. prepared the tables. V.D. contributed to the supervision of the work, provided critical feedback, and reviewed the manuscript for important intellectual content. All authors approved the final version of the manuscript and agree to be accountable for all aspects of the work. Data Availability The data used in this study was generated from firewall logs collected in a controlled experimental environment using Ubuntu UFW. Due to security and privacy concerns, the dataset is not publicly available. References Partovian, S.: Analysis of Log Files to Enable Smart-Troubleshooting in Industry 4.0: A Systematic Mapping Study. IEEE Access. 12 , 147640–147660 (2024) Marlaithong, T.: A Log Parsing Framework for ALICE O2 Facilities. IEEE Access. 11 , 69439–69457 (2023) Raeiszadeh, M.: ALogSCAN: A Self-Supervised Dual Network for Adaptive and Timely Log Anomaly Detection in Clouds. IEEE Trans. Mach. Learn. Commun. Netw., (2025) Horváth, A.: et. Al., Anomaly Detection Algorithms for Real-Time Log Data Analysis at Scale. IEEE Access., 13 , (2025) Alzamil, A.: DualBERT: Fusing Symbolic and Temporal Dynamics for High-Precision Log Anomaly Detection. IEEE Access., 14 , (2026) Aboudrar, Y.: AI-Driven Firewall Log Analysis: Enhancing Threat Detection with Deep Learning Techniques. Int. J. Adv. Comput. Sci. Appl. 16 (7), 808–815 (2025) Schufrin, M.: Visual Firewall Log Analysis – At the Border Between Analytical and Appealing. IEEE Visualization Workshop, (2018) Lee, J.K.: Traffic and Overhead Analysis of Applied Pre-filtering ACL Firewall on HPC Service Network. J. Commun. Netw. 23 (3), 192–200 (2021) Durante, L.: A Formal Model and Technique to Redistribute the Packet Filtering Load in Multiple Firewall Networks. IEEE Trans. Inf. Forensics Secur. 16 , 2637–2648 (2021) Black, G.: Descriptor: Firewall Attack Detections and Extractions (FADE). IEEE Data Descriptions, (2025) Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9249355","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":613672793,"identity":"213c318a-fe1b-42c4-beab-a5b849a6efab","order_by":0,"name":"Dhanakoti V","email":"","orcid":"","institution":"SRM Valliammai Engineering College","correspondingAuthor":false,"prefix":"","firstName":"Dhanakoti","middleName":"","lastName":"V","suffix":""},{"id":613672794,"identity":"b08bd7ae-c7ba-48c5-ab83-42e47d15c8b0","order_by":1,"name":"Charan Raj K","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA90lEQVRIiWNgGAWjYBACCQYeGPNg84MPQIqNnXgth9sMZ4C0MBOvhb1BGswmpEWy/ezBzxU1d/L5Gw82GNv82ibPx8zA+OFjDm4t0jx5yZJnjj2znHHgYMPj3L7bhm3MDMySM7fh1iInwWMg2cB22IABqMU4t+c2I1ALGzMvfi3GPxv+HTaQB2qRtuy5bU9Qi7QEj5lkY9thAwOQFoYftxMJapHsyTGzbOw7bGB44GCbYW/D7eQ2ZsZmvH6ROH7G+GbDt8MGcjeOP37w489t2/ntzQc/fMSjBUnzAQYGxjYQi7GBGPVAwA9S+IdIxaNgFIyCUTCiAAB55FUxMneqxAAAAABJRU5ErkJggg==","orcid":"","institution":"SRM Valliammai Engineering College","correspondingAuthor":true,"prefix":"","firstName":"Charan","middleName":"Raj","lastName":"K","suffix":""},{"id":613672795,"identity":"4749846c-3bb0-4d0e-9754-08117ae4d4cf","order_by":2,"name":"Akash D","email":"","orcid":"","institution":"SRM Valliammai Engineering College","correspondingAuthor":false,"prefix":"","firstName":"Akash","middleName":"","lastName":"D","suffix":""},{"id":613672796,"identity":"810bb8a6-b76a-499c-8347-93a1bb8a6ab9","order_by":3,"name":"Hamsahaasan G","email":"","orcid":"","institution":"SRM Valliammai Engineering College","correspondingAuthor":false,"prefix":"","firstName":"Hamsahaasan","middleName":"","lastName":"G","suffix":""}],"badges":[],"createdAt":"2026-03-28 03:38:02","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9249355/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9249355/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105784532,"identity":"83612ec0-0998-4f5d-915f-83059f6e2b47","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":55400,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 3.1 Architecture Diagram\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/e1fa583b931fae5df3f75f8c.jpg"},{"id":105904399,"identity":"7d2583e4-a134-4006-9f16-ac4f9beb0dc1","added_by":"auto","created_at":"2026-04-01 10:08:07","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":79969,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 3.2 System Component Specification\u003c/p\u003e","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/cd33e36044db47d3a8784f04.jpg"},{"id":105784534,"identity":"729f357a-cbb5-4884-b7f8-aa2e1bd70302","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":76825,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 3.3 Firewall Event Log Fields \u0026amp; Description\u003c/p\u003e","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/3f31f9c1aa582e580ab1674b.jpg"},{"id":105784535,"identity":"5155df8c-fd22-4fb4-abe3-bd220181b82f","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":43644,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 3.4 Comparison of Traditional \u0026amp; Proposed System\u003c/p\u003e","description":"","filename":"4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/9a56a858bf4c99102d8678d9.jpg"},{"id":105904474,"identity":"f68e9d6d-3935-4575-a89f-d77bd617f9a6","added_by":"auto","created_at":"2026-04-01 10:08:53","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":91211,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.1 Analytics Features \u0026amp; Description\u003c/p\u003e","description":"","filename":"5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/8adad3c9aba0426ff5facc1e.jpg"},{"id":105904116,"identity":"3dc804b6-6005-40ce-b3f7-1d271bd0259c","added_by":"auto","created_at":"2026-04-01 10:04:30","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":18775,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.2 Real-Time Overview\u003c/p\u003e","description":"","filename":"6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/e2128c2359b8fb14afdcea10.jpg"},{"id":105784538,"identity":"77552f2c-6c3b-4fd6-905c-ac53592db700","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":57632,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.3 Dashboard Charts\u003c/p\u003e","description":"","filename":"7.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/4420004a0629485cdd8236ca.jpg"},{"id":105904445,"identity":"e4a5f9d5-fb0c-4e8d-924c-3a7796276652","added_by":"auto","created_at":"2026-04-01 10:08:39","extension":"jpg","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":50934,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.4 Attack Map\u003c/p\u003e","description":"","filename":"8.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/15bbbba64b2f2aa40fcf8b24.jpg"},{"id":105784540,"identity":"271dc6fa-5d74-42a4-88dd-d90bfce4c1d0","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":91374,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.5 Live Event Feed\u003c/p\u003e","description":"","filename":"9.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/05d0d118b5988d5861aa5e46.jpg"},{"id":105784541,"identity":"05b25566-167a-40d3-bb30-b515bd594469","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":70921,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.6 Threat Intelligence Card\u003c/p\u003e","description":"","filename":"10.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/f02e70aa2c844946a0c782cf.jpg"},{"id":106401571,"identity":"a9acfd1a-83d5-4a1b-b689-0ed628670083","added_by":"auto","created_at":"2026-04-08 09:07:19","extension":"jpg","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":56798,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.7 Network Exposure Cards\u003c/p\u003e","description":"","filename":"11.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/a08ed3491bfd306f91465186.jpg"},{"id":105904400,"identity":"4825dfff-045d-4336-8580-c66cca9e9ca5","added_by":"auto","created_at":"2026-04-01 10:08:07","extension":"jpg","order_by":12,"title":"Figure 12","display":"","copyAsset":false,"role":"figure","size":61448,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.8 Temporal Analysis\u003c/p\u003e","description":"","filename":"12.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/7ba144b79a06a29f5d056e30.jpg"},{"id":105784543,"identity":"1f7717c8-7165-4981-b8c2-181ef50f5e91","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":13,"title":"Figure 13","display":"","copyAsset":false,"role":"figure","size":48005,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.9 Attack Behaviour\u003c/p\u003e","description":"","filename":"13.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/eaa565b22aa65aa2ad59ed63.jpg"},{"id":105784544,"identity":"8fbf03e4-bb5e-4dd1-a05e-96d6c6e318db","added_by":"auto","created_at":"2026-03-31 06:20:56","extension":"jpg","order_by":14,"title":"Figure 14","display":"","copyAsset":false,"role":"figure","size":41746,"visible":true,"origin":"","legend":"\u003cp\u003eFig: 4.10 Security Monitoring\u003c/p\u003e","description":"","filename":"14.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/0ab2b9d07da856371d0c2c9e.jpg"},{"id":106415022,"identity":"d93ccb59-f952-4432-90a5-18977ef38bb9","added_by":"auto","created_at":"2026-04-08 10:32:06","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1558715,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9249355/v1/5008b187-aff2-4550-8c32-7a1bb8679a59.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"MiniSIEM: A Log Analysis \u0026 Security Monitoring System","fulltext":[{"header":"I. Introduction","content":"\u003cp\u003eAs a result of the rapid growth of digital infrastructure, organizations relying primarily on the internet have become increasingly vulnerable to cyberattacks. An increasing number and types of critical services available through these networks provides attackers with significantly more opportunities to exploit system/application weaknesses supporting these services. Organization's use firewalls as their first layer of protection from both authorized/unattributed access to their protected resources by filtering incoming/outgoing network traffic through network traffic filtering techniques. While firewalls effectively block incoming malicious network traffic, firewalls generate large amounts of logs that contain detailed timestamps, connection attempts, blocked/non-blocked packet statistics, source/destination addresses, and port numbers. Due to the high volume of firewall logs generated, manually analyzing them is exceedingly difficult for networking staff.\u003c/p\u003e \u003cp\u003eFirewalls have traditionally provided very little analysis of network activity beyond filtering traffic, so some of the threat patterns could go undetected until they cause extensive damage to the network infrastructure. As a result, there is a very real and urgent need for intelligent monitoring systems that can continuously monitor firewall log data and identify suspicious activity in real time.\u003c/p\u003e \u003cp\u003eThe System firewall Security Analytics solution automates how you collect, analyze and visualize the firewall log as part of a methodology to enhance your ability to manage your organization\u0026rsquo;s network activity and respond to potential threats in a timely manner by collecting, analyzing and visualizing firewall logs from your organization\u0026rsquo;s monitored workstations so that administrators can monitor their organization\u0026rsquo;s network activity and alert them to possible suspicious activity quickly.\u003c/p\u003e"},{"header":"II. Related Work","content":"\u003cp\u003eThe Firewall Security Analytics System\u0026rsquo;s Architecture is built using many of today\u0026rsquo;s current principles in the development of systems for Network Security Monitoring (NSM), Intrusion Detection Systems (IDS), and Log Analytical Platform [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. This section will discuss previously published research related to firewall monitoring, attack detection methods, security visualization utilities etc.\u003c/p\u003e \u003cp\u003e \u003cb\u003eA. Log Analysis and Parsing Techniques\u003c/b\u003e \u003c/p\u003e \u003cp\u003eSince distributed computing environments can create a large number of different types of logs, performing log analysis has become an essential part of modern cybersecurity monitoring systems. Logs contain important information about the behaviour of the system, any operational problems that may have occurred, and security events that have occurred. A systematic mapping study by Partovian et al. identifies log analysis methods used for smart troubleshooting within Industry 4.0 environments. They found that automated log analysis methods (especially when using machine learning) are most frequently used to find anomalies and diagnose system failures in very large cyber-physical systems [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eLog parsing transforms unstructured log data into structured representations, which are necessary to support effective analysis. Automated log parsing frameworks have been developed by Marlaithong et al. specifically for the ALICE O2 computing infrastructure supporting the analysis of large volumes of runtime log data produced by distributed scientific computation environments [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. The authors propose that using TF\u0026ndash;IDF-based feature extraction and genetic programming allows for the automatic generation of log templates and improved accuracy of log parsing. These and other effective methods for log parsing allow unstructured log entries to be converted into structured events that can be used in analysing for anomalous activity or monitoring of systems.\u003c/p\u003e \u003cp\u003e \u003cb\u003eB. Log Anomaly Detection and AI-Based Security Analytics\u003c/b\u003e \u003c/p\u003e \u003cp\u003eAs distributed systems have become increasingly complex, researchers have begun applying machine learning and deep learning techniques so that they can detect anomalies in their corresponding system logs. Through their research, Raeiszadeh et al. developed ALogSCAN, which incorporates a self-supervised dual-network architecture that can adaptively detect anomalies within cloud computing environments. In doing this, the authors utilize dynamic frequency-based log filtering to be able to filter out infrequent but critical log events while also reducing the dependency on labeled training data [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eHorv́\u0026aacute;th et al. developed and tested several methods for detecting anomalies in log files by using real-time analyses of log data, including deep models based upon neural networks, clustering algorithms, and standard statistical techniques. Their evaluation study found that the more advanced neural networks were equally effective at detecting anomalies as the simpler statistical methods, depending upon the complexity of the system and the operational constraints under which it was maintained [4].\u003c/p\u003e \u003cp\u003eHybrid deep learning models have recently been examined to boost the effectiveness of anomaly detection systems. Alzamil et al. introduced DualBERT, a novel hybrid approach for anomaly detection based on using time-based features through the use of a Long Short Term Memory (LSTM) regression model combined with a transformer-based analysis of logs using DualBERT itself to analyse event sequences [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. The incorporation of symbolic event sequences into the temporal analysis of events leads to a substantial improvement in the accuracy of anomaly detection and the percentage of false positives associated with these systems [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eSimilarly, in relation to the analysis of firewall logs, Aboudrar et al. introduced a new AI-based security information and event management (or SIEM) system that utilize deep-learning models capable of analysing information from firewall logs and detecting malicious activity on the network [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. The proposed system integrates both event correlation and event classification based on neural networks to provide improved threat detection capabilities and a reduction in the number of false positives experienced with cybersecurity monitoring systems [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cb\u003eC. Firewall Monitoring, Visualization and Security Optimization\u003c/b\u003e \u003c/p\u003e \u003cp\u003eIn addition to using techniques to detect anomalies, many studies have been performed using firewall performance analysis as well as monitoring security events. Schufrin et al. (2010), developed an interactive visual log analysis system for performing firewall log analysis through visual analytics techniques. The system utilizes a number of different visual analytic interfaces to provide security analysts with access to similar functionality, including both an overview of the network security events as well as a detailed analysis of the network security events represented by firewall logs [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eLee et al. (2013), analysed firewalls event logs for high-performance computing networks and proposed a filtering mechanism to limit the amount of processing performed by the firewall. The results showed that the identification of firewall traffic patterns can be used to improve the performance of a firewall and provide the same (or better) security guarantee as compared to a similar amount of analysis of the same event logs without any performance consideration [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eAnother significant area that many researchers focus on in the field of cybersecurity is developing new datasets used to evaluate intrusion detection systems. For example, Black et al. created the Firewall Attack Detection and Extraction (FADE) dataset, which contains millions of labeled, benign, and malicious network requests to aid in developing and testing firewall attack detection systems [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eLastly, work is being done by Durante et al. to establish a formal model for distributing packet filtering rules across multiple firewalls within a network. The proposed rule redistribution algorithm will reduce the processing overhead related to firewalls by effectively balancing the filtering workload among all of the cascade firewalls in a given environment; thus, increasing the performance of packet processing within large, network-based infrastructures [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e].\u003c/p\u003e"},{"header":"III. Proposed System Architecture and Design","content":"\u003cp\u003eThe Firewall Security Analytics System uses a Modular Design by integrating multiple components of collecting logs from Firewalls, Processing Events, Detecting Attacks \u0026amp; Visualizing Events. Thus, allowing for capabilities of Scalability, Reliability and Efficiency when analyzing Network Events.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 3.1 Architecture Diagram\u003c/p\u003e \u003cp\u003e \u003cb\u003eA. Design Principles\u003c/b\u003e \u003c/p\u003e \u003cp\u003eThe architecture of the system is directed by 3 fundamental design principles.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eModularity and Scalability\u003c/strong\u003e \u003cp\u003eThe framework consists of 4 distinct modules providing functions for gathering logs, processing data, classifying attacks/attempts, and providing a means to visualize security events. The modular nature of the solution makes it possible to add new security features to the platform without having to modify already-existing modules.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003ePerformance and Efficiency\u003c/strong\u003e \u003cp\u003eTo support efficient processing of large volumes of log data, network monitoring systems require an appropriate architecture at the backend. This architecture is constructed to allow for continuous log ingestion and classification while maintaining the responsiveness of the dashboard interface.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eSecurity and Reliability\u003c/strong\u003e \u003cp\u003eGiven that any information transmitted through your systems could potentially include some form of sensitive network traffic, secure communications have been established between the logging service and back-end servers to provide confidentiality. Logging also provides authentication services to ensure that only authorized individuals can view the monitoring dashboard views.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eB. System Components\u003c/b\u003e \u003c/p\u003e \u003cp\u003eThe proposed system consists of three major layers that work together to deliver the platform\u0026rsquo;s functionality.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 3.2 System Component Specification\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eLog Collection Layer\u003c/strong\u003e \u003cp\u003eFirewall logs generated by the Ubuntu Uncomplicated Firewall are collected using secure SSH connections. These logs contain information such as source IP addresses, destination ports, protocol types, and timestamps.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 3.3 Firewall Event Log Fields \u0026amp; Description\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eBackend Processing Layer\u003c/strong\u003e \u003cp\u003eThe backend of the application was designed and implemented using the Flask web framework, which parses firewall log files, inserts records of events into an SQLite database, and executes an algorithm that classifies attacks against a network. The backend also exposes a number of REST APIs which provide processed data to the front-end user interface (for example, the dashboard).\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eVisualization Layer\u003c/strong\u003e \u003cp\u003eThe front-end dashboard application has been developed using React, and provides visual representations of network activity in nearly real-time. The dashboard displays statistics related to attacks, timelines accompanying logs, and displays geographic maps where possible attacks may occur to assist network/system administrators with monitoring for potential attacks.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 3.4 Comparison of Traditional \u0026amp; Proposed System\u003c/p\u003e"},{"header":"IV. Implementation \u0026 Module Functionality","content":"\u003cp\u003eThe MiniSIEM: A Log Analysis \u0026amp; Security Monitoring System was implemented as a modular cybersecurity analytics platform with firewall logs provides real-time threat intelligence by processing the firewall logs into various analytical modules. These analytical modules process the firewall logs collected from any Ubuntu-based firewall, providing insight into threats through multiple analytical modules. Each analytical module contributes to identifying malicious activity and to identifying attack patterns that are helpful in assisting security analysts in identifying the threats on a network.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.1 Analytics Features \u0026amp; Description\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.2 Real-Time Overview\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Timeline\u003c/strong\u003e \u003cp\u003eThe Attack Timeline Module displays a chronological distribution of the time when security events were detected through the use of timestamps from firewalls whereby security analysts can see the frequency of when attacks occur and can see any abnormal spike in malicious activity at a point in time in order to identify peak attack times.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eSeverity Distribution\u003c/strong\u003e \u003cp\u003eThe Severity Distribution Module breaks down the detected events based upon their threat level. Detects will fall under a certain severity based upon the attack method used and the potential impact it may have on a network (low, medium or high). This will produce graphical representations that will aid security analysts in assessing the security health of their network.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Type Distribution\u003c/strong\u003e \u003cp\u003eThe Attack Type Distribution Module provides insight to the types of attacks that were detected within a monitored environment. By utilizing a rule-based classification engine, the module is able to categorize various types of attacks (port scanning, brute-force attempts and connection bursts) and produce a graphical representation of the frequency of each attack. This will allow analysts to identify their highest, most prevalent threat types.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eTop Attacker Ips\u003c/strong\u003e \u003cp\u003eThrough usage of the Top Attacker IPs module, malicious activity from external IP Addresses can be identified and ranked according to the amount of detected events found within the firewall logs. By using this information, analyst can prioritize their efforts in investigating and mitigating attacks from the most active external IPs.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.3 Dashboard Charts\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Map\u003c/strong\u003e \u003cp\u003eThe Attack Map module shows you visually where attacks are originating from geographically; meaning you can see where cyber attacks originated by looking at the sources IP Address mapped to its approximate geographical location (using available IP geolocation services). This information allows you to see where attacks originated, as well as display how malicious activity is globally distributed within the network that you are monitoring.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.4 Attack Map\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eLive Event Feed\u003c/strong\u003e \u003cp\u003eThe Live Event Feed module provides live feeds of detected security events (threats), with events being displayed on the dashboard immediately after they have been detected (from the backend processing of the firewall logs). This module provides a near-real time view of threats as they happen, similar to a SOC (Security Operations Center) monitoring view.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.5 Live Event Feed\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Replay\u003c/strong\u003e \u003cp\u003eThe Attack Replay module allows analyst to look back, at attack sequences, historically; replaying attacks based on one or more attackers OR based on a specific time window. This feature allows for re-creation of an attack timeline, enabling security teams to gain insight into attacker behaviour and to then use this information to improve their security postures against such attacks.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eEvent Table\u003c/strong\u003e \u003cp\u003eThe Event Table module offers a well-defined structure to present a list of all events relating to firewalls stored in the system's DB (database). Each entry contains attributes like the time/date stamp, source IP address, destination Port, protocol used, severity level and the type of detected attack. The table allows for easy inspection and filtering of security events.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eSuspicious IP Reputation\u003c/strong\u003e \u003cp\u003eThe Suspicious IP Reputation module measures and rates the level of trust that an IP address has on the network. At the same time, the system tracks IPs that have been consistently causing high numbers of malicious events and then assigns these IPs with a flag marking them as suspicious. This information helps analysts to identify any potential repeating threat actors.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttacker Persistence\u003c/strong\u003e \u003cp\u003eIn the proposed system, the Attacker Persistence module tracks the number of repeated connection attempt from a single IP source. Through measuring how often and how long an attack is made, the system can determine whether that source IP is a persistent attacker against the network.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eIP Intelligence\u003c/strong\u003e \u003cp\u003eThe IP Intelligence module collects contextually relevant data on the attacking IP address. This can include information about where the attacking IP is physically located, which network provider is supplying the service to the attacker's IP, and the perceived threat level associated with that IP address. This contextual data improves situational awareness for the incident response team and helps them make more informed decisions during their response to an incident.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.6 Threat Intelligence Card\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eTargeted Port Analysis\u003c/strong\u003e \u003cp\u003eThe Targeted Port Analysis module finds the most attacked network ports. By analysing firewall logs' destination port field, the module can determine which services attackers may be trying to test or attack.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eProtocol Distribution\u003c/strong\u003e \u003cp\u003eThe Protocol Distribution module looks at the communication protocols used during events on the network. By reviewing the firewall logs, the module can verify whether or not the events were TCP, UD, or ICMP. Understanding protocol distribution can help identify abnormal traffic patterns.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eService Exposure Analysis\u003c/strong\u003e \u003cp\u003eThe Service Exposure Analysis module determines which services are most exposed to attempts at external access. By connecting targeted ports to service types, the module can identify those services that may require additional security hardening.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.7 Network Exposure Cards\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Trend\u003c/strong\u003e \u003cp\u003eThe Attack Trend module tracks long-term changes in attack activity. It will analyse historical event data to determine whether the number of attacks over a defined period of time has increased, decreased or remained constant.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Growth\u003c/strong\u003e \u003cp\u003eThe Attack Growth module calculates the rate of increase in the level of attack activity over a defined period of time. This provides security analysts with a metric for detecting the sudden acceleration of attack campaigns.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Heatmap\u003c/strong\u003e \u003cp\u003eThe Attack Heatmap Module records times and places (over time) where high volumes of attacks are occurring using a color-coded display.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Velocity\u003c/strong\u003e \u003cp\u003eThe Attack Velocity Module is used to rate the number of attacks that occur in very short periods of time and can be used to identify an attack that is happening due to an automated attack campaign or a bot.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.8 Temporal Analysis\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Intensity\u003c/strong\u003e \u003cp\u003eThe Attack Intensity Module is used to determine how many attacks are concentrated on a particular system during a defined period of time, with very high attack intensity values indicating either large-scale or coordinated attempts to attack.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAttack Correlation\u003c/strong\u003e \u003cp\u003eThe Attack Correlation Module identifies how different security events relate to one another. Through the analysis of event attributes such as their source (IP), port (target), and the time (timestamp), this correlations module can correlate and identify events that come from the same attack campaign or individual attack.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.9 Attack Behaviour\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAnomaly Detection\u003c/strong\u003e \u003cp\u003eThe Anomaly Detection Module identifies unusual patterns of traffic at the firewall. The anomaly detection module takes the current activity and compares it to historical activity, using the historical activity to detect any differences that could indicate an undetected or new form of attack.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFig: 4.10 Security Monitoring\u003c/p\u003e"},{"header":"V. Experimental Results and Discussion","content":"\u003cp\u003eA controlled laboratory environment was used to evaluate the MiniSIEM: A Log Analysis and Security monitoring System for firewall log monitoring in real-time, attack detection, and security analytics visualisation. The experimental setup included the following: An attacker machine generating simulated malicious traffic, a firewall machine running Ubuntu OS configured with Uncomplicated Firewall (UFW) and a host machine managing both the collecting of logs and processing of analytics. Several attack scenarios were created to assess the system's functionality, performance and reliability.\u003c/p\u003e \u003cp\u003e \u003cb\u003eA. Functionality and Correctness\u003c/b\u003e \u003c/p\u003e \u003cp\u003eFunctional testing was performed in order to establish if the solution would accurately retrieve, store and analyse firewall logs in real-time. The log collector retrieved firewalls logs from the Ubuntu Server using secure SSH communications by forwarding them to the backend processing engine. The log parser created structured security events from the firewall logs by extracting key attributes including source IP address, destination port, protocol type, timestamp and action performed by the firewall.\u003c/p\u003e \u003cp\u003eMultiple attack scenarios (i.e., port scans, repeated connection attempts and abnormal bursts of requests) were simulated as part of the evaluation of the detection capability. The rule classification engine identified each of these attacks correctly based upon the defined detection rules. The classification engine in the proposed system identifies sequential connection attempts from a single IP source across multiple ports as port scanning, and brute-force where repeated attempts to access the same service as potential behaviour of it.\u003c/p\u003e \u003cp\u003eThe system generated alerts based upon the detected suspicious activity and displayed the alerts via a real-time dashboard. The support of the visualization modules (e.g., attack timeline, severity distribution and attack types) displayed the processed events with high accuracy. It is clear from these results that the system can interpret firewall logs with accuracy and provide actionable cybersecurity insight based upon them.\u003c/p\u003e \u003cp\u003e \u003cb\u003eB. Performance and Scalability\u003c/b\u003e \u003c/p\u003e \u003cp\u003eThe evaluation of how well the system processes immense amounts of firewall logs placed priority upon whether or not it does so while responding in real-time. The backend processing engine was that it accomplishes this task by being subjected to continuous logging conditions, along with simulated attack traffic, as opposed to logs simply being processed at certain discrete intervals. The evaluation indicated all log ingestion and processing could occur with almost no delay.\u003c/p\u003e \u003cp\u003eThe log parsing function and the event classification function performed well when processing multiple logs/events at the same time. The backend of this application is developed with a lightweight architecture utilizing Flask and SQLite allowing for high performance and low system resource utilization. The dashboard user interface is also created with React, and displays security events (visually) at \"near real-time\" updates that refresh every five seconds.\u003c/p\u003e \u003cp\u003eBecause of its modular architecture, this solution can be expanded and adapted to support new types of detection modules and/or analytic modules for use with firewalls and firewall logs (shown below). The design of the system allows for these new modules to be added without affecting the core functionality of the system as a whole, therefore, making it easy to use in high volume, continuously generated, real-time firewall log monitoring environments.\u003c/p\u003e \u003cp\u003e \u003cb\u003eC. Security and Reliability\u003c/b\u003e \u003c/p\u003e \u003cp\u003eThe objective was to conduct a security evaluation to assess both the soundness of the authentication mechanisms utilized by the system and the reliability of the alert generation process. There is a JSON web token (JWT)-based authentication system in place that prevents unauthorized users from accessing the analytics dashboard.\u003c/p\u003e \u003cp\u003eBased on test results, unauthorized access attempts were blocked successfully. Communication channels between the application and server were appropriately secured so that data could be considered safe.\u003c/p\u003e \u003cp\u003eAn alert generation system was designed to be capable of accurately detecting and reporting suspicious activity. For example, anytime the classification engine detected a possible attack pattern, the alert generation system produced an alert and created an entry in the event record. All alerts were shown immediately on both the live event feed and in alert panels within the analytics dashboard.\u003c/p\u003e \u003cp\u003eIn addition to the real-time alert generation capabilities, the event record provides a historical record of security-related events that are available for use by security analysts as reference material when conducting investigations into past incidents or for evaluating patterns associated with cyber-attacks over an extended period of time. The reliable storage and retrieval of event records ensure that security-related data can be used as evidence in forensic investigations and incident response efforts.\u003c/p\u003e"},{"header":"VI. Future Work","content":"\u003cp\u003eThe proposed MiniSIEM: A Log Analysis \u0026amp; Security Monitoring System provides a solid foundation for real-time firewall monitoring, attack detection, and security analytics; nonetheless, there are many possible improvements that could make the system work better and allow it to do more. For example, one area of potential future development would be to integrate machine learning (ML) and artificial intelligence (AI) techniques into the attack classification engine, which would allow for intelligent detection of complex and unknown patterns of attacks. Through the use of historical log data collected from firewalls, ML models could discover abnormal behaviour and patterns of behaviour that traditional, rule-based detection systems would not have found. This would improve accuracy of detections and decrease false positive detections. In addition, the system could potentially support third-party threat intelligence sources (feeds) and third-party (IP) reputation services to provide additional contextual information about potentially malicious IP addresses, resulting in more accurate threat assessments. Future versions of the platform could also support automated incident response processes (e.g., updating firewall rules to block malicious IP addresses in real-time or temporarily restricting (black-listing) traffic from an IP address if that IP address has shown abnormal behaviour) and allowing automated incident response actions. Lastly, additional functionality could be added to the system to support multiple firewall vendors, distributed log collection from large enterprise networks, and providing additional forms of advanced analytics (e.g., predictive modelling of threats and long-term trends of attacks). The enhancements proposed for establishing an extensive, intelligent cybersecurity monitoring framework would enable the implementation of a much larger and more dynamic monitoring platform that would adequately fulfill today's security operations centers growing requirements for a robust technology base to support their operational needs.\u003c/p\u003e"},{"header":"VII. Conclusion","content":"\u003cp\u003eThe proposed Firewall Security Analytics System is aimed at providing improved methods of enhancing the monitoring of network security through real-time analysis of firewall logs along with smart rendering. Automated log collection; structured event processing; attack classification by rules; event-trigger alerts; and support for interactive analytics dashboards are all features that provide a means to convert general firewall log data into high-quality cybersecurity intelligence. The established architecture provides for continuous monitoring of network activities, while also providing the security administrator with an overview of attack patterns, threat levels, and potential unsafe behaviours. Through the use of the analytical modules of attack timeline, severity distribution, attacker IP analysis, and geographic location based risk visualization of attacks, the administrator is able to quickly identify anomalies and respond to potential cybersecurity incidents. Both the performance based evaluation and test results show that the system is effective at processing firewall logs to provide high degrees of accuracy for identifying suspicious activities, ultimately resulting in actionable information being provided through real-time dashboards. The proposed modular/scalable architecture will allow it to be adaptable as the requirements associated with cybersecurity continue to evolve and to support additional analytical functionality over a more extended period of time. This proposed solution offers an inexpensive, practical framework for enhancing the effectiveness of firewall-based threat detection, thereby enhancing the effectiveness of proactive cybersecurity defence mechanisms for today\u0026rsquo;s networks.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eC.R.K., A.D., and H.G. contributed to the conception, design, and development of the system, including implementation, testing, and validation. C.R.K. performed the data analysis and drafted the manuscript. A.D. prepared the figures, and H.G. prepared the tables. V.D. contributed to the supervision of the work, provided critical feedback, and reviewed the manuscript for important intellectual content. All authors approved the final version of the manuscript and agree to be accountable for all aspects of the work.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe data used in this study was generated from firewall logs collected in a controlled experimental environment using Ubuntu UFW. Due to security and privacy concerns, the dataset is not publicly available.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003ePartovian, S.: Analysis of Log Files to Enable Smart-Troubleshooting in Industry 4.0: A Systematic Mapping Study. IEEE Access. \u003cb\u003e12\u003c/b\u003e, 147640\u0026ndash;147660 (2024)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarlaithong, T.: A Log Parsing Framework for ALICE O2 Facilities. IEEE Access. \u003cb\u003e11\u003c/b\u003e, 69439\u0026ndash;69457 (2023)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaeiszadeh, M.: ALogSCAN: A Self-Supervised Dual Network for Adaptive and Timely Log Anomaly Detection in Clouds. IEEE Trans. Mach. Learn. Commun. Netw., (2025)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHorv\u0026aacute;th, A.: et. Al., Anomaly Detection Algorithms for Real-Time Log Data Analysis at Scale. IEEE Access., \u003cb\u003e13\u003c/b\u003e, (2025)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlzamil, A.: DualBERT: Fusing Symbolic and Temporal Dynamics for High-Precision Log Anomaly Detection. IEEE Access., \u003cb\u003e14\u003c/b\u003e, (2026)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAboudrar, Y.: AI-Driven Firewall Log Analysis: Enhancing Threat Detection with Deep Learning Techniques. Int. J. Adv. Comput. Sci. Appl. \u003cb\u003e16\u003c/b\u003e(7), 808\u0026ndash;815 (2025)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchufrin, M.: Visual Firewall Log Analysis \u0026ndash; At the Border Between Analytical and Appealing. IEEE Visualization Workshop, (2018)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee, J.K.: Traffic and Overhead Analysis of Applied Pre-filtering ACL Firewall on HPC Service Network. J. Commun. Netw. \u003cb\u003e23\u003c/b\u003e(3), 192\u0026ndash;200 (2021)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDurante, L.: A Formal Model and Technique to Redistribute the Packet Filtering Load in Multiple Firewall Networks. IEEE Trans. Inf. Forensics Secur. \u003cb\u003e16\u003c/b\u003e, 2637\u0026ndash;2648 (2021)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBlack, G.: Descriptor: Firewall Attack Detections and Extractions (FADE). IEEE Data Descriptions, (2025)\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Cybersecurity, Firewall Monitoring, Security Analytics, Intrusion Detection, Flask, React, Attack Classification, Network Security","lastPublishedDoi":"10.21203/rs.3.rs-9249355/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9249355/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCybercriminals now have access to a much larger attack surface due to the ever-increasing reliance on internet-connected devices. This has resulted in an increase in network-based attacks such as port scanning, brute-force logins, distributed denial-of-service attacks, and attempts to gain unauthorized access [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Firewalls usually serve as a first line of defence for most networks, however, they generate a lot of log data which is not often properly analysed due to difficulties with manual analysis [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. The objective of this study is to design and implement a Firewall Security Analytics System that will collect and analyse firewall logs to identify potentially malicious network activity, classify the attacks, and provide real-time monitoring of security events. The general design includes a client-server architecture using a Flask backend and a React-based Security Operations Center (SOC) dashboard. The firewall logs are collected from an Ubuntu server using SSH-based secure log ingestion into a structured database format. The classification of attack patterns is accomplished by an analysis engine that utilizes rules to determine whether or not an event is suspicious (for example: port scanning, brute-force attempts, and abnormal connection activity). If a security threshold is exceeded, an alerts engine will generate an alert to notify the administrator. In addition, the solution includes a dashboard where real-time analytics can be viewed to visualize current attacks on the network. Ultimately, the results indicate that the proposed platform is able to convert raw data from firewalls into action-based intelligence about the state of your network, giving the administrator a better understanding of the current status of their network, allowing for faster detection of potential threats.\u003c/p\u003e","manuscriptTitle":"MiniSIEM: A Log Analysis \u0026amp; Security Monitoring System","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-31 06:20:49","doi":"10.21203/rs.3.rs-9249355/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"080e2328-13f5-4457-80e9-d2217e6b1841","owner":[],"postedDate":"March 31st, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-03-31T06:20:51+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-31 06:20:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9249355","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9249355","identity":"rs-9249355","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}