Clustering Textual Features for Log Summarization in Large Software Systems

Clustering Textual Features for Log Summarization in Large Software Systems | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 4 February 2025 V1 Latest version Share on Clustering Textual Features for Log Summarization in Large Software Systems Authors : Vithor Bertalan 0000-0002-1585-7694 [email protected] and Daniel Aloise Authors Info & Affiliations https://doi.org/10.22541/au.173865299.90947187/v1 268 views 151 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Identifying which lines deserve attention within large software log files can be a challenging task. Log files have consistently increased, reflecting the growth of software development platforms that are becoming larger and integrated. However, software engineers rarely have the time to thoroughly analyze these files to identify important information. To address this issue, data mining methods have been proposed with the intent of summarizing log lines within large log datasets. In this work, we propose a supervised log summarization method based on clustering, which groups log data by using integrated information from (i) log line embeddings, (ii) identified variables extracted from parsed log lines, and (iii) the proximity between log lines. From the obtained clusters, we apply methods for topic modeling and word analysis to summarize and indicate which lines deserve more attention in a log file. Our quantitative analysis on various log datasets demonstrates that our approach outperforms state-of-the-art text summarization methods, thereby showing that the clustering method and the combination of (i)-(iii) are crucial in achieving high accuracy scores for diverse log structures. Finally, we outline the implementation of our method with our corporate partner, highlighting the feedback received and the adjustments made to enhance its practical use. Supplementary Material File (paper_journal_of_software__evolution_and_process.pdf) Download 203.06 KB Information & Authors Information Version history V1 Version 1 04 February 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords log summarization machine learning software mining Authors Affiliations Vithor Bertalan 0000-0002-1585-7694 [email protected] Polytechnique Montreal View all articles by this author Daniel Aloise Polytechnique Montreal View all articles by this author Metrics & Citations Metrics Article Usage 268 views 151 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Vithor Bertalan, Daniel Aloise. Clustering Textual Features for Log Summarization in Large Software Systems. Authorea . 04 February 2025. DOI: https://doi.org/10.22541/au.173865299.90947187/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); Cited by Priyanka Mudgal, REFLEX: Reference-Free Evaluation of Log Summarization via Large Language Model Judgment, 2025 1st International Conference on Emerging Trends in Information Systems and Informatics (ICETISI), (1-7), (2025). https://doi.org/10.1109/ICETISI67983.2025.11405982 Crossref Loading... View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.173865299.90947187/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9ff4a708aef658f4',t:'MTc3OTM3Nzc0Mg=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();