On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 19 June 2025 V1 Latest version Share on On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study Authors : Daniel Rodriguez-Cardenas 0000-0002-3238-1229 [email protected] , David N. Palacio , Anna Schmedding , Yiyang Lu , Bill Hudson , Chris Gourley , Michael Roytman , Chris Shenefiel , Evgenia Smirni , and Denys Poshyvanyk Authors Info & Affiliations https://doi.org/10.22541/au.175033470.05916058/v1 252 views 203 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Modern software systems demand earlier vulnerability severity detection to protect systems from critical issues such as data leaking or attacker access. Security analysts are in charge of triaging the vulnerability severity by computing a score following security standard methods. The triaging of vulnerability severity becomes critical as the number of vulnerabilities increases. In addition, this severity classification is also relevant for timely detection of vulnerabilities. Large Language Models (LLMs) have been demonstrated to assist security analysts in vulnerability detection and fixing. However, private models ( e.g., ChatGPT, Copilot, and Claude) require sending registered code and proprietary data to third-party companies. We present an industry experience report that uses registered and open data datasets to assess the effectiveness of predicting vulnerability severity. We designed an in-context learning solution to predict the severity score using open-source LLMs such as CodeLlama2 and Mistral . We observe that CodeLlama2-7B can predict the severity score with an average MSE of ≈7.62 in one shot. Our findings indicate CodeLlama2 is a promising LLM to assist security analysts in assessing the severity impact before deployment while protecting data exposure. Supplementary Material File (dtvhbctsfympfmsvvmfdjgxhtmgfjxqy.pdf) Download 973.72 KB Information & Authors Information Version history V1 Version 1 19 June 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords large language models prediction security severity vulnerability Authors Affiliations Daniel Rodriguez-Cardenas 0000-0002-3238-1229 [email protected] Willam & Mary Department of Applied Science View all articles by this author David N. Palacio Willam & Mary Department of Applied Science View all articles by this author Anna Schmedding Willam & Mary Department of Applied Science View all articles by this author Yiyang Lu Willam & Mary Department of Applied Science View all articles by this author Bill Hudson Cisco Systems Inc View all articles by this author Chris Gourley Cisco Systems Inc View all articles by this author Michael Roytman Cisco Systems Inc View all articles by this author Chris Shenefiel Willam & Mary Department of Applied Science View all articles by this author Evgenia Smirni Willam & Mary Department of Applied Science View all articles by this author Denys Poshyvanyk Willam & Mary Department of Applied Science View all articles by this author Metrics & Citations Metrics Article Usage 252 views 203 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Daniel Rodriguez-Cardenas, David N. Palacio, Anna Schmedding, et al. On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study. Authorea . 19 June 2025. DOI: https://doi.org/10.22541/au.175033470.05916058/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.175033470.05916058/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a0028c2f5c190db4',t:'MTc3OTUyMzQ0Mw=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();