Lottery Ticket Hypothesis in Large Language Models: A Comprehensive Analysis of GPT-2 Weight Dynamics and Pruning Efficiency

Lottery Ticket Hypothesis in Large Language Models: A Comprehensive Analysis of GPT-2 Weight Dynamics and Pruning Efficiency | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 28 October 2025 V1 Latest version Share on Lottery Ticket Hypothesis in Large Language Models: A Comprehensive Analysis of GPT-2 Weight Dynamics and Pruning Efficiency Authors : Ngoc Kim Khanh Nguyen 0000-0003-3542-0518 [email protected] , Gia Khang Vo , Ngoc Quan Phan , and Quang Nguyen Authors Info & Affiliations https://doi.org/10.22541/au.176168540.05810555/v1 424 views 128 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract The Lottery Ticket Hypothesis (LTH) posits that within randomly initialized neural networks, there exist subnetworks that can achieve comparable performance to the full network when trained in isolation. While extensively studied in computer vision tasks, LTH’s applicability to large language models (LLMs) remains largely unexplored. This paper presents a comprehensive analysis of weight dynamics and pruning efficiency in GPT-2, investigating the hypothesis across three model variants: randomly initialized, pre-trained, and fine-tuned. We employ Iterative Magnitude Pruning (IMP) with layer-wise progressive sparsity, achieving up to 60% weight removal while maintaining minimal performance degradation (perplexity increase < 10%). Our analysis reveals distinct weight distribution patterns across model states and demonstrates that LTH holds significant promise for LLM compression. The findings provide novel insights into transformer weight dynamics and establish practical guidelines for efficient model pruning in natural language processing applications. Supplementary Material File (engineeringreports_lottery_ticket_hypothesis.docx) Download 867.28 KB Information & Authors Information Version history V1 Version 1 28 October 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords gpt-2 lottery ticket hypothesis model pruning weight analysis Authors Affiliations Ngoc Kim Khanh Nguyen 0000-0003-3542-0518 [email protected] VNUHCM-Ho Chi Minh City University of Technology View all articles by this author Gia Khang Vo VNUHCM-Ho Chi Minh City University of Technology View all articles by this author Ngoc Quan Phan VNUHCM-Ho Chi Minh City University of Technology View all articles by this author Quang Nguyen VNUHCM-Ho Chi Minh City University of Technology View all articles by this author Metrics & Citations Metrics Article Usage 424 views 128 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Ngoc Kim Khanh Nguyen, Gia Khang Vo, Ngoc Quan Phan, et al. Lottery Ticket Hypothesis in Large Language Models: A Comprehensive Analysis of GPT-2 Weight Dynamics and Pruning Efficiency. Authorea . 28 October 2025. DOI: https://doi.org/10.22541/au.176168540.05810555/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.176168540.05810555/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a0065cfb1dee8650',t:'MTc3OTU2MzQ1Mg=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();