Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories

Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 15 May 2025 V1 Latest version Share on Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories Author : Faruk Alpay 0009-0009-2207-6528 [email protected] Authors Info & Affiliations https://doi.org/10.22541/au.174733679.99964219/v1 114 views 58 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract The Information Bottleneck (IB) objective I(X; Z) − βI(Z; Y) is known to exhibit instability and abrupt phase transitions as its trade-off parameter β is varied. These instabilities manifest as volatile jumps in the encoder p(z|x) and sparse, hard cluster formations, posing challenges in high-sensitivity applications where smooth and stable representations are required. In this work, I introduce a convexified IB optimization framework that replaces the classical linear compression term with a strictly convex function u(I(X; Z)) (e.g., u(t) = t 2), and incorporate a small entropy regularization −ϵH(Z|X) to smooth encoder transitions. To trace solutions continuously as β increases, I develop a symbolic continuation method based on an implicit first-order ODE for the encoder, which serves as a predictor-corrector mechanism for following the IB path without bifurcating. My approach yields a stable IB solver that avoids sudden representation shifts by design. I demonstrate on synthetic datasets (binary symmetric channel and structured 8 × 8 distributions) that the method I propose eliminates abrupt phase changes, achieving smooth evolution of mutual information metrics and graceful cluster formation. The convexified and entropy-regularized IB solutions maintain higher stability and predictive performance across all β regimes, confirming the theoretical improvements. Supplementary Material File (stable_and_convexified_information_bottleneck_optimization_via_symbolic_continuation_and_entropy_regularized_trajectories-3.pdf) Download 1.25 MB Information & Authors Information Version history V1 Version 1 15 May 2025 Copyright This work is licensed under a Creative Commons Attribution 4.0 International License Keywords continuation methods convex optimisation dynamic and nonequilibrium phase transitions (general) in statistical mechanics entropy regularisation information bottleneck Authors Affiliations Faruk Alpay 0009-0009-2207-6528 [email protected] Independent Researcher View all articles by this author Metrics & Citations Metrics Article Usage 114 views 58 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Faruk Alpay. Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories. Authorea . 15 May 2025. DOI: https://doi.org/10.22541/au.174733679.99964219/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); Cited by Ruchi Garg, Sumit Kumar, Optimizing Supervised Learning via Entropy Regularization, 2026 1st International Conference on Advancing Sustainable Solutions through Technologies (ICASST), (1-6), (2026). https://doi.org/10.1109/ICASST68917.2026.11484499 Crossref Loading... View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.174733679.99964219/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a00419e4ea281b23',t:'MTc3OTUzOTczMw=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();