Benford’s Law in Basic RNN and Long Short-Term Memory (LSTM) and their Associations

Benford’s Law in Basic RNN and Long Short-Term Memory (LSTM) and their Associations | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL Applied AI Letters This is a preprint and has not been peer reviewed. Data may be preliminary. 8 February 2025 V1 Latest version Share on Benford’s Law in Basic RNN and Long Short-Term Memory (LSTM) and their Associations Author : Farshad Ghassemi Toosi 0000-0002-1105-4819 [email protected] Authors Info & Affiliations https://doi.org/10.22541/au.173901291.14107238/v1 301 views 231 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Context: Benford’s Law describes the distribution of numerical patterns, specifically focusing on the frequency of the leading digit in a set of natural numbers. It divides these numbers into nine groups based on their first digit, with the largest category comprising numbers beginning with 1, followed by those starting with 2, and so on. Objective: Each neuron within a Neural Network (NN) is associated with a numerical value called a weight, which is updated according to specific functions. This research examines the Degree of Benford’s Law Existence (DBLE) across two language model methodologies: (1) Recurrent Neural Networks (RNNs), and (2) Long Short-Term Memory (LSTM). Additionally, this study investigates whether models with higher performance exhibit a stronger presence of DBLE. Methods: Two neural network language models, namely: (1) Simple RNN and (2) LSTM, were selected as the subject models for the experiment. Each model is tested with five different optimizers and four different datasets (textual corpora selected from Wikipedia). This results in a total of 20 different configurations for each model. The neuron weights for each configuration were extracted at each epoch, and the following metrics were measured at each epoch: (1) DBLE, (2) training set accuracy, (3) training set error, (4) test set accuracy, and (5) test set error. Results: The results show that the weights in both models, across all optimizers, follow Benford’s Law. Additionally, the findings indicate a strong correlation between DBLE and the performance on the training set in both language models. This means that models with higher performance on training set, exhibit a stronger of DBLE. Benford’s Law, Recurrent Neural Network, Machine Learning, Language Model, Text Generation, Natural Language Processing, LSTM Supplementary Material File (farshad ghassemi toosi.pdf) Download 2.50 MB Information & Authors Information Version history V1 Version 1 08 February 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Collection Applied AI Letters Keywords bedford's law model accuracy neural network weight Authors Affiliations Farshad Ghassemi Toosi 0000-0002-1105-4819 [email protected] Munster Technological University Faculty of Engineering & Science View all articles by this author Metrics & Citations Metrics Article Usage 301 views 231 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Farshad Ghassemi Toosi. Benford’s Law in Basic RNN and Long Short-Term Memory (LSTM) and their Associations. Authorea . 08 February 2025. DOI: https://doi.org/10.22541/au.173901291.14107238/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); Cited by Mohsin Raza, Zahid Kumail, Tahsin Nawaz, Syed Zia Uddin, Advancing Energy Systems with CFD from Renewable Technologies to Nuclear Reactor Safety, Advanced Computational and Mathematical Techniques for Wave Energy Converter Systems, (119-153), (2026). https://doi.org/10.1007/978-3-032-10804-3_6 Crossref Mohsin Raza, Muazzam Faiz, Waqar U. I. Hassan, Muzamil Abbas, Jawad Raza, Zahid Kumail, Tahsin Nawaz, Sania Shabir, Ali Jan, Feng-Chen Li, AI-powered optimization and numerical techniques for nanofluid heat transfer systems-a review, Multiscale and Multidisciplinary Modeling, Experiments and Design, 8 , 7, (2025). https://doi.org/10.1007/s41939-025-00891-3 Crossref Loading... View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.173901291.14107238/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9ff48af02884aa64',t:'MTc3OTM3NjU5MQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();