Delocalized Photonic Deep Learning on the Internet's Edge

Delocalized Photonic Deep Learning on the Internet's Edge | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Physical Sciences - Article Delocalized Photonic Deep Learning on the Internet's Edge Alexander Sludds, Saumil Bandyopadhyay, Zaijun Chen, Zhizhen Zhong, and 12 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1513759/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Advances in deep neural networks (DNNs) are transforming science and technology. However, the increasing computational demands of the most powerful DNNs limit deployment on low-power devices, such as smartphones and sensors – and this trend is accelerated by the simultaneous move towards Internet-of-Things (IoT) devices. Numerous efforts are underway to lower power consumption, but a fundamental bottleneck remains due to energy consumption in matrix algebra, even for analog approaches including neuromorphic, analog memory and photonic meshes. Here we introduce and demonstrate a new approach that sharply reduces energy required for matrix algebra by doing away with weight memory access on edge devices, enabling orders of magnitude energy and latency reduction. At the core of our approach is a new concept that decentralizes the DNN for delocalized, optically accelerated matrix algebra on edge devices. Using a silicon photonic smart transceiver, we demonstrate experimentally that this scheme, termed Netcast, dramatically reduces energy consumption. We demonstrate operation in a photon-starved environment with 40 aJ/multiply of optical energy for 98.8% accurate image recognition and <1 photon/multiply using single photon detectors. Furthermore, we show realistic deployment of our system, classifying images with 3 THz of bandwidth over 86 km of deployed optical fiber in a Boston-area fiber network. Our approach enables computing on a new generation of edge devices with speeds comparable to modern digital electronics and power consumption that is orders of magnitude lower. Full Text Additional Declarations Yes there is potential Competing Interest. Matthew Streshinsky is the leader of silicon photonics at Nokia Corporation. Michael Hochberg is president of Luminous computing. Ari Novack is a system architect at Luminous computing. Tom Baehr-Jones is Vice-President of engineering at Luminous computing. Darius Bunandar is Chief Scientist at Lightmatter. Ryan Hamerly and Dirk Englund have filed a patent related to Netcast: PCT/US21/43593. Manya Ghobadi, Zhizhen Zhong, Liane Bernstein, Alexander Sludds, Ryan Hamerly and Dirk Englund have filed a provisional patent related to Netcast: 63/191,120 Other authors declare no competing interests. Supplementary Files SupplementaryDelocalizedPhotonicDeepLearningontheInternetsEdge.pdf Supplementary Materials for Delocalized Photonic Deep Learning on the Internet's Edge Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1513759","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Physical Sciences - Article","associatedPublications":[],"authors":[{"id":95834047,"identity":"c1660133-d0e2-441f-9724-ac578583cbb9","order_by":0,"name":"Alexander Sludds","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAsklEQVRIiWNgGAWjYFAC5gYGHjYGOQZmEIeNKC2MYC3GDMzMJGpJbGAgVgt//8E2iTdldelr2/kPMHwoO0xYi8SNxDbJOecO5247zMzAOOMcEVoYbjC2SfO2HQBrYeZtI0KL/PmDIC116WYgLX+J0WJwIBGkhTkBrIWRGC2GNxKbLYF+MQQ6zOBgz7l0wlrkzh8+eAMYYvJm5w8+fPCjzJqwFhRwgET1o2AUjIJRMApwAQBZmjnjBmp/7QAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-7443-1378","institution":"Massachusetts Institute of Technology","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Alexander","middleName":"","lastName":"Sludds","suffix":""},{"id":95834048,"identity":"99ee48a4-c40c-45bb-a320-b87d9f68d2c3","order_by":1,"name":"Saumil Bandyopadhyay","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Saumil","middleName":"","lastName":"Bandyopadhyay","suffix":""},{"id":95834049,"identity":"4febf051-a00c-48f9-834f-e3a63fde1c43","order_by":2,"name":"Zaijun Chen","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Zaijun","middleName":"","lastName":"Chen","suffix":""},{"id":95834050,"identity":"36debc61-5004-4bdb-a221-acf566392fc5","order_by":3,"name":"Zhizhen Zhong","email":"","orcid":"https://orcid.org/0000-0003-3131-374X","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Zhizhen","middleName":"","lastName":"Zhong","suffix":""},{"id":95834051,"identity":"a671144b-fe06-4798-9687-c830b02a43fb","order_by":4,"name":"Jared Cochrane","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Jared","middleName":"","lastName":"Cochrane","suffix":""},{"id":95834052,"identity":"876fc8df-888e-4bb5-9727-98fe76326457","order_by":5,"name":"Liane Bernstein","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Liane","middleName":"","lastName":"Bernstein","suffix":""},{"id":95834053,"identity":"12573eec-3fc1-46b4-971b-aa28d1c4b330","order_by":6,"name":"Darius Bunandar","email":"","orcid":"https://orcid.org/0000-0002-8218-5656","institution":"MIT","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Darius","middleName":"","lastName":"Bunandar","suffix":""},{"id":95834054,"identity":"448f606b-c645-45bd-a0f7-274a982a4622","order_by":7,"name":"P. Ben Dixon","email":"","orcid":"","institution":"MIT Lincoln Laboratory","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"P.","middleName":"Ben","lastName":"Dixon","suffix":""},{"id":95834055,"identity":"ed780521-9709-4fde-92b8-a618076f0cd6","order_by":8,"name":"Scott Hamilton","email":"","orcid":"","institution":"MIT Lincoln Laboratory","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Scott","middleName":"","lastName":"Hamilton","suffix":""},{"id":95834056,"identity":"b83c60ad-383a-42e8-8a2e-cebe65091884","order_by":9,"name":"Matthew Streshinsky","email":"","orcid":"","institution":"Nokia Corporation","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Matthew","middleName":"","lastName":"Streshinsky","suffix":""},{"id":95834057,"identity":"9e1aadf2-52c9-44a9-8d12-f5cc571a080b","order_by":10,"name":"Ari Novack","email":"","orcid":"","institution":"Nokia Corporation","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ari","middleName":"","lastName":"Novack","suffix":""},{"id":95834058,"identity":"1da9abab-7196-4151-8cf2-55fe258476d7","order_by":11,"name":"Tom Baehr-Jones","email":"","orcid":"","institution":"Coriant","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Tom","middleName":"","lastName":"Baehr-Jones","suffix":""},{"id":95834059,"identity":"2b12a966-5a86-4e9a-b62c-ec4a418ff034","order_by":12,"name":"Michael Hochberg","email":"","orcid":"","institution":"University of Washington","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Hochberg","suffix":""},{"id":95834060,"identity":"d4573db6-f557-417a-bfb6-b5c217953486","order_by":13,"name":"Manya Ghobadi","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Manya","middleName":"","lastName":"Ghobadi","suffix":""},{"id":95834061,"identity":"2a1573aa-c3fd-48a2-9b51-f46a59baa2a5","order_by":14,"name":"Ryan Hamerly","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ryan","middleName":"","lastName":"Hamerly","suffix":""},{"id":95834062,"identity":"b3aa207c-a011-4142-8ddc-b356c7854efe","order_by":15,"name":"Dirk Englund","email":"","orcid":"https://orcid.org/0000-0002-1043-3489","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Dirk","middleName":"","lastName":"Englund","suffix":""}],"badges":[],"createdAt":"2022-04-01 13:56:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1513759/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1513759/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":20034611,"identity":"8efc81e0-e200-4f34-9a7b-bd281c704c56","added_by":"auto","created_at":"2022-04-06 17:21:51","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3227533,"visible":true,"origin":"","legend":"Article File","description":"","filename":"DelocalizedPhotonicDeepLearningontheInternetsEdge.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1513759/v1_covered.pdf"},{"id":20034610,"identity":"75d2fa55-7b02-4e3a-8bc0-8563934e5fa5","added_by":"auto","created_at":"2022-04-06 17:21:40","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":13688439,"visible":true,"origin":"","legend":"Supplementary Materials for Delocalized Photonic Deep Learning on the Internet's Edge","description":"","filename":"SupplementaryDelocalizedPhotonicDeepLearningontheInternetsEdge.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1513759/v1/b617fc06e19d0937b907a161.pdf"}],"financialInterests":"\u003cb\u003eYes\u003c/b\u003e there is potential Competing Interest.\nMatthew Streshinsky is the leader of silicon photonics at Nokia Corporation.\r\n\r\nMichael Hochberg is president of Luminous computing.\r\nAri Novack is a system architect at Luminous computing.\r\nTom Baehr-Jones is Vice-President of engineering at Luminous computing. \r\n\r\nDarius Bunandar is Chief Scientist at Lightmatter.\r\n\r\nRyan Hamerly and Dirk Englund have filed a patent related to Netcast: PCT/US21/43593. \r\n\r\nManya Ghobadi, Zhizhen Zhong, Liane Bernstein, Alexander Sludds, Ryan Hamerly and Dirk Englund have filed a provisional patent related\r\nto Netcast: 63/191,120 \r\n\r\nOther authors declare no competing interests.","formattedTitle":"Delocalized Photonic Deep Learning on the Internet's Edge","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-1513759/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-1513759/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1513759/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Advances in deep neural networks (DNNs) are transforming science and technology. However, the increasing computational demands of the most powerful DNNs limit deployment on low-power devices, such as smartphones and sensors – and this trend is accelerated by the simultaneous move towards Internet-of-Things (IoT) devices. Numerous efforts are underway to lower power consumption, but a fundamental bottleneck remains due to energy consumption in matrix algebra, even for analog approaches including neuromorphic, analog memory and photonic meshes. Here we introduce and demonstrate a new approach that sharply reduces energy required for matrix algebra by doing away with weight memory access on edge devices, enabling orders of magnitude energy and latency reduction. At the core of our approach is a new concept that decentralizes the DNN for delocalized, optically accelerated matrix algebra on edge devices. Using a silicon photonic smart transceiver, we demonstrate experimentally that this scheme, termed Netcast, dramatically reduces energy consumption. We demonstrate operation in a photon-starved environment with 40 aJ/multiply of optical energy for 98.8% accurate image recognition and \u003c1 photon/multiply using single photon detectors. Furthermore, we show realistic deployment of our system, classifying images with 3 THz of bandwidth over 86 km of deployed optical fiber in a Boston-area fiber network. Our approach enables computing on a new generation of edge devices with speeds comparable to modern digital electronics and power consumption that is orders of magnitude lower.","manuscriptTitle":"Delocalized Photonic Deep Learning on the Internet's Edge","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-04-06 17:21:39","doi":"10.21203/rs.3.rs-1513759/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6f734026-5824-47b2-9b28-a0d7cffce308","owner":[],"postedDate":"April 6th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2022-04-20T14:42:33+00:00","versionOfRecord":[],"versionCreatedAt":"2022-04-06 17:21:39","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1513759","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1513759","identity":"rs-1513759","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}