The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches toppings

The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches toppings | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches toppings Esther Kok, Aneesh Chauhan, Michele Tufano, Edith Feskens, Guido Camps This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4647979/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Accurate measurement of dietary intake without interfering in natural eating habits is a long-standing problem in nutritional epidemiology. We explore the applicability of hyperspectral imaging and machine learning for dietary assessment of home-prepared meals, by building a proof-of-concept which automatically detects food ingredients inside closed sandwiches. Individual spectra were selected from 24 hyperspectral images of assembled closed sandwiches, preprocessed with SNV-filtering, derivatives, and subsampling, and fed into a multilayer perceptron. The resulting models had an accuracy score of ~ 80% prediction of the type of bread, ~ 60% for predicting butter, and ~ 24% for filling type. Further analysis on non-homogeneous mixed food items, using computer vision techniques, will contribute towards a generalizable system. While there are still significant technical challenges to overcome before such a system can be routinely implemented in studies of free-living subjects, we believe it holds promise as a future tool for nutrition research and population intake monitoring. hyperspectral imaging machine learning neural networks dietary assessment nutritional epidemiology spectral classification Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4647979","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":329065857,"identity":"01b9bd75-35fa-4aec-bf57-61ad7fa8c0d7","order_by":0,"name":"Esther Kok","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyklEQVRIiWNgGAWjYHACNiCWAEI2xgdgPjtBHcwgLRYgLcwGEAHitFSAtLBJEKWFn//8sQcf90gwSLa3pVXztt1jMCekRXJGMrvhjGcSDNI8x47d5m0rZrBsJqDF4AYzmzTPAQkGOYn0ttu5bQkMBocJaLE/f5hN+g9USzFRWgwYktmkGYBapCXSjjETpUXiRrKZZM8BCR7JnmPJ0n/OJfAQ1MLff/CZxI8DdXISx9sMP84oS5AzON5AQA8U8GAwRsEoGAWjYBRQAABPXDdXsiki+wAAAABJRU5ErkJggg==","orcid":"","institution":"Wageningen University and Research (WUR)","correspondingAuthor":true,"prefix":"","firstName":"Esther","middleName":"","lastName":"Kok","suffix":""},{"id":329065860,"identity":"8736e2b0-8abc-40de-a410-522a192b3d6e","order_by":1,"name":"Aneesh Chauhan","email":"","orcid":"","institution":"Wageningen University and Research (WUR)","correspondingAuthor":false,"prefix":"","firstName":"Aneesh","middleName":"","lastName":"Chauhan","suffix":""},{"id":329065862,"identity":"199ffe23-6cd5-4f7a-b33f-19468147af82","order_by":2,"name":"Michele Tufano","email":"","orcid":"","institution":"Wageningen University and Research (WUR)","correspondingAuthor":false,"prefix":"","firstName":"Michele","middleName":"","lastName":"Tufano","suffix":""},{"id":329065863,"identity":"7d0f6755-86b9-476b-8106-4d2ae7543be4","order_by":3,"name":"Edith Feskens","email":"","orcid":"","institution":"Wageningen University and Research (WUR)","correspondingAuthor":false,"prefix":"","firstName":"Edith","middleName":"","lastName":"Feskens","suffix":""},{"id":329065864,"identity":"36044af0-a972-428b-b014-739bce9efa77","order_by":4,"name":"Guido Camps","email":"","orcid":"","institution":"Wageningen University and Research (WUR)","correspondingAuthor":false,"prefix":"","firstName":"Guido","middleName":"","lastName":"Camps","suffix":""}],"badges":[],"createdAt":"2024-06-27 10:58:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4647979/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4647979/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":66646789,"identity":"13ba597b-965b-45e9-b1c5-34c4d16cd41b","added_by":"auto","created_at":"2024-10-15 07:02:08","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":525897,"visible":true,"origin":"","legend":"","description":"","filename":"SWIRHSISandwichEKokManuscriptsubmission27062024.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4647979/v1_covered_d7c036e8-1fed-4b94-a19e-86fba7af58d5.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches toppings","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"hyperspectral imaging, machine learning, neural networks, dietary assessment, nutritional epidemiology, spectral classification","lastPublishedDoi":"10.21203/rs.3.rs-4647979/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4647979/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAccurate measurement of dietary intake without interfering in natural eating habits is a long-standing problem in nutritional epidemiology. We explore the applicability of hyperspectral imaging and machine learning for dietary assessment of home-prepared meals, by building a proof-of-concept which automatically detects food ingredients inside closed sandwiches.\u003c/p\u003e \u003cp\u003eIndividual spectra were selected from 24 hyperspectral images of assembled closed sandwiches, preprocessed with SNV-filtering, derivatives, and subsampling, and fed into a multilayer perceptron.\u003c/p\u003e \u003cp\u003eThe resulting models had an accuracy score of ~\u0026thinsp;80% prediction of the type of bread, ~\u0026thinsp;60% for predicting butter, and ~\u0026thinsp;24% for filling type.\u003c/p\u003e \u003cp\u003eFurther analysis on non-homogeneous mixed food items, using computer vision techniques, will contribute towards a generalizable system. While there are still significant technical challenges to overcome before such a system can be routinely implemented in studies of free-living subjects, we believe it holds promise as a future tool for nutrition research and population intake monitoring.\u003c/p\u003e","manuscriptTitle":"The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches toppings","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-24 19:06:59","doi":"10.21203/rs.3.rs-4647979/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3326e5fc-3e14-40d8-9456-2ce01137d186","owner":[],"postedDate":"July 24th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-10-15T06:53:53+00:00","versionOfRecord":[],"versionCreatedAt":"2024-07-24 19:06:59","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4647979","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4647979","identity":"rs-4647979","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}