GCAM: Gaussian and causal-attention model of food fine-grained recognition

GCAM: Gaussian and causal-attention model of food fine-grained recognition | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article GCAM: Gaussian and causal-attention model of food fine-grained recognition Guohang Zhuang, Yue Hu, Tianxing Yan, Jiazhang Gao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4134165/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 20 Jun, 2024 Read the published version in Signal, Image and Video Processing → Version 1 posted 7 You are reading this latest preprint version Abstract Currently, most food recognition relies on deep learning for category classification. However, these approaches struggle to effectively distinguish between visually similar food samples, highlighting the pressing need to address fine-grained issues in food recognition. To address these issues, we advocate for a Gaussian and causal-attention model specifically designed for nuanced object recognition. This model involves training to capture Gaussian characteristics in targeted areas, followed by extracting detailed features from the objects, thus improving the target regions’ feature mapping capabilities. To counter data drift caused by skewed data distributions, we implement a counterfactual reasoning strategy. Through counterfactual interventions, the effect of the learned image attention mechanism on network predictions is examined, allowing for the optimization of attention weights in detailed image recognition. A learnable loss strategy is also developed to ensure consistent training across various modules, thereby enhancing the precision of the ultimate recognition task. Our method has been validated on four 1 pertinent datasets, where it demonstrated superior performance. Specifically, the Gaussian and Causal-Attention Model (GCAM) has outperformed existing state-of-the-art methods on the ETH-FOOD101, UECFOOD256, and Vireo-FOOD172 datasets and achieved leading results on the CUB-200 dataset. Gaussian function Counterfactuals are inferences Fine-grained identification of food attention mechanism Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 20 Jun, 2024 Read the published version in Signal, Image and Video Processing → Version 1 posted Editorial decision: Revision requested 12 Apr, 2024 Reviews received at journal 09 Apr, 2024 Reviewers agreed at journal 21 Mar, 2024 Reviewers invited by journal 20 Mar, 2024 Submission checks completed at journal 20 Mar, 2024 Editor assigned by journal 20 Mar, 2024 First submitted to journal 20 Mar, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4134165","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":282995214,"identity":"f7a68509-775c-420a-9d62-1584bdfd489c","order_by":0,"name":"Guohang Zhuang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDklEQVRIiWNgGAWjYLCCB3CGDYMciD7wAJdSGEhgYJCAMNIMjMFaEkjRktgAFcEJ5Nt7H35IqKmrM28/e/BBQsKf9Plhhx8CbbGT023AroWx57ixRMKxwxIyZ/KSDRISDHI33k4zAGpJNjY7gF0Ls0Qag0QC2wEJCYYcM4nEH0AtsxNAWg4kbsOhhU0ijflHwr86CQn+N+Y/gLakG85O/4BXC49EGptEYhuzhIREjhkDUEuCvHQOflskeI6xWST2HZacIfEuWSIhwdhwg3ROwYEEA9x+kW9vY77x4VsdvwR/7sEPHxLk5OVnp2/+8KHCTg6XFmQ3QigDsEoDgsqRtMg3EKV6FIyCUTAKRhAAAOnhXZEYwT8GAAAAAElFTkSuQmCC","orcid":"","institution":"Hefei University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Guohang","middleName":"","lastName":"Zhuang","suffix":""},{"id":282995215,"identity":"98aaa32c-d141-47ed-ab99-b2a33d6c3969","order_by":1,"name":"Yue Hu","email":"","orcid":"","institution":"Northern University for Nationalities","correspondingAuthor":false,"prefix":"","firstName":"Yue","middleName":"","lastName":"Hu","suffix":""},{"id":282995216,"identity":"9a1d7103-901e-480e-97cd-d7c0cd33154a","order_by":2,"name":"Tianxing Yan","email":"","orcid":"","institution":"Xinjiang University","correspondingAuthor":false,"prefix":"","firstName":"Tianxing","middleName":"","lastName":"Yan","suffix":""},{"id":282995217,"identity":"1191f9ef-e7bb-41aa-b108-98ed9d177484","order_by":3,"name":"Jiazhang Gao","email":"","orcid":"","institution":"Xinjiang University","correspondingAuthor":false,"prefix":"","firstName":"Jiazhang","middleName":"","lastName":"Gao","suffix":""}],"badges":[],"createdAt":"2024-03-20 04:59:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4134165/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4134165/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s11760-024-03383-y","type":"published","date":"2024-06-21T00:26:08+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":58841959,"identity":"385fc831-980e-49c2-b10a-9b4735d5e40f","added_by":"auto","created_at":"2024-06-22 00:26:19","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3731787,"visible":true,"origin":"","legend":"","description":"","filename":"GCAMGaussianandcausalattentionmodeloffoodfinegrainedrecognition.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4134165/v1_covered_beb67fb8-c7db-4324-aa28-fdfc571f54f9.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"GCAM: Gaussian and causal-attention model of food fine-grained recognition","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Gaussian function, Counterfactuals are inferences, Fine-grained identification of food, attention mechanism","lastPublishedDoi":"10.21203/rs.3.rs-4134165/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4134165/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Currently, most food recognition relies on deep learning for category classification. However, these approaches struggle to effectively distinguish between visually similar food samples, highlighting the pressing need to address fine-grained issues in food recognition. To address these issues, we advocate for a Gaussian and causal-attention model specifically designed for nuanced object recognition. This model involves training to capture Gaussian characteristics in targeted areas, followed by extracting detailed features from the objects, thus improving the target regions’ feature mapping capabilities. To counter data drift caused by skewed data distributions, we implement a counterfactual reasoning strategy. Through counterfactual interventions, the effect of the learned image attention mechanism on network predictions is examined, allowing for the optimization of attention weights in detailed image recognition. A learnable loss strategy is also developed to ensure consistent training across various modules, thereby enhancing the precision of the ultimate recognition task. Our method has been validated on four 1 pertinent datasets, where it demonstrated superior performance. Specifically, the Gaussian and Causal-Attention Model (GCAM) has outperformed existing state-of-the-art methods on the ETH-FOOD101, UECFOOD256, and Vireo-FOOD172 datasets and achieved leading results on the CUB-200 dataset.","manuscriptTitle":"GCAM: Gaussian and causal-attention model of food fine-grained recognition","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-25 16:02:34","doi":"10.21203/rs.3.rs-4134165/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-04-13T02:46:24+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-04-10T02:09:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"3bd1a5d5-16c0-448b-8df2-0488d4b43181","date":"2024-03-21T14:21:25+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-03-20T20:13:37+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-03-20T08:48:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-03-20T08:48:47+00:00","index":"","fulltext":""},{"type":"submitted","content":"Signal, Image and Video Processing","date":"2024-03-20T04:55:09+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"ae16add4-541d-44c8-b29b-b8afd726db16","owner":[],"postedDate":"March 25th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-06-22T00:26:08+00:00","versionOfRecord":{"articleIdentity":"rs-4134165","link":"https://doi.org/10.1007/s11760-024-03383-y","journal":{"identity":"signal-image-and-video-processing","isVorOnly":false,"title":"Signal, Image and Video Processing"},"publishedOn":"2024-06-21 00:26:08","publishedOnDateReadable":"June 21st, 2024"},"versionCreatedAt":"2024-03-25 16:02:34","video":"","vorDoi":"10.1007/s11760-024-03383-y","vorDoiUrl":"https://doi.org/10.1007/s11760-024-03383-y","workflowStages":[]},"version":"v1","identity":"rs-4134165","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4134165","identity":"rs-4134165","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}