Deep learning struggles to forecast yields in food-insecure, low-production regions

Deep learning struggles to forecast yields in food-insecure, low-production regions | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Deep learning struggles to forecast yields in food-insecure, low-production regions Di Qiao, Qianlong Dang, Junhu Ruan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6800608/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Despite the growing prominence of deep learning in yield forecasting, we show that deep learning-based models consistently underperform where they are needed most: in food-insecure, low-yield regions. Our analysis reveals that while these models perform well in medium- and high-yield regions, they fail in areas characterized by high yield variability and persistent food security challenges. This discrepancy arises in part because the use of natural environmental predictors primarily reflects short-term variability, while overlooking baseline yield disparities that define low-yield regions. Furthermore, conventional training practices—focused on optimizing overall accuracy—systematically neglect rare but critical low-yield outcomes. Recognizing these limitations, we propose an enhanced deep learning framework that combines local socio-economic features with the Fit-Gen Adaptive Attention (FGAA) algorithm, specifically designed to optimize for extreme yield events. Our approach significantly improves predictive robustness in low-yield regions, providing a practical pathway to better assess and manage food security risks in vulnerable areas. Crop yield forecast Food-insecure region Socio-economic feature Extremely low yield Deep learning Full Text Additional Declarations The authors declare no competing interests. Supplementary Information is not available with this version Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6800608","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":465211556,"identity":"72c184c7-2e03-4c49-aa88-ffef95f2dc26","order_by":0,"name":"Di Qiao","email":"","orcid":"","institution":"College of Economics and Management, Northwest A\u0026F University, Yangling 712100, China","correspondingAuthor":false,"prefix":"","firstName":"Di","middleName":"","lastName":"Qiao","suffix":""},{"id":465211557,"identity":"2345faca-0f20-4466-949b-135b594cb0b6","order_by":1,"name":"Qianlong Dang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYDACZgaDA0BKhgFEfjCwsSNaCw9IC+OMgrRkYuwxYIBpYeb5cIixgaD648wbD/zcYc3Dd/zs4dc2BgeYGdgPH92AT4tkM1vBwd4z6TySZ/LSrHMM7vAx8KSl3cCnhZ+Zx+AAb9thIJljZpxj8IyZQYLHDK8WNqCWg39BWs6/MTO2MDjM2EBIC8iWw2BbbuQYP2YgRgvIL4dl24B+ufHGjLHHIC2ZjZBfDM4f3vzxbZu1HN/5HOMPP/7Y2PGzHz6GVwsUMIP9JQEmiVAO18L8gUjVo2AUjIJRMMIAAFV5Sxb5VlNdAAAAAElFTkSuQmCC","orcid":"","institution":"College of Science, Northwest A\u0026F University, Yangling 712100, China","correspondingAuthor":true,"prefix":"","firstName":"Qianlong","middleName":"","lastName":"Dang","suffix":""},{"id":465211558,"identity":"118f7f2c-1bd4-4821-8d9d-0a6d29fc64b3","order_by":2,"name":"Junhu Ruan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAxUlEQVRIiWNgGAWjYFCCM4wPwDTzAQYGHiK1MBuAabYEorXwsEmQpsW88eyx6sK2usT+NgbGB2/bGOTNCWmROXAu7fbMtsOJM44xMBvObWMw3NlAQIsEwxmz27zbDiQ23G9gk+ZtY0gwOECElmLebXWJ848xsP8mWgsz7zbmxA3HGNiYidViLM3777DxxmOMzZJzzkkYbiCoReKM4WeeM3Wy844xH/zwpsxGnqAtDBJwFYwNYFsJA/4GIhSNglEwCkbByAYApqE+z1jwO38AAAAASUVORK5CYII=","orcid":"","institution":"College of Economics and Management, Northwest A\u0026F University, Yangling 712100, China","correspondingAuthor":true,"prefix":"","firstName":"Junhu","middleName":"","lastName":"Ruan","suffix":""}],"badges":[],"createdAt":"2025-06-02 09:23:13","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-6800608/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6800608/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":83858698,"identity":"e9cf1970-5857-4212-9aad-067961a60a3b","added_by":"auto","created_at":"2025-06-03 18:25:47","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1524511,"visible":true,"origin":"","legend":"","description":"","filename":"PreprintManuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6800608/v1_covered_44fa544d-5538-4df8-be55-27613305427a.pdf"}],"financialInterests":"\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003eSupplementary Information is not available with this version\u003c/p\u003e","formattedTitle":"\u003cp\u003eDeep learning struggles to forecast yields in food-insecure, low-production regions\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[{"identity":"d6794089-3d92-4eef-8200-5632683dc36f","identifier":"10.13039/501100013071","name":"Major Program of National Fund of Philosophy and Social Science of China","awardNumber":"23\u0026ZD123","order_by":0}],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Northwest A\u0026F University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Crop yield forecast, Food-insecure region, Socio-economic feature, Extremely low yield, Deep learning","lastPublishedDoi":"10.21203/rs.3.rs-6800608/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6800608/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDespite the growing prominence of deep learning in yield forecasting, we show that deep learning-based models consistently underperform where they are needed most: in food-insecure, low-yield regions. Our analysis reveals that while these models perform well in medium- and high-yield regions, they fail in areas characterized by high yield variability and persistent food security challenges. This discrepancy arises in part because the use of natural environmental predictors primarily reflects short-term variability, while overlooking baseline yield disparities that define low-yield regions. Furthermore, conventional training practices\u0026mdash;focused on optimizing overall accuracy\u0026mdash;systematically neglect rare but critical low-yield outcomes. Recognizing these limitations, we propose an enhanced deep learning framework that combines local socio-economic features with the Fit-Gen Adaptive Attention (FGAA) algorithm, specifically designed to optimize for extreme yield events. Our approach significantly improves predictive robustness in low-yield regions, providing a practical pathway to better assess and manage food security risks in vulnerable areas.\u003c/p\u003e","manuscriptTitle":"Deep learning struggles to forecast yields in food-insecure, low-production regions","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-03 17:53:35","doi":"10.21203/rs.3.rs-6800608/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c830cfd5-fad4-48a9-9bed-002102008830","owner":[],"postedDate":"June 3rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-06-03T17:53:36+00:00","versionOfRecord":[],"versionCreatedAt":"2025-06-03 17:53:35","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6800608","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6800608","identity":"rs-6800608","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}