Synthetic Meets Authentic: Leveraging Text-to-Image Generated Datasets for Apple Detection in Orchard Environments

Synthetic Meets Authentic: Leveraging Text-to-Image Generated Datasets for Apple Detection in Orchard Environments | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Synthetic Meets Authentic: Leveraging Text-to-Image Generated Datasets for Apple Detection in Orchard Environments Ranjan Sapkota, Dawood Ahmed, Manoj Karkee This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4147237/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 11 Nov, 2024 Read the published version in Smart Agricultural Technology → Version 1 posted You are reading this latest preprint version Abstract Training machine learning (ML) models for computer vision-based object detection process typically requires large, labeled datasets, a process often burdened by significant human effort and high costs associated with imaging systems and image acquisition. This research aimed to simplify image data collection for object detection in orchards by avoiding traditional fieldwork with different imaging sensors. Utilizing OpenAI's DALLE, a large language model (LLM) for realistic image generation, we generated and annotated a cost effective dataset. This dataset, exclusively generated with text-to-image prompts/inputs, was then utilized to train a deep learning model, YOLOv8, for apple detection, which was then tested with real-world (outdoor orchard) images captured by a digital (Nikon D5100) camera as well as a machine vision camera (IntelRealsense D435i). The model achieved a training precision of 0.83, recall of 0.99, an F1 score of 0.92, and mAP@50 at 0.96. Validation tests against actual images collected over two different varieties of apples (Honeycrisp and Envy) in a commercial orchard environment showed a precision of 0.82 and 0.75, recall of 0.88 and 0.63, and mAP@50 of 0.92 and 0.70, each respectively. The inference time of the model was 0.015 seconds for the digital camera-based images and 0.012 seconds for the machine vision camera based images. This study presents a pathway for generating large image datasets in challenging agricultural fields with minimal or no labor-intensive efforts in field data-collection, which could accelerate the development and deployment of computer vision and robotic technologies in orchard environments. Agricultural Engineering dalle text to image generative ai image generation ChatGPT OpenAI precision agriculture Large Language Models (LLMs) Generative Adversarial Network (GAN) Generative Artificial Intelligence DALL.E DALLE Image Generation Text-to-Image Generation AI Image Generation GPT4 Machine Vision Computer Vision Deep Learning Machine Learning YOLO YOLOv8 Object Detection Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Published Journal Publication published 11 Nov, 2024 Read the published version in Smart Agricultural Technology → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4147237","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":282525394,"identity":"30bf034d-b0d9-49ff-b291-7c14882ef85f","order_by":0,"name":"Ranjan Sapkota","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+klEQVRIiWNgGAWjYJACAwjFBiJsIGweorUcYEgjTgsDkpbDhLWYt589UPBzB4M9/+y2xMcfas4nzp+RwPjgbRtuLTJn8hIMe88wJM64c+ywwYFjtxM33EhgNpyLR4sEQ46BAW8bQwLDjfQ2iQNst3M3SCSwSfPi08L/xsDwbxuDvTxYy79zuUCHsf/Gq0Uix8AYqIBxw420YxIH2w7kNtxIYGPGr+WNgbFsm0TixjvHkg3O9iXXbzjzsFlyzjl8DssxM3zbZmMvd7vN8EHFNztj+fbkgx/elOHWAgRsBqBQACEoYGzAqx4ImB9A7COkbhSMglEwCkYsAADU51OD50PTfgAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-5417-6744","institution":"Washington State University","correspondingAuthor":true,"prefix":"","firstName":"Ranjan","middleName":"","lastName":"Sapkota","suffix":""},{"id":282525395,"identity":"12bfb1f7-5e6a-499e-a785-f31188b94d0a","order_by":1,"name":"Dawood Ahmed","email":"","orcid":"https://orcid.org/0000-0003-4201-0621","institution":"Washington State University","correspondingAuthor":false,"prefix":"","firstName":"Dawood","middleName":"","lastName":"Ahmed","suffix":""},{"id":282525396,"identity":"507e411b-146a-443f-9862-5f0254ac4485","order_by":2,"name":"Manoj Karkee","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4klEQVRIiWNgGAWjYLACCYYEBgYeIOMDhG9AvBbGGURrYYBqYeYhRotu+9mDDywY0uTNew4ffGzbVpfYwN68TQKfFrMzeckGEgw5hnPOtiUb57YdTmzgOVaGX8uBHDMJCYYKxhn8PGbSudsOJDZIAEXwajn/BqzFfgY///ffltuADpN/Q0DLDbAtOYkzeHvYmBm3MQNt4SGk5Y2xgYRBWvIMnmPGkr3/Dhu38aQVW+B3WI7hY4mKZNsZPMkPP/w4Uyfbz3544w18WkCAWQI5ItgIKQcBxg/EqBoFo2AUjIKRCwCgKEPYrn+IAgAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0001-5337-4848","institution":"Washington State University","correspondingAuthor":true,"prefix":"","firstName":"Manoj","middleName":"","lastName":"Karkee","suffix":""}],"badges":[],"createdAt":"2024-03-22 05:35:56","currentVersionCode":1,"declarations":{"humanSubjects":true,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":true,"humanSubjectConsent":true,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-4147237/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4147237/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1016/j.atech.2024.100614","type":"published","date":"2024-11-12T00:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":78546960,"identity":"51b6dd3e-9d24-4eb2-b596-e093ea937e7e","added_by":"auto","created_at":"2025-03-14 17:58:46","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1648365,"visible":true,"origin":"","legend":"","description":"","filename":"SyntheticMeetsAuthenticManuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4147237/v1_covered_950c4542-36a3-4e7e-9ee2-e46635173bbb.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eSynthetic Meets Authentic: Leveraging Text-to-Image Generated Datasets for Apple Detection in Orchard Environments\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"Washington State University","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"dalle, text to image, generative ai, image generation, ChatGPT, OpenAI, precision agriculture, Large Language Models (LLMs), Generative Adversarial Network (GAN), Generative Artificial Intelligence, DALL.E, DALLE Image Generation, Text-to-Image Generation, AI Image Generation, GPT4, Machine Vision, Computer Vision, Deep Learning, Machine Learning, YOLO, YOLOv8, Object Detection","lastPublishedDoi":"10.21203/rs.3.rs-4147237/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4147237/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTraining machine learning (ML) models for computer\u0026nbsp;vision-based object detection process typically requires\u0026nbsp;large, labeled datasets, a process often burdened by significant human effort and high costs associated with imaging systems and image acquisition. This research aimed to simplify image data collection for object detection in orchards by avoiding traditional fieldwork \u0026nbsp;with different imaging sensors. Utilizing OpenAI's DALLE, a large language model (LLM) for realistic\u0026nbsp;image generation, we generated and annotated a cost effective dataset. This dataset, exclusively generated with\u0026nbsp;text-to-image prompts/inputs, was then utilized to train a\u0026nbsp;deep learning model, YOLOv8, for apple detection, which was then tested with real-world (outdoor orchard) images captured by a digital (Nikon D5100) camera as well as a\u0026nbsp;machine vision camera (IntelRealsense D435i). The \u0026nbsp;model achieved a training precision of 0.83, recall of 0.99, an F1 score of 0.92, and mAP@50 at 0.96. Validation tests against actual images collected over two\u0026nbsp;different varieties of apples (Honeycrisp and Envy) in a commercial orchard environment showed a precision of 0.82 and 0.75, recall of 0.88 and 0.63, and mAP@50 of 0.92 and 0.70, each respectively. The inference time of the\u0026nbsp;model was 0.015 seconds for the digital camera-based\u0026nbsp;images and 0.012 seconds for the machine vision camera based images. This study presents a pathway for generating large image datasets in challenging agricultural fields with minimal or no labor-intensive\u0026nbsp;efforts in field data-collection, which could accelerate the development and deployment of computer vision and robotic technologies in orchard environments.\u003c/p\u003e","manuscriptTitle":"Synthetic Meets Authentic: Leveraging Text-to-Image Generated Datasets for Apple Detection in Orchard Environments","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-25 16:16:56","doi":"10.21203/rs.3.rs-4147237/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ee4b3f43-e85f-4ff6-b00d-e8b043cea805","owner":[],"postedDate":"March 25th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":29747387,"name":"Agricultural Engineering"}],"tags":[],"updatedAt":"2025-03-14T17:58:41+00:00","versionOfRecord":{"articleIdentity":"rs-4147237","link":"https://doi.org/10.1016/j.atech.2024.100614","journal":{"identity":"smart-agricultural-technology","isVorOnly":true,"title":"Smart Agricultural Technology"},"publishedOn":"2024-11-12 00:00:00","publishedOnDateReadable":"November 12th, 2024"},"versionCreatedAt":"2024-03-25 16:16:56","video":"","vorDoi":"10.1016/j.atech.2024.100614","vorDoiUrl":"https://doi.org/10.1016/j.atech.2024.100614","workflowStages":[]},"version":"v1","identity":"rs-4147237","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4147237","identity":"rs-4147237","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}