Human limits in Machine Learning: Prediction of plant phenotypes using soil microbiome data

Human limits in Machine Learning: Prediction of plant phenotypes using soil microbiome data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Human limits in Machine Learning: Prediction of plant phenotypes using soil microbiome data Rosa Aghdam, Xudong Tang, Shan Shan, Richard Lankau, Claudia Solis-Lemus This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3957562/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The preservation of soil health is a critical challenge in the 21st century due to its significant impact on agriculture, human health, and biodiversity. We provide the first deep investigation of the predictive potential of machine learning models to understand the connections between soil and biological phenotypes. We investigate an integrative framework performing accurate machine learning-based prediction of plant phenotypes from biological, chemical, and physical properties of the soil via two models: random forest and Bayesian neural network. We show that prediction is improved when incorporating environmental features like soil physicochemical properties and microbial population density into the models, in addition to the microbiome information. Exploring various data preprocessing strategies confirms the significant impact of human decisions on predictive performance. We show that the naive total sum scaling normalization that is commonly used in microbiome research is not the optimal strategy to maximize predictive power. Also, we find that accurately defined labels are more important than normalization, taxonomic level or model characteristics. In cases where humans are unable to classify samples accurately, machine learning model performance is limited. Lastly, we provide domain scientists via a full model selection decision tree to identify the human choices that optimize model prediction power. Biological sciences/Computational biology and bioinformatics Biological sciences/Computational biology and bioinformatics/Computational models Biological sciences/Computational biology and bioinformatics/Data processing Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Microbiology/Applied microbiology Biological sciences/Microbiology/Communities Full Text Additional Declarations No competing interests reported. Supplementary Files supplementaryfile.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3957562","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":275612562,"identity":"1ed9da71-d11d-471d-b481-de01cfc229d4","order_by":0,"name":"Rosa Aghdam","email":"","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Rosa","middleName":"","lastName":"Aghdam","suffix":""},{"id":275612563,"identity":"aeb56606-e8fc-4ff8-bce8-b00c58c50839","order_by":1,"name":"Xudong Tang","email":"","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Xudong","middleName":"","lastName":"Tang","suffix":""},{"id":275612564,"identity":"63810405-bf24-46e3-ad76-832ebb659bf7","order_by":2,"name":"Shan Shan","email":"","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Shan","middleName":"","lastName":"Shan","suffix":""},{"id":275612565,"identity":"33dfacf9-84f2-48de-9d21-8951fc8ae68f","order_by":3,"name":"Richard Lankau","email":"","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Richard","middleName":"","lastName":"Lankau","suffix":""},{"id":275612566,"identity":"d119eaf3-4f93-43f4-af19-058e4c33aaa1","order_by":4,"name":"Claudia Solis-Lemus","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2UlEQVRIiWNgGAWjYFAC5oYDDAVgxgGGBLBIAiEtjEAtBiAGWwLxWhggWngMoCIEtMi3NzYe/GFgE21w/sw3iQd/7Bj42XMM8GoxOHOw4TCPQVruhhu52yQS25IZJHveENAikdhwmMHgMFAL77YbiSB/3SBgi/yMxAagw/7nbjh/5tmNhD8HGOwJaWEAmcxjcCB3w4EcthsJbEBbJIjzS3LuzBtp5j+AfuGROPOsAL/D2psPf/xRYZfbd/7wY8Mff+zk+NuTN+B3GDrgIU35KBgFo2AUjAKsAAC8BlC3A3HjlgAAAABJRU5ErkJggg==","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Claudia","middleName":"","lastName":"Solis-Lemus","suffix":""}],"badges":[],"createdAt":"2024-02-15 04:16:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3957562/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3957562/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":55264130,"identity":"c47ae9f4-b5b5-4679-88ad-32865e72bb1c","added_by":"auto","created_at":"2024-04-25 01:37:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":629010,"visible":true,"origin":"","legend":"","description":"","filename":"paper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3957562/v1_covered_244ff34e-212c-4676-bb74-6b8de0d3d486.pdf"},{"id":51843612,"identity":"6ffa8407-22ff-4d36-b3ca-1e603f2c59ed","added_by":"auto","created_at":"2024-03-01 06:29:53","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1029439,"visible":true,"origin":"","legend":"","description":"","filename":"supplementaryfile.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3957562/v1/99e59b1bf4cafa29697cbaa2.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Human limits in Machine Learning: Prediction of plant phenotypes using soil microbiome data","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3957562/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3957562/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The preservation of soil health is a critical challenge in the 21st century due to its significant impact on agriculture, human health, and biodiversity. We provide the first deep investigation of the predictive potential of machine learning models to understand the connections between soil and biological phenotypes. We investigate an integrative framework performing accurate machine learning-based prediction of plant phenotypes from biological, chemical, and physical properties of the soil via two models: random forest and Bayesian neural network. We show that prediction is improved when incorporating environmental features like soil physicochemical properties and microbial population density into the models, in addition to the microbiome information. Exploring various data preprocessing strategies confirms the significant impact of human decisions on predictive performance. We show that the naive total sum scaling normalization that is commonly used in microbiome research is not the optimal strategy to maximize predictive power. Also, we find that accurately defined labels are more important than normalization, taxonomic level or model characteristics. In cases where humans are unable to classify samples accurately, machine learning model performance is limited. Lastly, we provide domain scientists via a full model selection decision tree to identify the human choices that optimize model prediction power.","manuscriptTitle":"Human limits in Machine Learning: Prediction of plant phenotypes using soil microbiome data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-01 06:29:48","doi":"10.21203/rs.3.rs-3957562/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f27d1514-3e69-40c6-b90d-d11246fe6195","owner":[],"postedDate":"March 1st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":29054892,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":29054893,"name":"Biological sciences/Computational biology and bioinformatics/Computational models"},{"id":29054894,"name":"Biological sciences/Computational biology and bioinformatics/Data processing"},{"id":29054895,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":29054896,"name":"Biological sciences/Microbiology/Applied microbiology"},{"id":29054897,"name":"Biological sciences/Microbiology/Communities"}],"tags":[],"updatedAt":"2024-04-22T14:03:14+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-01 06:29:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3957562","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3957562","identity":"rs-3957562","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}