Identifying Trustworthiness Challenges in Deep Learning Models for Continental-Scale Water Quality Prediction

Identifying Trustworthiness Challenges in Deep Learning Models for Continental-Scale Water Quality Prediction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Identifying Trustworthiness Challenges in Deep Learning Models for Continental-Scale Water Quality Prediction Tongliang Liu, Xiaobo Xia, Xiaofeng Liu, Jiale Liu, Kuai Fang, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6248689/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Water quality is foundational to environmental sustainability, ecosystem resilience, and public health. Deep learning models, particularly Long Short-Term Memory (LSTM) networks, offer transformative potential for large-scale water quality prediction and scientific insights generation. However, their widespread adoption in high-stakes decision-making, such as pollution mitigation and equitable resource allocation, is prevented by unresolved trustworthiness challenges including fairness, uncertainty, interpretability, robustness, generalizability, and reproducibility. In this work, we present the first comprehensive evaluation of trustworthiness in a continental-scale multi-task LSTM model predicting 20 water quality variables (encompassing physical/chemical processes, geochemical weathering, and nutrient cycling) across 482 U.S. basins. Our investigation uncovers systematic patterns of model performance disparities linked to basin characteristics, the inherent complexity of biogeochemical processes, and variable predictability, emphasizing critical performance fairness concerns. We further propose methodological frameworks for quantitatively evaluating critical aspects of trustworthiness, including uncertainty, interpretability, and robustness, identifying key limitations that could challenge reliable real-world deployment. This work serves as a timely call to action for advancing trustworthy data-driven methods for water resources management and provides a pathway to offering critical insights for researchers, decision-makers, and practitioners seeking to leverage artificial intelligence (AI) responsibly in environmental management. Earth and environmental sciences/Biogeochemistry Earth and environmental sciences/Environmental sciences Full Text Additional Declarations There is NO Competing Interest. Supplementary Files SI.pdf Support Information for the submission Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6248689","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":433256047,"identity":"139e90aa-938f-4a6e-90bb-4b948f734b64","order_by":0,"name":"Tongliang Liu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABKklEQVRIie2QMUvDQBTH33HSLCldbyn9CieBUChY/SZ3BNIlaMGlYIknhXTROheK/QpxcY4cxCU4B3Soizh0yOQk4l1SnBLp6HC/4X/3Hu/HgwdgMPxXNlQnFglM1JuUPdw4betgpYKUku2tQKUAivZQhtaN/GDjo9PeMbqS7buQL15mKYHJgAsro7Vb7Ge/z6h3fpggIdsPki9fU59ANuLCDuoVEriUUczvRakkPM4Dl6BIcgF/Kpc7ZRUq5eyToG+ldLZNirNhVPI1aEVgvaVF1J8L0rAly1x15CceYyQeV6l0lrnv9Fk6ciLyPq5TrPm1UxRfF3w9n8liOw27i9x7y4vpoHvb8eL6M0OL6KSyqk4EwAFhut8wr8CFzp6oquFvx2AwGAw7fgCqh2oWHKKRTwAAAABJRU5ErkJggg==","orcid":"","institution":"University of Sydney","correspondingAuthor":true,"prefix":"","firstName":"Tongliang","middleName":"","lastName":"Liu","suffix":""},{"id":433256048,"identity":"b124daf5-9712-4eff-9dac-baaa63d6cb7f","order_by":1,"name":"Xiaobo Xia","email":"","orcid":"https://orcid.org/0000-0003-3615-0919","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Xiaobo","middleName":"","lastName":"Xia","suffix":""},{"id":433256049,"identity":"e4db0599-4e6d-4dde-ba26-6663e088c593","order_by":2,"name":"Xiaofeng Liu","email":"","orcid":"https://orcid.org/0000-0002-9721-6152","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Xiaofeng","middleName":"","lastName":"Liu","suffix":""},{"id":433256050,"identity":"78157773-1d77-4e55-b14d-039c368fb6f3","order_by":3,"name":"Jiale Liu","email":"","orcid":"","institution":"Penn State University","correspondingAuthor":false,"prefix":"","firstName":"Jiale","middleName":"","lastName":"Liu","suffix":""},{"id":433256051,"identity":"fb6bea8e-cef6-47d4-846d-429b8a04e8cc","order_by":4,"name":"Kuai Fang","email":"","orcid":"","institution":"Stanford University","correspondingAuthor":false,"prefix":"","firstName":"Kuai","middleName":"","lastName":"Fang","suffix":""},{"id":433256052,"identity":"c682ceeb-0612-4074-9681-959304570172","order_by":5,"name":"Lu Lu","email":"","orcid":"https://orcid.org/0000-0002-5476-5768","institution":"Yale University","correspondingAuthor":false,"prefix":"","firstName":"Lu","middleName":"","lastName":"Lu","suffix":""},{"id":433256053,"identity":"9a907e41-b66f-4cb3-8e1a-f6971ac73d6d","order_by":6,"name":"Samet Oymak","email":"","orcid":"","institution":"University of Michigan","correspondingAuthor":false,"prefix":"","firstName":"Samet","middleName":"","lastName":"Oymak","suffix":""},{"id":433256054,"identity":"e0ed8b9e-b6e1-42b0-9701-4dfd80754a18","order_by":7,"name":"William Currie","email":"","orcid":"","institution":"University of Michigan","correspondingAuthor":false,"prefix":"","firstName":"William","middleName":"","lastName":"Currie","suffix":""}],"badges":[],"createdAt":"2025-03-18 02:40:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6248689/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6248689/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":82578811,"identity":"5c7c8822-2797-4670-a673-df64cae45444","added_by":"auto","created_at":"2025-05-13 06:11:51","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6060667,"visible":true,"origin":"","legend":"Article File","description":"","filename":"MAINTEXT.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6248689/v1_covered_60690db7-364b-4cc0-a141-22fdc1d7ce63.pdf"},{"id":79143909,"identity":"31873b10-6262-4c3d-861d-b8960a4243e3","added_by":"auto","created_at":"2025-03-25 02:36:17","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":4921320,"visible":true,"origin":"","legend":"Support Information for the submission","description":"","filename":"SI.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6248689/v1/6d7ffd5a7060bc2345a1254d.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Identifying Trustworthiness Challenges in Deep Learning Models for Continental-Scale Water Quality Prediction","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6248689/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6248689/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Water quality is foundational to environmental sustainability, ecosystem resilience, and public health. Deep learning models, particularly Long Short-Term Memory (LSTM) networks, offer transformative potential for large-scale water quality prediction and scientific insights generation. However, their widespread adoption in high-stakes decision-making, such as pollution mitigation and equitable resource allocation, is prevented by unresolved trustworthiness challenges including fairness, uncertainty, interpretability, robustness, generalizability, and reproducibility. In this work, we present the first comprehensive evaluation of trustworthiness in a continental-scale multi-task LSTM model predicting 20 water quality variables (encompassing physical/chemical processes, geochemical weathering, and nutrient cycling) across 482 U.S. basins. Our investigation uncovers systematic patterns of model performance disparities linked to basin characteristics, the inherent complexity of biogeochemical processes, and variable predictability, emphasizing critical performance fairness concerns. We further propose methodological frameworks for quantitatively evaluating critical aspects of trustworthiness, including uncertainty, interpretability, and robustness, identifying key limitations that could challenge reliable real-world deployment. This work serves as a timely call to action for advancing trustworthy data-driven methods for water resources management and provides a pathway to offering critical insights for researchers, decision-makers, and practitioners seeking to leverage artificial intelligence (AI) responsibly in environmental management.","manuscriptTitle":"Identifying Trustworthiness Challenges in Deep Learning Models for Continental-Scale Water Quality Prediction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-25 02:36:12","doi":"10.21203/rs.3.rs-6248689/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4e6619b0-2ee3-4ff7-894d-ada6951a44d1","owner":[],"postedDate":"March 25th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":46133257,"name":"Earth and environmental sciences/Biogeochemistry"},{"id":46133258,"name":"Earth and environmental sciences/Environmental sciences"}],"tags":[],"updatedAt":"2025-05-13T05:55:37+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-25 02:36:12","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6248689","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6248689","identity":"rs-6248689","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}