Evaluating the Functional Realism of Deep Learning Rainfall-Runoff Models Using Catchment Hydrology Principles

Deep learning (DL) models are expected to exploit correlations in data rather than causal processes. However, the nature and variability of spurious learning in DL rainfall–runoff models remain poorly understood. To explore these gaps, we propose a hydrologic-specific Explainable AI (XAI) framework to extract nonlinear and time-varying Impulse Response Functions (IRFs) used by Long Short-Term Memory (LSTM) models to simulate streamflow. IRFs reveal how LSTMs emulate streamflow generation and celerity propagation processes in response to impulses of precipitation (P), temperature (T), and potential-evapotranspiration (PET), enabling the evaluation of LSTMs’ functional realism under short-term and long-term varying climate/weather conditions. Applying this framework to 672 catchments in USA and Canada, we found that while LSTMs achieve exceptionally high predictive accuracy, their extracted functionality often contradicts established hydrologic principles. Unexpectedly, the isolated effects of T or PET short-term variations on LSTM’s simulated streamflow and celerity rate are often positive in direction. For example, in over 70% of rain-dominated catchments, particularly along the Pacific Coast, increased T within 1–14 days prior to streamflow events is associated with higher streamflow and celerity rates. Similarly, in the southeastern USA and California, LSTMs often predict increased streamflow solely in response to PET rises. In snow-dominated catchments—particularly in the Rockies—LSTMs exploit the temporal alignment of PET with snowmelt-driven streamflow increases, assigning PET a stronger positive influence than T, even though temperature is the primary driver of snowmelt. These behaviors—likely driven by seasonality, data non-homogeneity, simplicity bias, and missing causal factors—undermine LSTMs’ scientific reliability for streamflow forecasting and climate impact assessments. Our XAI framework integrates DL with hydrologic context through differentiable modeling, offering a screening tool to evaluate functional realism in forecasting, climate change studies, and applications to ungauged basins.