How Software Implementation Masks and Constrains Critical Analytic Decisions: Tracing Encoded Choices in Geospatial Research

Researchers increasingly call for transparent reporting of analytic decisions as foundations for reproducible research and collective knowledge production. However, these calls for openness often overlook a critical dimension: software environments encode implicit computational decisions that are not always visible to researchers, even when the same statistical methods are applied. Analyses may be technically reproducible while the epistemological function of reproduction to evaluate and validate prior conclusions is undermined by the opacity of software design. Moreover, unexamined software behavior can compromise the conceptual rigor of research by shaping analytic choices unbeknownst to researchers. While attempting a reproduction of Chakraborty (2021) we investigated how software-encoded decisions influenced intermediate analytic outputs, propagated into downstream inference, and ultimately shaped the conceptualization of research design. Through systematic comparison across software, we demonstrate that 1) software-encoded decisions structure both intermediate results and final analytic conclusions; 2) researchers must remain attentive to the decisions and materials required to execute an analysis, understand what software is doing on their behalf, and consider the numerical and conceptual ramifications of those decisions; and 3) cross-software sensitivity analysis is particularly important when analytic workflows span multiple platforms or when outputs from one system are used as inputs to another.