Parameter Sensitivity of Network-Based Statistical Inference

Abstract The network-based statistic (NBS) is a popular method for performing edge-wise statistical inference on brain networks, with a known limitation in the form of a need for the user to pre-define an arbitrary cluster-forming threshold. Recently a new method, the “Threshold Free Network Based Statistic” (TFNBS), was proposed to attempt to overcome this necessity. While TFNBS does not require the a priori definition of a hard cluster-forming threshold to generate edge-wise significance values, it does require definition of the statistical enhancement parameters intrinsic to the method. In this work, we explore the practical consequences of parameter choice on reported results using both methods, and assess whether TFNBS indeed provides the research community with a significant increase in the fidelity of results. We do so by applying both NBS and TFNBS to a previously well-characterized cohort with temporal lobe epilepsy in a case-control study of diffusion MRI-derived connectivity, and observing the variation of statistical inference outcomes depending on the values of enhancement parameters utilised. Our results exhibit substantial variability for both TFNBS and NBS, indicating that the choice of parameters for both methods influences the extent of the inferred network changes; this therefore imposes a restriction on the precision with which the outcomes of statistical inference using either method may be interpreted.