Comparing fMRI inter-subject correlations between groups using permutation tests

Inter-subject correlation (ISC) based analysis is a conceptually simple approach to analyze functional magnetic resonance imaging (fMRI) data acquired under naturalistic stimuli such as a movie. We describe and validate the statistical approaches for comparing ISCs between two groups of subjects implemented in the ISC toolbox, which is an open source software package for ISC-based analysis of fMRI data. The approaches are based on permutation tests. We validated the approaches using five different data sets from the ICBM functional reference battery tasks. First, we created five null datasets (one for each task) by dividing the subjects into two matched groups and assumed that no group difference exists. Second, based on one null dataset, we created datasets with simulated ISC differences of varying size between the two groups. Based on the experiments with these two types of data, we recommend the use of subject-wise permutations, instead of element-wise permutations. The tests based on subject-wise permutations led to correct false positive rates. We observed that the null-distributions should be voxel-specific and not based on pooling all voxels across the brain as is typical in fMRI. This was the case even if studentized permutation tests were used. Additionally, we experimented with an fMRI dataset acquired using a dance movie stimulus for comparison of a group of adult males on the autism spectrum to a matched typically developed group. The experiment confirmed the differences between voxel-based permutation tests and global model based permutation tests.