Model mimicry limits conclusions about neural tuning and can mistakenly imply unlikely priors

6 In a recent issue of Nature Communications, Harrison, Bays, and Rideaux1 use electroencephalography 7 (EEG) to infer population tuning properties from human visual cortex, and deliver a major update to 8 existing knowledge about the most elemental building block of visual perception – orientation tuning. 9 Using EEG together with simulations in an approach they refer to as “generative forward modeling”, the 10 authors adjudicate between two competing population tuning schemes for orientation tuning in visual 11 cortex. They claim that a redistribution of orientation tuning curves can explain their observed pattern 12 of EEG results, and that this tuning scheme embeds a prior of natural image statistics that exhibits a 13 previously undiscovered anisotropy between vertical and horizontal orientations. If correct, this 14 approach could become widely used to find unique neural coding solutions to population response data 15 (e.g., from EEG) and to yield a “true” population tuning scheme deemed generalizable to other 16 instances. However, here we identify major flaws that invalidate the promise of this approach, which we 17 argue should not be used at all. First, we will examine the premise of Harrison and colleagues1, to 18 subsequently explain why “generative forward modeling” cannot circumvent model mimicry pitfalls and 19 can deliver many possible solutions of unknowable correctness. Finally, we show a tentative alternative 20 explanation for the data. 21 22 23 24 Abbreviated title: Commentary of Harrison et al. (2023) 25 Corresponding authors: [email protected] & [email protected] 26 Number of figures: 3 main & 1 supplemental 27 Word count: 1290 28 Conflict of interest: The authors declare no conflict of interest 29

We want to thank the multiple groups of scientists whose work we re-analyzed here. 30 We were able to test our alternative hypotheses thanks to their efforts of ensuring readily availab le and 31 well-documented data. We’d also like to thank Tommy Sprague and John Serences for reading an early 32 version of this manuscript, and for their valuable feedback.33 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 2 Main Text 34 Invasive neural recording techniques are the gold standard and only direct measurement tool for 35 quantifying neural orientation tuning properties in visual cortex . Harrison and colleagues 1 point to 36 previous research as precedence for the overrepresentation of horizontal compared to vertical selective 37 neurons in visual areas of mouse2, cat3, and macaque4, and summarize these three studies in Figure 1b of 38 their papera. However, Harrison and colleagues seem to misinterpret data from these studies that do not, 39 in fact, set out to test for or convincingly show anisotropies between vertical and horizontal orientations. 40 Roth and collegues2 show a weak trend of more horizontal versus vertical selectivity in mouse V1, but an 41 opposite trend in a later visual area (Posteromedial area). Wang and collegues3 show a similar weak trend 42 favoring horizontal selectivity in cat visual cortex, but other cat studies that show no or opposite trends 43 can be easily found (e.g., 5,6). Fang and colleageus4 analyzed data from a total of 48 V1 hemispheres and 44 38 V4 hemispheres (in over 34 macaques), showing no trends favoring horizontal orientations in V4b, and 45 an opposite trend in V1c. Thus, e ven this small selection of research from a field that spans decades 46 (starting with Hubel & Wiesel in 19627) shows no consistent differences between vertical and horizontal 47 neural selectivity, and it is unclear to us how Harrison and colleagues can argue otherwise. If systematic 48 anisotropies between vertical and horizontal selectivity exist, w e are not aware of research that has 49 systematically evaluated the existing literature or actually applied quantitative tests. Furthermore, the 50 implied evolutionary justification for an anisotropy favoring horizontal over vertical orientations, 51 seemingly mirrored in the statistics of natural but not man -made scenes d, is difficult to reconcile with 52 decades of physiological research emphasizing developmental influences shaping orientation tuning8–10. 53 Aside from its shaky premise, the central flaw in Harrison and collegues 1 lies with the fact that EEG 54 decoding results cannot inform about the underlying neural or population tuning, due to an inherent 55 inverse problem and model mimicry. The inverse problem is where an underlying cause cannot be inferred 56 from a (measurable) effect, such as the inability to estimate neural causes from non-invasive imaging 57 results11,12. Relatedly, model mimicry refers to cases where many possible models can generate the same, 58 or very similar, outcomes and model fits. To test what population tuning properties explain their pattern 59 of EEG results, Harrison and colleagues1 claim they can use “generative forward modeling” (see also13) to 60 differentiate between two possible population tuning schemes: differences in tuning widths and 61 differences in tuning preferences. This claim is false. Using the same simulation approach as Harrison and 62 colleagues (but a slightly different decodere), we show some examples of population tuning schemes that 63 all yield the same pattern of results at the macro-level (Figure 1, Supplemental Methods). Importantly, 64 a How these data were derived and plotted is not described, and statistical tests of vertical-horizontal anisotropies are not reported. b Incidentally, the first data figure in Fang and collegues 4 shows a single example V4 hemisphere, and here a trend for more horizontal-preferring neurons can be observed. This trend is absent in the full V4 data with 38 hemispheres. c This higher selectivity for vertical is statistically significant, but likely due to concurrent radial biases. d The justification for an embedded prior based on natural scene statistics (i.e., the green line in Harrison and colleagues Figure 7b) comes from measurements by Girshick and colleagues8 over 6 levels of image resolution. It is unclear which resolution the green line is based on or why. e None of the various decoding metrics (accuracy, precision, bias) in Harrison and collegues 1 are specific to the commonly used inverted encoding model (IEM) in their paper. To illustrate this, we use a Mahalanobis distance decoder15 that yields qualitatively similar results as the IEM decoder (Supplemental Figure 1). We made other minor analysis changes to improve consistency and robustness (see Supplemental Methods), such as using channels and time-points as features for decoding, wider orientation bins, using repeated stratified random folds to split both real and simulated data, etc. (see Supplemental Methods). .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 3 this includes the population tuning scheme with different tuning widths that Harrison and colleagues 65 argued could not fit the ir data. This is because they did not consider a wider parameter space for this 66 model. Even when adjudicating between just two models, there are many possibly “sensible”, but fairly 67 arbitrary parameters to pick from (e.g., the number of tuning functions, the range of tuning function 68 widths, etc.) that can all generate outcomes that mimic each other . Excluding parts of this parameter 69 space (such as models with wider tuning at cardinals compared to obliques ) on the basis of previous 70 physiology findings is not an option, as any subsequent claims about orientation tuning would amount to 71 reverse inference. In addition to models with differences in tuning width or preference, a model with 72 differences in gain modulation 14 could also explain the data (Figure 1c), but was not considered by 73 Harrison and colleagues1. In fact, even a uniform set of tuning curves can approximate the data, as long 74 as the signal-to-noise ratio (SNR) is modulated across orientation space (Figure 1d). To make matters even 75 more complex, model specification is not limited to “sensible” choices only – the “tuning functions” used 76 by Harrison and colleques 1 are well-motivated models 15, but a set of arbitrarily shaped functions could 77 also be used to simulate data and/or recover decoding metrics16. 78 79 Figure 1. Model mimicry: many models produce the same pattern of results. In “generative forward modeling”, EEG 80 data are simulated from models that use different sets of orientation tuning functions (top row). Decoding results 81 (relative decoding accuracy, relative precision, and bias) as a function of orientation are shown (3 bottom rows) for 82 simulations using different underlying example models. A. Preferred tuning model: Tuning functions are unevenly 83 spaced along the orientation space, with more clustering at vertical, and even more at horizontal orientations. This 84 is the “best fitting” model from Harrison and colleagues 1. B. Width model: Tuning curve widths are uneven, with 85 narrowest tuning for obliques, wider tuning for vertical and widest tuning for horizontal. C. Gain model: Uneven 86 tuning curve gain across orientations space, with more gain at cardinals that is highest for horizontal orientations . 87 D. SNR model: Tuning curves are uniform, but signal strength is orientation specific. 88 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 4 Across two EEG data sets, Harrison and colleagues 1 show that horizontal orientations result in notably 89 better and more consistent decoding than vertical orientations. Given that “generative forward modeling” 90 cannot be used to infer orientation tuning anisotropies in human visual cortex as a plausible explanation, 91 what other factors might be driving these EEG results? Using the same decoding method as for the 92 simulations (Figure 1), we replicate the pattern of results both for the openly available data 13 used by 93 Harrison and colleagues1, as well as for another openly available EEG data-set where orientated grating 94 stimuli were presented centrally17 (Figure 2A). However, this effect is not replicated for EEG data sets18,19 95 where orientation gratings were presented laterally (Figure 2B), where no differences between vertical 96 and horizontal orientations are evident. 97 98 Figure 2. Cardinal anisotropies for orientation decoding are specific to central stimuli in EEG. A. Re-analyses of 99 experiments reported by Harrison and colleagues1 (left) and Wolff and colleagues17 (right) where central orientations 100 were shown to participants. Line plots show relative Mahalanobis distance-based decoding metrics as a function of 101 orientation, with shaded areas indicating the cardinal orientation bins used to compute differences between 102 horizontal (green) and vertical (purple) orientations. Box plots show decoding metric differences for horizontal minus 103 vertical orientations. Top: Relative accuracy (mean-centered cosine vector mean of pattern similarity curve), Middle: 104 Relative precision (1 minus the circular standard deviation of decoded orientation across trials) , Bottom: Bias of 105 pattern similarity curves, in degrees . Error bars are 95% C.I. Both data-sets show consistent differences between 106 horizontal and vertical orientations (* p < 0.05), with better decoding for horizontal orientations, and a stronger 107 attraction bias toward vertical orientations. B. Re-analyses of experiments with laterally presented orientations18,19. 108 Same conventions as in A. No consistent differences between horizontal and vertical orientations. 109 Why do we see this difference between centrally and laterally presented stimuli? We hypothesize that 110 the cardinal anisotropy seen only for centrally presented gratings is driven by visual field anisotropies – 111 i.e., anisotropies of location instead of orientation. Human visual performance is higher for stimuli 112 presented along the horizontal compared to the vertical meridian, especially peripherally, and human V1 113 has about double the cortical surface area dedicated to the horizontal compared to the vertical 114 meridian20,21. In EEG, further anisotropies may arise due to the organization of the visual field map in 115 cortex, which determines how well activity from different portions of cortex are captured by EEG scalp 116 electrodes. For example, locations along the vertical meridian are processed closer to, and inside of, the 117 longitudinal fissure22, which is more difficult to measure with scalp electrodes. 118 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 5 That differences between the vertical and horizontal meridians of the visual field play a role in EEG 119 measurements, becomes evident when looking at location decoding from these signals. We re-analyzed 120 multiple openly available EEG datasets where participants were presented with a single dot at one of 121 many possible locations around fixation23–25. We see clear and systematic differences in location decoding 122 accuracies, with highest relative decoding for locations close to the horizontal meridian, and lowest for 123 locations close to the vertical meridian (Figure 3a). The eccentricity at which dot stimuli were presented 124 in four of these datasets (outer ring in Figure 3a, bottom) overlaps with the edge of the grating stimulus 125 used by Harrison and colleagues (overlaid grey dotted line in Figure 3a, bottom)f. Concretely, for centrally 126 presented stimuli, this difference in sensitivity across the visual field means lower SNR at the upper and 127 lower stimulus edges than at the left and right stimulus edges (Figure 3b, left). 128 Importantly, these SNR differences can interact with second order stimulus properties (stimulus edge 129 effects or “vignetting”) that have been argued to at least in part be related to the decoded signal obtained 130 from non-invasive neuroimaging 26,27. Vignetting refers to the interaction between stimulus orientation 131 and stimulus aperture, such that for circular gratings the “orientation energy” is strongest on the edges 132 of the grating aligned with the orientation (Figure 3b, right). A vertically orientated grating presented 133 centrally will therefore evoke more activity in the periphery of the vertical meridian, a visual field location 134 where sensitivity is lower. A centrally presented horizontal grating will evoke more activity in the 135 periphery of the horizontal meridian, where sensitivity is higher . This may not be true for laterally 136 presented stimuli, where the decoded orientation energy falls into a part of the visual field where 137 measurement sensitivity is more evenly distributed. 138 We do not claim that th is explanation is definitive or exhaustive. For example, spatial attention to the 139 endpoints of orientated gratings 28 could interact with visual field anisotropies in a manner similar to 140 vignetting effects. Other factors may also interact with measurement of orientation selectivity, such as 141 stimulus contrast 29 or radial bias 4,30. We want to highlight the importance of considering stimulus and 142 measurement biases that can interact with orientation decoding. Finally, the well-known “oblique effect” 143 which describes better perceptual performance for cardinal over oblique orientations31 and is mirrored in 144 the overrepresentation of orientation-tuned neurons that prefer cardinal over obliques2–4, aligns with the 145 EEG results for both centrally and laterally presented gratings (see Figure 2). This implies that at least 146 some form of orientation anisotropy may be genuinely measurable with EEG. That said, we argue that the 147 observed attenuation of decoding metrics for vertical compared to horizontal orientations, specific for 148 centrally presented gratings, is likely driven by second-order stimulus properties that interact with 149 location-specific measurement differences. 150 f The same is true for the eccentricity of dot stimuli in the dataset from Bae 24, shown as the inner ring in Figure 3a (bottom), which is close to edge of the full-field gratings used in Wolff and collegues 19 where cardinal anisotropies are also observed (Figure 2a). .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 6 151 Figure 3. Differences in measurement sensitivity across the visual field that interact with stimulus vignetting can 152 explain decoding differences between vertical and horizontal orientations that are presented centrally . A. Top: 153 Relative location decoding accuracy as a function of presented stimulus location (re-analyses of 23–25). Green and 154 purple shadings highlight stimuli presented on the horizontal and vertical meridians, respectively. Bottom: Location 155 decoding across the visual field, plotted to scale for the various experiments: Outer, thicker ring represents possible 156 locations of dot stimuli used in Foster and colleagues 24,25, presented at 3.8° –4° eccentricity and 1.6° in diameter. 157 Inner ring represents possible locations of the dot stimuli used in Bae 23, presented at 2.3° eccentricity with 0.35° 158 diameter. Dashed grey circles represent stimulus sizes of the central orientations used in Wolff and colleagues17 and 159 Harrison colleagues1 (radii of 2.88° and 4.2°, respectively). B. Vignetting26 for gratings presented at the center of the 160 screen: “Orientation energy” is highest on the stimulus edges aligned with the orientation. This means relatively 161 higher orientation energy along the vertical meridian for vertical orientations ( top) where SNR is low, and along the 162 horizontal meridian for horizontal orientations (bottom) where SNR high. 163 In conclusion: Despite decades of research, invasive neural recordings in animals have not found the 164 anisotropies between vertical and horizontal orientations seen in the EEG data reported by Harrison and 165 colleagues1. This pattern of results cannot be explained on the basis of the underlying orientation tuning, 166 because “generative forward modeling” 1,13 suffers from an inherent inverse problem, where many 167 possible population tunings can approximate the patterns of reported EEG data equally well . Given that 168 the pattern of EEG results does not replicate for laterally presented stimuli, cardinal anisotropies are likely 169 driven by other factors, such as differences in visual field sensitivity between the vertical and horizontal 170 meridian and their interaction with second-order stimulus effects. 171 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 7

172 1. Harrison, W. J., Bays, P. M. & Rideaux, R. Neural tuning instantiates prior expectations in the human 173 visual system. Nat. Commun. 14, 5320 (2023). 174 2. Roth, M. M., Helmchen, F. & Kampa, B. M. Distinct Functional Properties of Primary and 175 Posteromedial Visual Area of Mouse Neocortex. J. Neurosci. 32, 9716–9726 (2012). 176 3. Wang, G., Ding, S. & Yunokuchi, K. Difference in the representation of cardinal and oblique contours 177 in cat visual cortex. Neurosci. Lett. 338, 77–81 (2003). 178 4. Fang, C., Cai, X. & Lu, H. D. Orientation anisotropies in macaque visual areas. Proc. Natl. Acad. Sci. 179 119, e2113407119 (2022). 180 5. Berman, N. E., Wilkes, M. E. & Payne, B. R. Organization of orientation and direction selectivity in 181 areas 17 and 18 of cat cerebral cortex. J. Neurophysiol. 58, 676–699 (1987). 182 6. Dragoi, V., Turcu, C. M. & Sur, M. Stability of Cortical Responses and the Statistics of Natural Scenes. 183 Neuron 32, 1181–1192 (2001). 184 7. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the 185 cat’s visual cortex. J. Physiol. 160, 106–154 (1962). 186 8. Blakemore, C. & Cooper, G. F. Development of the Brain depends on the Visual Environment. Nature 187 228, 477–478 (1970). 188 9. Hirsch, H. V. B. & Spinelli, D. N. Visual Experience Modifies Distribution of Horizontally and Vertically 189 Oriented Receptive Fields in Cats. Science 168, 869–871 (1970). 190 10. Sengpiel, F., Stawinski, P. & Bonhoeffer, T. Influence of experience on orientation maps in cat visual 191 cortex. Nat. Neurosci. 2, 727–732 (1999). 192 11. Liu, T., Cable, D. & Gardner, J. L. Inverted encoding models of human population response conflate 193 noise and neural tuning width. J. Neurosci. 2453–17 (2017) doi:10.1523/JNEUROSCI.2453-17.2017. 194 12. Sprague, T. C. et al. Inverted Encoding Models Assay Population-Level Stimulus Representations, Not 195 Single-Unit Neural Tuning. eNeuro ENEURO.0098-18.2018 (2018) doi:10.1523/ENEURO.0098-196 18.2018. 197 13. Rideaux, R., West, R. K., Rangelov, D. & Mattingley, J. B. Distinct early and late neural mechanisms 198 regulate feature-specific sensory adaptation in the human visual system. Proc. Natl. Acad. Sci. 120, 199 e2216192120 (2023). 200 14. Wei, X.-X. & Stocker, A. A. A Bayesian observer model constrained by efficient coding can explain 201 ‘anti-Bayesian’ percepts. Nat. Neurosci. 18, 1509–1517 (2015). 202 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 8 15. Sprague, T. C., Boynton, G. M. & Serences, J. T. The Importance of Considering Model Choices When 203 Interpreting Results in Computational Neuroimaging. eNeuro 6, (2019). 204 16. Gardner, J. L. & Liu, T. Inverted Encoding Models Reconstruct an Arbitrary Model Response, Not the 205 Stimulus. eNeuro 6, (2019). 206 17. Wolff, M. J., Ding, J., Myers, N. E. & Stokes, M. G. Revealing hidden states in visual working memory 207 using electroencephalography. Front. Syst. Neurosci. 9, (2015). 208 18. Wolff, M. J., Jochim, J., Akyürek, E. G. & Stokes, M. G. Dynamic hidden states underlying working-209 memory-guided behavior. Nat. Neurosci. 20, 864–871 (2017). 210 19. Wolff, M. J., Jochim, J., Akyürek, E. G., Buschman, T. J. & Stokes, M. G. Drifting codes within a stable 211 coding scheme for working memory. PLOS Biol. 18, e3000625 (2020). 212 20. Himmelberg, M. M., Winawer, J. & Carrasco, M. Polar angle asymmetries in visual perception and 213 neural architecture. Trends Neurosci. 46, 445–458 (2023). 214 21. Himmelberg, M. M., Winawer, J. & Carrasco, M. Linking individual differences in human primary 215 visual cortex to contrast sensitivity around the visual field. Nat. Commun. 13, 3309 (2022). 216 22. Wandell, B. A., Dumoulin, S. O. & Brewer, A. A. Visual Field Maps in Human Cortex. Neuron 56, 366–217 383 (2007). 218 23. Bae, G.-Y. Neural evidence for categorical biases in location and orientation representations in a 219 working memory task. NeuroImage 240, 118366 (2021). 220 24. Foster, J. J., Sutterer, D. W., Serences, J. T., Vogel, E. K. & Awh, E. The topography of alpha-band 221 activity tracks the content of spatial working memory. J. Neurophysiol. 115, 168–177 (2015). 222 25. Foster, J. J., Bsales, E. M., Jaffe, R. J. & Awh, E. Alpha-Band Activity Reveals Spontaneous 223 Representations of Spatial Position in Visual Working Memory. Curr. Biol. 27, 3216-3223.e6 (2017). 224 26. Carlson, T. A. Orientation Decoding in Human Visual Cortex: New Insights from an Unbiased 225 Perspective. J. Neurosci. 34, 8373–8383 (2014). 226 27. Roth, Z. N., Heeger, D. J. & Merriam, E. P. Stimulus vignetting and orientation selectivity in human 227 visual cortex. eLife 7, e37241 (2018). 228 28. Bae, G.-Y. & Luck, S. J. Dissociable Decoding of Spatial Attention and Working Memory from EEG 229 Oscillations and Sustained Potentials. J. Neurosci. 38, 409–422 (2018). 230 29. Maloney, R. T. & Clifford, C. W. G. Orientation anisotropies in human primary visual cortex depend 231 on contrast. NeuroImage 119, 129–145 (2015). 232 30. Freeman, J., Heeger, D. J. & Merriam, E. P. Coarse-Scale Biases for Spirals and Orientation in Human 233 Visual Cortex. J. Neurosci. 33, 19695–19703 (2013). 234 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 9 31. Lennie, P. Distortions of Perceived Orientation. Nature. New Biol. 233, 155–156 (1971).235 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 10 Supplemental Methods 236 Simulations (generative forward modelling) 237 We simulated different population tuning models using largely the same approach as in Harrison and 238 colleagues1 by adapting their published Matlab scripts. Briefly, data for 36 "subjects", from 32 "EEG 239 channels" and 6480 trials per “subject”, was simulated for each model (see below). Each model had a 240 given number of tuning functions, or “ model channels”. The modelled channel responses to each 241 orientation (1° to 180°, in steps of 1°) “shown” to a given model were transformed to EEG sensor space 242 via matrix multiplication between the orientation-specific model response of each trial, and a random 243 weights matrix (number of channel functions by number of "EEG channels”, sampled from a uniform 244 distribution over 0 to 1). Trial-specific noise was added to each simulated "EEG channel" sampled from a 245 normal distribution (s.d. = 6), which also ensures differences in simulated responses to trials on which 246 identical orientations were “shown”. 247 For all models we will only mention any deviations from Harrison and colleagues 1 “Preferred tuning 248 model”. The purpose of our simulations was to demonstrate that there is no unique model that best 249 describes the data, even when only considering models that could be argued to be plausible. Our models 250 (Figure 1) are by no means the “best fitting” models , as searching for best fitting solutions in this very 251 large parameter space would be computationally intractable. Indeed, we derived at our models through 252 mere trial and error and stopped once we obtained decoding results that resembled Harrison and 253 colleagues1 “Preferred tuning model”. Thus, the models and their parameters described below should not 254 be considered “definitive”; they are snapshots out of many more possibilities. 255 Preferred tuning model: For this we used the exact script published by Harrison and colleagues 1, which 256 generates the preferred tuning model. This model consisted of 16 tuning functions with constant widths 257 (κ = 2). Preference was modulated by shifting the tuning functions based on the sum of two von Mises 258 derivative functions (κ = 0.5) centered on 0° (amplitude = 14) and on 90° (amplitude = 8), which has the 259 effect that there are relatively more tuning functions around horizontal (0°) compared to vertical (90°), 260 and fewest tuning functions around obliques (45° and 135°). Note that these values in the scripts uploaded 261 by the original authors at the time of writing, differ slightly from the values described in the manuscript 262 (which states the amplitudes were 15 and 10). The resulting difference between these two parameter 263 settings is marginal however, and we decided to stick to those parameters in the script as uploaded by 264 the authors, without changing anything. 265 Width model: Instead of changing the tuning preferences across the orientation space, tuning functions 266 were evenly spaced, but their widths were modulated. This modulation was derived from the inverse of 267 the sum of two von Mises functions ( κ = 0.5), one centered on 0° (amplitude = 15) and one centered on 268 90° (amplitude = 4). Given the inversion, tuning widths were wider for cardinals than for obliques, with 269 horizontal widths being wider than vertical widths. The possible tuning widths were rescaled such that 270 they ranged from κ = 7 (the widest) to κ = 19 (the narrowest). The number of tuning functions were 271 increased to 24 (from the original 16) and every tuning function were scaled to range from 0 to 1.5. 272 Gain model: The gain model comprised 16 evenly spaced tuning functions with constant widths ( κ = 2), 273 but differences in scaled amplitude ("gain"). Gains were modulated from the sum of two von Mises 274 functions (κ = 0.5), one centered on 0° (amplitude = 15) and the other on 90° (amplitude = 8). The range 275 of gains were scaled from 0.7 (at the obliques) to 1.4 (at 0 degrees, which is horizontal). 276 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 11 SNR model: Here we used a uniform distribution of 16 identical tuning functions, all with the same width 277 (κ = 2) and all with the same gain (amplitude of 1). Unlike the models above, here we do not manipulate 278 the underlying tuning response functions, but instead modify the signal strengths of the simulated activity 279 patterns across the "EEG channels". Specifically, the signal strength was modulated for simulated 280 response patterns generated from each of the 180 orientations using the sum of two von Mises functions 281 (κ = 0.5), centered on 0 degrees (amplitude = 15) and on 90 (amplitude = 8). Signal strength modulation 282 ranged from 0.68 (68% signal strength) to 1 (100% of signal strength). The signal strengths of the 283 orientation-specific patterns were modulated after transforming activations from the tuning response 284 functions to every possible orientation (1° –180°) into sensor space (as described in Harrison and 285 collegues1 and above), meaning that the orientation-specific patterns of the simulated EEG sensors were 286 multiplied by the corresponding signal strengths (0.68 to 1), before adding the same amount of Gaussian 287 noise to each (s.d. = 6). 288 EEG Data 289 We reanalyzed openly available EEG datasets of 9 experiments across 7 publications 1,17–19,23–25, where 290 human participants viewed either circular orientation gratings, or locations. For the present manuscript, 291 the stimulus sizes and locations that participants viewed while EEG was recorded are of particular interest, 292 and are described in more detail below. Other details are available in the methods sections of the original 293 publications. 294 Harrison et al. (2023) 1: Participants ( N = 36) viewed serially presented, randomly orientated circular 295 gratings (4.2° radius) centered around fixation. Each grating was presented for 50ms, with an ISI of 150ms 296 between consecutive gratings. The task was to detect gratings with a lower spatial frequency. 297 Wolff et al. (2015)17: Participants (N = 24) performed a visual working memory task, where the orientation 298 of a grating had to be memorized for up to 2.6 seconds. Each circular grating was centrally presented 299 (2.88° radius) for 200ms, followed by a blank delay of at least 1.17 seconds. 300 Wolff et al. (2017)18: Only experiment 1 was reanalyzed. Here, participants (N = 30) performed a retro-cue 301 visual working memory task. Two randomly orientated circular gratings (radius of 3.345° each) were 302 simultaneously presented on the horizontal meridian at 6.69° eccentricity. The presentation time was 250 303 ms, followed by a blank delay of 800ms. The orientations of both gratings were behaviorally relevant 304 during encoding. 305 Wolff et al. (2020) 19: Participants ( N = 26) also performed a retro-cue visual working memory task wit h 306 laterally presented, randomly orientated circular gratings. The gratings (radius of 4.255° each) were 307 presented at 6.08° eccentricity for 200 ms followed by a blank delay of 400 ms. The orientations of both 308 gratings were behaviorally relevant during encoding. 309 Foster et al. (2015)24: Participants performed a spatial working memory task in all three experiments. The 310 visual stimulus on each trial in all three experiments was a dark gray circle (0.8° radius) presented on a 311 random location of an invisible circle at 4° eccentricity. The participants’ task was to memorize the location 312 for a delay of at least 1s (variable across experiments). In experiment 1 and 3, the circle was presented 313 for 250 ms and in experiment 2 for 1s. Sample size was N = 15 in all experiments. 314 Foster et al. (2017)25: We reanalyzed experiment 1 (N = 10). Here, in each trial a single a randomly colored 315 circle (0.8° radius) was presented on a random location of an invisible circle at 3.8° eccentricity. Stimulus 316 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 12 duration was 100ms, followed by a 1.2s blank delay. The participants’ task was to memorize and report 317 the color of the colored circle in each trial. 318 Bae (2021) 23: We reanalyzed experiment 1, where participants ( N = 22), performed a spatial working 319 memory task. The visual stimulus was a small circle (0.175° radius) presented for 200ms on one of 16 320 discrete locations on an invisible circle at 2.3° eccentricity. A blank delay (1.3s) followed after the offset 321 of the circle. The task was memorize and report the location of the circle on each trial. 322 Preprocessing 323 For all experiments, we used the voltage data the way it was published and preprocessed by the original 324 authors. 325 For the subsequent decoding analyses, we used the voltage traces from 50 to 450 ms relative to stimulus 326 onsets from the posterior electrodes, in line with Harrison and colleagues 1. Data from ref 17–19 and the 327 reanalyzed experiment 1 from ref 23 all used the same electrode coverage, and the same 17 posterior 328 channels were included in the corresponding analyses (P7, P5, P3, P1, Pz, P4, P6, P8, PO7, PO3, POz, PO4, 329 PO8, O1, Oz and O2). The same electrodes were included for the data of ref1 in addition to the electrodes 330 Iz, P9, and P10. The electrode coverage was lower for the reanalyzed experiments in ref 24,25 and the 331 included posterior electrodes for these data-sets were PO3, PO4, P3, P4, O1, O2, POz, and Pz. 332 Instead of decoding at each time-point separately within the time-window of interest and then averaging 333 (as in Harrison and collegues1), we first reformatted the data in a manner similar to previous work19 before 334 feeding it to the decoder: To take advantage of the fact that stimulus-specific information is not only 335 present in the activity patterns across electrodes, but also in the temporal pattern of the evoked voltage 336 changes, we combined the channel and temporal dimensions to improve the sensitivity of the decoder. 337 To do so, we first down-sampled the signal from the time-window of interest (50 ms to 450 ms, relative 338 to stimulus onset) to 50hz (51.2 Hz for Harrison and collegues 1, due to the original sampling rate of 339 1024Hz), and removed the mean activity level within each trial and electrode. The resulting, mean-340 centered 20 voltage values of each channel in each trial were then combined with the channel dimension. 341 The number of dimensions for the decoder increased therefore 20-fold (number of down-sampled time-342 points by number of channels). 343 Stimulus decoding 344 We used a Mahalanobis distance based decoder 19 to decode orientations from the simulated data and 345 orientations/locations from the spatiotemporal signals from the EEG data-sets. The approach was 346 identical for both orientation and location decoding apart from taking into account that orientations are 347 in 180° space, while locations are in 360° space. We used an 8-fold cross-validation approach. First, trials 348 were assigned to the closest of 16 evenly spaced orientations/locations (variable, see below). The trials 349 were then randomly split into 8 folds using stratified sampling. The trials of 1 fold were held out for 350 “testing” and the trials of the remaining 7 folds were part of the “training data". The covariance of the 351 train trials was estimated using a shrinkage estimator 32, before the number of trials in each 352 orientation/location bin of the train data was equalized through random subsampling. The subsampled 353 trials within each bin of the training set were then averaged. And the averaged bins were then convolved 354 with a half cosine basis set raised to the 15 th power 33 to pool information across similar 355 orientations/locations. The Mahalanobis distances between the left-out test trials and the averaged train 356 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 13 bins were then computed. This procedure was repeated for all train/test fold combinations. The 357 experiment of one dataset 23 used exactly 16 evenly spaced locations. Here the original location labels 358 were used, rendering the aforementioned binning unnecessary. All remaining datasets used random 359 orientations/locations, for which the above procedure was run separately for 8 possible ways of binning 360 the orientations (with bins centered at 0˚ to 168.75˚, at 1.40625˚ to 170.1563˚, at 2.8125˚ to 171.5625˚, 361 at 4.2188˚ to 172.9688˚, at 5.625˚ to 174.375˚, at 7.0313˚ to 175.7813˚, at 8.4375˚ to 177.1875˚, or at 362 9.8438˚ to 178.5938˚, each in 16 steps of 11.25) or location spaces (same as for orientation, but converted 363 to 360˚ space by multiplying all values by two). This means that for each trial, we obtained 16 times 8 = 364 128 Mahalanobis distance bins (with the exception of the data-set with only 16 discrete orientation, which 365 resulted in exactly 16 distances per trial). Given the randomness of the initial folds and the subsampling 366 within folds, the above procedure was repeated 20 times to obtain more robust results. Once all distances 367 were obtained and averaged over repetitions, distances for each trial were mean centered by subtracting 368 the average distance across all Mahalanobis bins from each. The distances were then ordered as a function 369 of angular difference between test and train bin, obtaining “pattern similarity curves” for each trial. For 370 experiments with two simultaneously presented orientation gratings, one on each side 18,19, each 371 orientation was decoded separately. 372 Relative decoding accuracy, relative precision, and bias 373 “Decoding accuracy” was obtained for each trial by computing the cosine vector mean of the “pattern 374 similarity curve”18. “Decoding accuracy” was then averaged as a function of orientation/location using a 375 sliding window (width = 11.25° for orientations, width = 25° for locations) that moved over angular space 376 in steps of 1.40625˚/ 2.8125˚ for orientations/locations. “Relative decoding accuracy” was obtained by 377 mean-centering the resulting orientation/location –specific decoding accuracy curve. "Precision" was 378 obtained by taking the circular means of the trial-wise "pattern similarity curves", and calculating the 379 inverse circular standard deviation over these means. “Relative precision” was obtained by mean -380 centering the precision curve (same as for "relative decoding accuracy). “Bias” was obtained by computing 381 the circular mean of the averaged “pattern similarity curves” of all trials within each angular window 382 (same as above). 383 For visualization, the angular relative decoding, relative precision and bias curves were smoothed across 384 orientations/locations with a Gaussian smoothing kernel (s.d. = 2˚/4˚ for orientations/locations). 385 To explicitly test differences in decoding accuracy and precision between vertical and horizontal 386 orientations (as shown in Figure 2), the respective decoding metrics were averaged from -22.5° to +22.5° 387 relative to 0° and 90° degrees. For the bias we assumed that, given equal attraction towards each cardinal, 388 the effect should be maximal for orientations 22.5° away from the cardinals, i.e., halfway the distance to 389 the obliques, where the influence of each cardinal should be cancelled out. We thus averaged the bias 390 values from -12.5° to 12.5° relative to 22.5° and 157.5° for horizontal orientations, and relative to 67.5° 391 and 112.5° for vertical orientations, after sign reversing bias values such that positive values always 392 correspond to attraction to the nearest cardinal. 393 Edge effects (“vignetting”) 394 We used the "perfect cube model" 26 to illustrate a possible relationship between location-specific SNR 395 differences, and “orientation energy”, strongest at the edges for circular gratings. We used the exact 396 stimulus and model parameters as described in ref 26. Briefly, two sine-wave gratings (one vertical, the 397 other horizontal) were convolved with eight distinctly oriented 2D Gabor “filters” (0° to 157.5°, in steps 398 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 14 of 22.5°), which all had the same spatial frequency as the sine-wave gratings. The output of each filter was 399 normalized before taking the sum over all eight, resulting in the 2D “orientation energy” plot in Figure 3b. 400 Statistical significance testing 401 The reported differences between horizontal and vertical orientations (Figure 2) were tested for 402 significance using a permutation t-test with 10,000 permutation as implemented by the python toolbox 403 MNE. All tests were two-sided and the statistical significance threshold was p < 0.05. 404 Code availability 405 The code used to generate the figures and results reported in this manuscript are available at 406 https://github.com/mijowolff/model-mimicry-and-unlikely-priors. 407 Data availability 408 All data used in this study is openly available17–19,23–25. For convenience, data from17–19 was reduced in 409 size by only including electrodes and time-points of interest, and is available at https://osf.io/bdf74/ 410 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 15 Supplemental References 411 32. Ledoit, O. & Wolf, M. Honey, I shrunk the sample covariance matrix. J. Portf. Manag. 30, 110–119 412 (2004). 413 33. Myers, N. E. et al. Testing sensory evidence against mnemonic templates. eLife 4, e09000 (2015). 414 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint 16 415 Supplemental figure 1. Same underlying models as in Figure 1, but the results and figures are obtained 416 from Harrison and colleagues 1 implementation of an IEM -based decoder and visualization approach by 417 using their published Matlab code, but folded 10 times as described in their Methods. 418 .CC-BY-NC 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted February 2, 2024. ; https://doi.org/10.1101/2024.01.31.578040doi: bioRxiv preprint