{"paper_id":"714db37e-702a-4147-b2d8-be83952249b6","body_text":"Bayesian Prior Uncertainty and Surprisal Elicit Distinct 1 \nNeural Patterns During Sound Localization in Dynamic 2 \nEnvironments 3 \n 4 \nBurcu Bayram* (a), David Meijer (b), Roberto Barumerli (b), Michelle Spierings (c), (d), 5 \nRobert Baumgartner (b), Ulrich Pomper (a)  6 \n 7 \n(a) Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, 8 \nUniversity of Vienna, Vienna, Austria 9 \n(b) Austrian Academy of Sciences, Acoustics Research Institute, Vienna, Austria 10 \n(c) Department of Behavioral and Cognitive Biology, University of Vienna, Vienna, Austria 11 \n(d) Department of Behavioral Biology, Leiden University, Leiden, Netherlands 12 \n 13 \n*corresponding author: 14 \nBurcu Bayram (burcu.bayram@univie.ac.at) 15 \n 16 \n 17 \n 18 \nKeywords  19 \nPerception, Bayesian inference, Neural oscillations, Auditory, Localization, EEG, Prior 20 \nuncertainty, Surprisal 21 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nAbstract 22 \nEstimating the location of a stimulus is a key function in sensory processing, and widely 23 \nconsidered to result from the integration of prior information and sensory input according to 24 \nBayesian principles. A deviation of sensory input from the prior elicits surprisal, depending 25 \non the uncertainty of the prior.  26 \nWhile this mechanism is increasingly understood in the visual domain, much less is known 27 \nabout its implementation in audition, especially regarding spatial localization. Here, we 28 \ncombined human EEG with computational modeling to study auditory spatial inference in a 29 \nnoisy, volatile environment and analyzed behavioral and neural patterns associated with prior 30 \nuncertainty and surprisal. 31 \nFirst, our results demonstrate that participants indeed used prior information during periods 32 \nof stable environmental statistics, but showed evidence of surprisal and discarded prior 33 \ninformation following environmental changes. Second, we observed distinct EEG activity 34 \npatterns associated with prior uncertainty and surprisal in both the time- and time-frequency 35 \ndomain, which are in line with previous studies using visual tasks. Third, these EEG activity 36 \npatterns were predictive of our participants’ sound localization error, response uncertainty, 37 \nand prior bias on a trial-by-trial basis.  38 \nIn summary, our work provides novel behavioral and neural evidence for Bayesian inference 39 \nduring dynamic auditory localization. 40 \n 41 \nIntroduction 42 \n 43 \nIn stable environments, perception can benefit from past experiences, especially when our 44 \nsensory representations are unreliable. Here, a mismatch between prior and sensory input 45 \nresults in prediction error, which can be used to update predictions and increase perceptual 46 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\naccuracy. However, following an abrupt change in the environment, prior information 47 \nquickly becomes irrelevant or even detrimental for the perceptual decision-making process1,2. 48 \nOne way to conceptualize optimal decision making in such dynamic environments is via 49 \nBayesian inference, in which perception is based on the integration of prior knowledge and 50 \nnew sensory observations, weighted by their reliabilities and the inferred probability of an 51 \nenvironmental change3,4.  52 \nIn the past, many studies have examined Bayesian inference using change-point paradigms, 53 \nin which a non-stationary environment is simulated by pseudo-randomly changing stimulus 54 \nstatistics throughout the experimental task4,5. In such dynamic environments, changes in the 55 \nenvironment lead to large surprisal and indicate that the prior is not relevant anymore. 56 \nThereby, humans have been shown to perform similar to an ideal Bayesian observer in 57 \nsettings such as visual spatial localization6,7, visual orientation discrimination8, and auditory 58 \npitch discrimination9, although sub-optimality in decision making has also been reported10-12. 59 \nFor instance, participants might not weight the sensory evidence and prior information 60 \naccording to their reliabilities12, especially when the task complexity is high10.  61 \nAt the neural level, a number of studies have found prediction signals (i.e. the prior) reflected 62 \nin beta-band (14 – 30 Hz)13-15 and surprisal reflected in gamma-band (40-100 Hz) oscillatory 63 \nactivity14-16. Sedley and colleagues15 employed a pitch discrimination task to disentangle 64 \nneural patterns associated with surprisal and prediction precision (i.e. inverse of prior 65 \nuncertainty). They observed that surprisal was reflected in gamma-band oscillations starting 66 \nfrom ~250 ms post-stimulus, and prior uncertainty about the next stimulus was positively 67 \ncorrelated with alpha-band oscillations (8-12 Hz) starting from ~280 ms post-stimulus. This 68 \nsuggests that the prior, its precision and surprisal are coded in distinct neural patterns.  69 \nSimilarly, Chao et al.14 used a hierarchical predictive coding model to differentiate the 70 \nfeedforward and feedback signals during a tone sequence discrimination task. They observed 71 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nfeedback prediction signals in beta range oscillations during the pre-stimulus time period, and 72 \nfeedforward prediction error signals reflected in gamma-band oscillations following stimulus 73 \npresentation. From a predictive coding perspective17, the above studies demonstrate that 74 \nsurprisal (or prediction errors) and prior are coded in higher (i.e. gamma) and lower (beta) 75 \nfrequency bands, respectively, while uncertainty associated with the prior information is 76 \nreflected in alpha-band oscillations. In addition to gamma-band oscillations, surprisal is 77 \ncommonly observed to scale with the amplitude of the P3 event-related potential (ERP), 78 \nalong with the updating of the prior information (i.e. belief update)8,18-20. Nassar and 79 \ncolleagues8 found that the effect of P3 amplitude and surprisal on belief update was 80 \ndependent on the statistical context in the environment. They used a change-point paradigm 81 \nin which larger surprisal values would indicate a true change in the environment as well as an 82 \noddball paradigm in which the oddball stimuli would trigger large surprisal responses without 83 \na change in the environment. Their results showed that contrary to the change-point 84 \nparadigm, participants did not update their beliefs even with high surprisal values in response 85 \nto oddball stimuli, even though in both conditions P3 amplitude scaled with surprisal.   86 \nSo far, evidence for a Bayesian inference mechanism comes primarily from the visual 87 \ndomain, as well as from auditory pitch or temporal estimation tasks6,15,20. However, 88 \nKrishnamurthy et al.21 have provided an intriguing behavioral study on Bayesian integration 89 \nof prior information and sensory evidence in auditory spatial localization. In their task, 90 \nparticipants had to first predict and then estimate the actual location of a sound source, as the 91 \npredictability of its location varied over time. They observed that changes in stimulus 92 \npredictability lead to changes in the magnitude of prior-driven biases, dependent on the 93 \nrelevance and reliability of prior expectations. In other words, periods of stable stimulus 94 \nstatistics enhanced prior usage, while recent changes reduced it. However, their study 95 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nprovided an additional visual representation for all the sounds establishing the prior, which 96 \narguably leads to multisensory (i.e. audio-visual) rather than purely auditory spatial priors.  97 \nAs humans perform better at localization of visual compared to auditory stimuli in unimodal 98 \ntasks22,23, vision is usually the dominant modality during multisensory localization24,25. In 99 \naddition, multisensory recalibration effects can transfer to subsequent unimodal tasks26 and 100 \nsubjects trained with audio-visual stimuli are more accurate in their sound localization 101 \nresponses compared to subjects trained using only auditory stimuli27, while, modality 102 \ndominance is reduced or even reversed with decreasing reliability of the visual stimuli28,29. 103 \nFor these reasons, it is unclear to what extend the results of Krishnamurthy et al.21 104 \nspecifically reflect Bayesian inference in auditory spatial localization, or are at least partly 105 \ndriven by visual priors. Consequently, the aim of our present study is to expand upon the 106 \nliterature by (1) investigating whether behavioral responses in a unimodal auditory 107 \nlocalization task adhere to the principles of Bayesian inference; (2) testing how this spatial 108 \ninference process is altered by the presence of additional visual priors (as in Krishnamurthy et 109 \nal.21); (3) study the neural patterns associated with prior uncertainty and surprisal, along with 110 \ntheir impact on subsequent behavioral responses. 111 \nParticipants listened to sequences of sounds coming from pseudorandom locations and 112 \nreported the location of the last sound of each trial. We conducted both an audio-visual 113 \ncondition (as in Krishnamurthy et al.21) and a modified audio-only condition while recording 114 \nhigh density EEG, and fitted a near optimal Bayesian observer model to participants’ 115 \nresponses. Our results reveal neural patterns reflecting prior uncertainty and surprisal for both 116 \nconditions in time-domain as well as the lower (i.e. alpha/beta) and higher (i.e. gamma) 117 \nfrequency range oscillations. Critically, we observed a significant relationship between the 118 \nneural activity associated with prior uncertainty and surprisal and the behavioural location 119 \nestimation error, response uncertainty, and prior bias. In summary, our results indicate 120 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nbehaviourally relevant electrophysiological patterns reflecting Bayesian inference processes 121 \nduring both auditory-only and audio-visual spatial localization in dynamic environments. 122 \n 123 \nMaterials and Methods 124 \nParticipants 125 \nThirty-five participants took part in the study, in exchange for monetary compensation (10 126 \nEuros per hour). Our sample size is based on a previous study employing a similar auditory 127 \nchange-point paradigm21. Three participants had a mean estimation error of  > 25° during the 128 \ntraining sessions and were excluded prior to the main experiment. We excluded three 129 \nadditional participants due to a large number of noisy EEG channels (> 10%). The remaining 130 \n29 participants were between 19 and 37 years old (15 females, Mage = 24.4, SDage = 3.8) and 131 \nright-handed. They reported no hearing impairments or neurological deficits, had normal or 132 \ncorrected-to-normal vision, gave informed consent, and were naïve to the purpose of the 133 \nexperiment. The study was conducted in accordance with the standards of the Declaration of 134 \nHelsinki (1996). We further followed the Austrian Universities Act of 2002, which states that 135 \nonly medical universities or studies conducting applied medical research are required to 136 \nobtain additional approval by an ethics committee. Therefore, no additional ethical approval 137 \nwas required for our study. 138 \n 139 \nExperimental setup and procedure 140 \nThe experiment was run using MATLAB (2018b, MathWorks, Natick, MA) and 141 \nPsychophysics Toolbox30. We presented visual stimuli via an LCD monitor (48 x 27 cm) with 142 \na refresh rate of 60 Hz and auditory stimuli via tube earphones (ER2; Etymotic Research, Elk 143 \nGrove Village, IL). Individual auditory stimuli consisted of 50 ms pink noise burst (10 ms 144 \non- and off-set ramps), high-pass filtered using a 4th order Butterworth filter with a 250 Hz 145 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\ncut-off frequency. Each sound was rendered at a specific direction by employing each 146 \nparticipant’s head-related transfer function (HRTF, measured prior to the experimental 147 \nsessions, using the same approach as in Ignatiadis et al.31) using the Auditory Modeling 148 \nToolbox32. 149 \nLow visual and auditory stimulus presentation latencies were verified via an oscilloscope (M 150 \n= 0 ms, SD = 4.4 ms). Participants sat at a desk in a dark and sound attenuated room, with 151 \ntheir head in a chin rest (75 cm distance to the screen) to minimize movement.  152 \nThe experiment was divided into two sessions, conducted on separate days (less than 3 days 153 \napart). The first session consisted of nine training blocks (not included in our analysis), to 154 \nfamiliarize participants with the sound localization task and response method. In the second 155 \nsession, participants completed two further training blocks and six main task blocks. Each 156 \nblock consisted of 50 trials, resulting in 300 trials for the main task.  157 \nThroughout the main task, we collected 128-channel high-density EEG (actiCAP with 158 \nactiCHamp; Brain Products GmbH, Gilching, Germany) and eye tracking data (EyeLink 159 \n1000 Plus; SR Research, Osgoode, Ontario, Canada), at a sampling rate of 1 kHz. Electrode 160 \nimpedances were kept below 25 kΩ and the signal was recorded against ‘FCz’ as reference 161 \nelectrode. In addition to the scalp electrodes, we recorded an auxiliary audio channel via a 162 \nstimulus tracking device (StimTrak, Brain Products GmbH, Gilching, Germany) to later align 163 \nthe EEG triggers offline with the onset of sounds and ensure correct trigger timing. 164 \n 165 \nTraining task 166 \nAt the beginning of each trial, participants fixated on a central dot (0.5° radius) displayed at 167 \nthe centre of the screen. Around the fixation dot, we displayed a semi-arc (0.75° width) that 168 \nrepresented the range of azimuth angles on the frontal horizontal plane (see Figure 1).  After 169 \n750-1000 ms of fixation, jittered on each trial, we presented a single sound from different 170 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nlocations in azimuth (from 90° to -90°). 950 ms after the sound offset, a mouse cursor 171 \nappeared at the location of the fixation dot, which signalled participants to respond and 172 \nturned into a line when moved close to the semi-arc. Their task then was to indicate the 173 \nazimuth location of the sound, by moving the line along the semi-arc (response resolution 174 \n<1°) via a mouse in their right hand, and click the left mouse button at the desired location. 175 \nSubsequently, they indicated their level of response uncertainty by marking an 80% 176 \nconfidence-interval around their response location. After that, feedback was provided for 500 177 \nms via a red line presented on the arc at the true sound location, followed by a new trial. At 178 \nthe end of each training block, participants received additional feedback on their mean 179 \nabsolute error (in degrees) and the percentage of times that their marked area of response 180 \nuncertainty included the true sound location (goal was 80%). 181 \nImportantly, trials alternated between an auditory-only (A) (as described above) and an 182 \naudio-visual (AV) condition. In the AV condition, a line indicating the sound location 183 \nappeared on the arc simultaneously with the sound presentation. We added this condition to 184 \nthe training, to help participants establish a mapping between the auditory sound location and 185 \nits abstract spatial representation on the visual arc on screen. 186 \n 187 \nMain task 188 \nThe main task was highly similar to the training, with the important difference that each trial 189 \nconsisted of a sequence of sounds, with a stimulus-onset asynchrony of 500 ms. Sound 190 \nlocations were randomly sampled from a normal distribution whose (generative) mean was 191 \nsampled from a uniform distribution bounded between 60 and -60 degrees, and a constant 192 \nstandard deviation of 10° (i.e. experimental noise). Each sound of the sequence had a 1/6 193 \nprobability of being a change-point, at which point a new generative mean was sampled from 194 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nthe bounded uniform distribution (60° to -60°), thus resulting in a sudden change of the mean 195 \nsound location.  196 \nThe participants’ task was to indicate the location of the last sound of the sequence (i.e. the 197 \nprobe sound). Each sequence contained between 1 and 43 sounds and each sound had a 1/12 198 \nprobability of being the probe sound, thus rendering the trial length unpredictable and 199 \nencouraging participants to pay attention to each sound location. Prior to the experiment, 200 \nparticipants were only superficially informed about the concepts of change-points and 201 \nexperimental noise, without mentioning specific parameter values.  202 \nAkin to the training blocks, we presented the A and AV condition trials in alternating order. 203 \nIn AV trials, all sounds except for the final probe sound were presented along with a 204 \nsimultaneous visual representation of the sound location via a line on the semi-arc. To ensure 205 \ncomparability of the two conditions, we presented identical trials in the A and AV conditions. 206 \nIn other words, each A trial had a corresponding AV trial, which only differed in the 207 \nadditional visual representations for the latter. The order of trials was randomized within 208 \nsubjects, and the corresponding A and AV trials were never presented in direct succession. 209 \nAs in the training task, participants gave localization and response uncertainty responses at 210 \nthe end of each trial. However, they did not receive immediate feedback on the true sound 211 \nlocation after each trial, but only a summary performance feedback at the end of each block, 212 \nregarding their mean absolute error (in degrees) and the percentage of times that their marked 213 \narea of response uncertainty included the true sound location. 214 \n 215 \nBayesian model 216 \nWe fitted a near optimal Bayesian observer model to our participants’ localization responses 217 \n(for modeling details see Supplement 1) to obtain two latent variables (i.e. not directly 218 \nobservable variables, which we inferred via our model) for every stimulus with optimized 219 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nparameters for each participant. The first latent variable is the prior uncertainty (𝑃𝑈), which 220 \nwe defined as standard deviation of the preceding posterior distribution (see Supplement 1, 221 \nEq. 25). The second latent variable is the information theoretic quantity of surprisal (𝑆𝑈). 222 \nThis is defined as the negative logarithm of the probability density of the full prior 223 \ndistribution, evaluated at the latest observation (see Supplement 1, Eq. 26) 224 \n 225 \nBehavioural analysis 226 \nUsing the participants’ responses, we derived three behavioural metrics for each trial and 227 \nanalysed them over two experimental factors. The behavioural metrics are: (i) the estimation 228 \nerror, which is calculated by computing the absolute difference between the localization 229 \nresponse and the sound’s true location; .(ii) the response uncertainty, indicating  the 230 \nconfidence area (in degrees) that participants marked around their response line at the end of 231 \neach trial; (iii) the prior bias,   computed as the estimation error on the current trial, divided 232 \nby the difference (in degrees) between the previous and the current sound locations: 233 \nPrior bias = (xresponse – x(t))/(x(t-1)- xt). 234 \n 235 \nThese metrics were evaluated over two main independent variables. The first one is the 236 \nsensory modality condition (AV vs. A), the second one is how many individual sounds had 237 \nbeen presented since the last change-point (sounds after change-point, SAC). In other words, 238 \nthe latter is a proxy of the strength of the present prior and the expected surprise elicited by 239 \nthe current sound. 240 \nTo investigate how our experimental manipulations affected the behavioral variables, we 241 \ncomputed a linear mixed effect model (LMM) using the glmmTMB33 package in R (version 242 \n4.3.2) with Modality (AV and A) and SAC level (1 to 6) as fixed effect variables and subjects 243 \nas random effect variable. Single sound trials are excluded from the analyses. 244 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 245 \nEEG preprocessing 246 \nAll EEG data preprocessing and analysis was performed using EEGlab34 (version 2022.1), 247 \nFieldtrip35 (version 20220827), and custom scripts. 248 \nFirst, we aligned our EEG triggers to sound onsets using the Stimtrak external channel. Next, 249 \nwe downsampled the continuous EEG signal to 250 Hz, applied a high-pass filter with a cut-250 \noff frequency of 0.25 Hz (Hamming window, zero-phase finite impulse response filter) using  251 \nthe ‘pop_eegfiltnew’ function from EEGlab and removed line noise using the ‘nt_zapline’ 252 \nfunction from NoiseTools toolbox36. We then epoched the data from -1 to 2 seconds relative 253 \nto the onset of each sound stimulus in the experiment, visually inspected the data and 254 \nexcluded epochs containing excessive noise as well as eye blinks close to the stimulus onset.  255 \nAs mentioned above, identical trials were presented in the A and AV conditions, although in 256 \nrandomized order. To maximize comparability between the A and AV conditions, for each 257 \nparticipant, we only analyzed trial epochs that were present in both conditions after artifact 258 \nrejection. Therefore, if we rejected an epoch in one condition, we also rejected the 259 \ncorresponding epoch in the other condition.  260 \nWe then interpolated noisy channels via spherical interpolation, added the reference electrode 261 \n‘FCz’ back to the dataset and re-referenced the data to the average of all electrodes.  262 \nNext, we performed independent component analysis (ICA) to identify and remove ocular 263 \nand heart-related artifacts. Particularly, in order to avoid using overlapping data segments for 264 \nthe ICA we performed the ICA decomposition on slightly differently preprocessed data: We 265 \nhigh-pass filtered the data at 1 Hz cut-off frequency37 and extracted the epochs between 0 to 266 \n500 ms for each sound (thus containing no overlapping data periods between epochs). The 267 \nremaining preprocessing steps were identical to the original dataset. We obtained ICA 268 \nweights from this alternatively preprocessed dataset using the Picard algorithm with PCA 269 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n(principal component analysis) to control for data rank deficiency caused by channel 270 \ninterpolation. Then we applied the resulting ICA weights back to the original dataset and 271 \nrejected ocular and heart-related components by visual inspection using the IClabel38 plugin. 272 \nFinally, we baseline corrected each epoch using the time period from -100 to 0 ms, relative to 273 \nthe sound onset.   274 \n 275 \nSpectral analysis 276 \nPrior to the spectral analysis, we subtracted the mean activity of all trials from each trial in to 277 \nanalyze only the induced oscillatory power. For the lower frequency range (4 to 30 Hz), we 278 \nperformed short-time fast Fourier transform, using a single Hanning taper with a window 279 \nlength of 300 ms in steps of 16 ms and a frequency resolution of 1 Hz. For the higher 280 \nfrequency range (40 to 100 Hz), we used the multi-taper method, with varying window length 281 \nof 250-100 ms (window length shortens with higher frequencies) in steps of 16 ms and a 282 \nfrequency resolution of 2 Hz. The resulting spectral power was then expressed as relative 283 \nsignal change to the mean of the time period from 0 to 500 ms around each sound (i.e. the 284 \nentire interval between the onset of the current and the next sound).  285 \n 286 \nRegression analysis 287 \nNext, our goal was to identify neural activity patterns associated with our Bayesian model 288 \nlatent variables PU (prior uncertainty) and SU (surprise). To do so, we performed three 289 \nseparate ordinary least squares linear regressions with both PU and SU values as predictors 290 \nand EEG amplitudes as the output: one for the time-domain data (i.e. ERPs), one for the 291 \nlower, and one for the higher frequency range. For the time-domain data, independent 292 \nregressions were performed for each EEG time-point and channel, while for the time-293 \nfrequency domain data, regressions also included the frequency domain. 294 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nBoth EEG data and latent variables were z-transformed prior to the regression. We excluded 295 \nthe final probe sounds of each sequence from this analysis, since they contain only auditory 296 \nstimuli in both A and AV conditions. Additionally, not including the final sounds that 297 \nparticipants responded upon in the regression analysis, allowed us to keep them as an 298 \nindependent data set to test the behavioral relevance of the regression results (see next 299 \nsubsection). The first sounds were also excluded from the analysis, as they do not have a 300 \nreliable estimate of the model latent variable values considering there are no preceding 301 \nsounds to form a prior. 302 \nFor each subject, this analysis resulted in a beta coefficient per time-point, channel, and 303 \nfrequency (for the time-frequency domain data), separately for each latent variable.  304 \nFor group statistics, we performed a total of six cluster-corrected permutation tests39, one for 305 \neach combination of latent variable (PU, SU) and EEG data (time domain, low – and high 306 \ntime-frequency domain), to compare whether the obtained beta coefficients differed 307 \nsignificantly from zero across participants. Cluster correction for multiple comparisons was 308 \napplied using a cluster-level alpha of .001, and permutation of the data points was performed 309 \nover 1000 iterations, and we only considered clusters with a duration of > 5ms length. For 310 \nPU, we tested the time period of -250 ms to 100 ms relative to every sound onset, as the prior 311 \nuncertainty should be neurally represented already pre-stimulus, and up until completion of   312 \ninitial sensory processing. For SU, on the other hand, we ran the test for the time period of 0 313 \nto 500 ms around each sound onset, as meaningful neural correlates of surprisal should only 314 \nappear following stimulus presentation.  315 \nAdditionally, we tested how the neural activity patterns associated with PU and SU variables 316 \ndiffered between the AV and A conditions. Again, we used a cluster-based permutation test 317 \nwith the cluster level alpha set to .001 and permutation of the data points performed over 318 \n1000 iterations.  319 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 320 \nBehavioral correlates of latent variable brain activation patterns 321 \nFinally, we were interested in whether the neural activity patterns associated with PU and SU, 322 \nas identified in the previous step, were predictive of the behavioral metrics on a trial-by-trial 323 \nbasis. Therefore, within each subject, we regressed EEG activity around the final probe sound 324 \nof each trial against the three behavioral variables estimation error, response uncertainty, and 325 \nprior bias. Specifically, we averaged the EEG activity around the probe sound across all 326 \nconstituting data points (time, channels, and frequencies) for every significant PU and SU 327 \nregression cluster. This resulted in a single value per trial, which we then regressed against 328 \nthe behavioral outcome of that trial (all variables were z-transformed prior to regression). For 329 \nthe group-level statistic, we performed one sample t-test (α = 0.05) for each regression 330 \nbetween EEG activity and behavioral outcome variables, to see whether the resulting beta 331 \ncoefficients differed significantly from zero. Single sound trials are excluded from the 332 \nanalyses. 333 \n 334 \nResults 335 \nBehavioral results 336 \nFigure 2 shows the behavioral and model data as a function of our two main independent 337 \nvariables, stimulus condition (AV vs. A) and SAC level (1 to 6). For the behavioral data, the 338 \nlinear mixed effects model revealed a smaller sound localization estimation error in the AV 339 \ncompared to the A condition (p < .01; see Table 1 for detailed statistical outcomes), as well as 340 \na larger prior bias (p < .001). In line with our expectations, these results suggest that the 341 \nadditional visual stimulus during non-probe sounds helped establishing a prior that eventually 342 \nimproved probe localization. Further, we observed a main effect of SAC level (p < .01) for 343 \nprior bias, indicating that bias was small immediately following a change-point, but increased 344 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nsubsequently. No other main effects or interactions were observed for the behavioral data (p 345 \n> .05). 346 \nFor the latent model variable PU, the LMM analysis revealed a main effect of Modality (p < 347 \n.001). This was due to larger PU values in the A compared to the AV condition, in line with 348 \nour expectation of a weaker prior in the former. Further, we found a main effect of SAC level 349 \n(p < .001), with the largest PU values at the second sound after a change-point, indicating that 350 \nchange-points led to a transient increase in PU.  351 \nFor the model variable SU, we found a significant main effect of Modality (p < .001), due to 352 \noverall larger SU values in the AV compared to the A condition. Again, this is likely a 353 \nconsequence of a stronger prior in the former, which leads to stronger surprisal following a 354 \nchange-point. Finally, we observed a main effect of SAC level (p < .001), due to surprisal 355 \nbeing largest directly after a change-point, and then decreasing with increasing numbers of 356 \nsounds after the change-point.  No interactions were observed for the model variables (p > 357 \n.25). 358 \n 359 \nEEG regression results 360 \nFigure 3 displays the results of the regression (i.e. beta coefficients) between the time-domain 361 \nEEG data and the model variables PU and SU (see Supplementary Table 2 for the cluster 362 \nstatistics). For both the AV and A condition, there are two time periods in which PU 363 \nsignificantly predicts the EEG activity patterns. The first is between around -250 to -100 ms 364 \nprior to sound onset, in line with activated prior information in expectation of a stimulus. The 365 \nsecond period is between 0 and 100 ms following a sound, potentially tracking the 366 \ncomparison between the prior and the sensory stimulus. Further, both periods are comprised 367 \nof a medio-central- and an occipital cluster of electrodes with opposing beta coefficient 368 \nvalues, likely due to the dipolar pattern of an underlying event-related potential. 369 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nFor the regression against SU, both AV and A conditions show significantly associated EEG 370 \ntime-domain patterns aggregated between 100 and 500 ms following a sound, in line with a 371 \nsurprisal response to the comparison between prior and sensory input. Again, the topography 372 \nshows a dipolar pattern, with medio-central and occipital clusters of opposing beta 373 \ncoefficients. In summary, the time domain regression shows large, spatially overlapping, but 374 \ntemporally distinct EEG activity patterns associated with PU and SU. 375 \nIn the time-frequency domain (Figure 4), regressions revealed that pre-stimulus 376 \nsynchronization in the delta-theta range was positively associated with PU, in both AV and A 377 \nconditions. Only for the AV condition, PU was negatively associated with synchronization 378 \naround the stimulus onset in the alpha and beta frequency range. Finally, also in the AV 379 \ncondition, pre-stimulus gamma power (~75 Hz) was positively associated with PU. 380 \nSeveral time-frequency activity patters are also significantly linked to SU values. In both the 381 \nAV and A condition, SU is associated with reduced low frequency activity around the initial 382 \nonset of the stimulus. Subsequently, however, around 200 ms, the relationship becomes 383 \npositive for the theta to alpha range, arguably due to enhanced processing of surprising 384 \nstimuli. Likewise, early post-stimulus activity also correlates positively with beta-band 385 \npower. During the later time period around 400 – 500 ms following the stimulus, 386 \nsynchronization in the theta-alpha as well as in the beta range (the latter only for AV) is 387 \nnegatively associated with SU. Lastly, gamma-band power (~70 – 90 Hz) in the AV 388 \ncondition was positively associated with SU at around 200 ms following the stimulus, and 389 \nnegatively associated later between 400 and 500 ms post-stimulus.  390 \nAs in the time-domain, PU and SU had temporally distinct, but spatially and spectrally 391 \noverlapping associated EEG activity patterns, suggesting a processing cascade from the 392 \nrepresentation of the prior to the computation of a prediction error and a subsequent surprise. 393 \n 394 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nNeural patterns in the AV versus A condition 395 \nOverall, regression coefficients in both time domain and time-frequency domain were larger 396 \nfor the AV compared to the A condition, resulting in more and larger significant clusters. In 397 \nthe time domain, comparison of the regression coefficients associated with PU between the 398 \ntwo conditions revealed two centrally located positive clusters and four negative fronto-399 \ntemporal clusters in the pre-stimulus time period (Figure 5). In the time-frequency domain, 400 \nwe found an earlier positive posterior cluster (~ -250 to -50ms) in theta range and a negative 401 \nfrontal cluster in beta range (~12-15 Hz) starting around 100 ms prior to the sound onset. No 402 \nsignificant clusters were found for PU in the higher frequency range. For SU in the time 403 \ndomain, we found three positive and three negative clusters between ~150 to 300ms, possibly 404 \nreflecting larger underlying ERPs in the AV compared to the A condition. In the lower 405 \nfrequency range, one negative fronto-central cluster in beta range (~12-14 Hz), and two 406 \nnegative clusters and one positive occipital cluster in theta range were found for SU, similarly 407 \ndue to the larger coefficients in AV condition. Finally, in the high frequency range we 408 \nobserved three positive clusters (~150 to 220 ms) between ~ 66-90 Hz associated with SU, 409 \nindicating larger coefficient values in the AV condition.  410 \n 411 \nBehavioral relevance of PU and SU related neural activity 412 \nAfter establishing the above distinct PU- and SU-related activity patterns for both the AV and 413 \nA conditions, we tested which of those patterns were, on a trial-by-trial level, predictive of 414 \nthe behavioral metrics (error, response uncertainty, prior bias; see Figure 6). Importantly, as 415 \ndetailed in the Methods section, we did so by regressing the behavioral variables against the 416 \nEEG data around the final probe sound of each trial. Thus, these data are independent of the 417 \ndata used for the regression against the model variables reported above, which are taken from 418 \neach sound of the sequence except the final probe sound.  419 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nThe participants’ estimation error in the A condition was significantly predicted by the 420 \nactivity of a frontal, pre-stimulus time-domain cluster positively associated with PU, as well 421 \nas an occipital post-stimulus cluster positively associated with SU. 422 \nIn the AV condition, estimation error was predicted by an early (0-150ms) posterior beta-423 \nband cluster positively associated with SU and a later (400 – 500ms) posterior beta-band 424 \ncluster negatively associated with SU. 425 \nPrior bias in the A condition was predicted by a posterior time-domain cluster (100 – 200ms) 426 \npositively associated with SU. In the AV condition, prior bias was predicted by a large time-427 \ndomain cluster (150 – 300ms) negatively associated with SU.  As can be seen from Figure 4, 428 \ntopographies and time-courses are similar for both AV and A SU time-courses, yet behavioral 429 \ncorrelation with prior bias became significant for two different, consecutive time periods for 430 \nthe two conditions, with different ERP polarities. Thus, although their association to SU is 431 \noppositional, these clusters are likely part of the same process, which switches EEG 432 \namplitude polarity at around 180ms. In the AV condition, prior bias was further predicted by 433 \ntwo time-frequency activity clusters. First, a large posterior cluster in the theta-alpha range 434 \n(350 – 500ms), in which a desynchronization was associated with more SU and eventually, 435 \nless prior bias. Second, a small left-posterior cluster in the gamma range (~470ms), in which 436 \na desynchronization was likewise associated with more SU and eventually, less prior bias. 437 \nFinally, the participants’ response uncertainty ratings in the A condition were predicted by 438 \ntwo clusters in the time-domain. First, an early (~75 to 150ms) large central cluster and 439 \nsecond, a small later (~320ms) posterior cluster, in both of which negative ERP amplitudes 440 \nwere associated with more surprisal and more response uncertainty. In the AV condition, 441 \nresponse uncertainty was associated with three clusters in the time-frequency domain: A 442 \ncentral-left cluster in the alpha-beta range (~ -50 – 100ms), in which a desynchronization was 443 \nassociated with more PU as well as more response uncertainty; and two occipital clusters in 444 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nthe gamma-range (~425 – 450ms), in which a desynchronization was associated with more 445 \nSU as well as less response uncertainty. 446 \n 447 \nDiscussion 448 \nIn the present study, we investigated behavioral and neural evidence for Bayesian inference 449 \nduring auditory spatial localization in dynamic environments.  450 \nWe report three main findings. First, our results show that participants continuously 451 \nintegrated prior knowledge into their estimations, subject to dynamic changes in a volatile 452 \nenvironment.  Second, these patterns of results were similar but amplified by the presence of 453 \nadditional visual location priors. Third, we observed distinct EEG activity patterns associated 454 \nwith Prior Uncertainty (PU) and Surprisal (SU), which are in line with previous studies using 455 \nvisual and/or temporal or pitch-related tasks. Importantly, these EEG activity patterns were 456 \npredictive of the error, response uncertainty, as well as the prior bias of our participants’ 457 \nbehavioral responses on a trial-by-trial basis.  458 \n 459 \nDynamic environmental changes impact behavioral and Bayesian model data 460 \nIn our experiment, we used random change-points and experimental noise to simulate a noisy 461 \ndynamic environment with momentary changes to the reliability of prior information and 462 \nsensory evidence. Data from both experimental conditions showed stronger prior bias with 463 \nincreasing SAC levels (i.e. accumulated sensory evidence). A similar effect was present for 464 \nPU, however with a peak at SAC level 2, as expected given the change-point at the previous 465 \nsound. Together, these results indicate that participants rely more on prior information, as 466 \nsensory evidence is accumulated during a period of stable environmental statistics and the 467 \nprior uncertainty decreases.  468 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nAt the same time, estimation error and SU were largest at SAC level 1 (i.e. immediately 469 \nfollowing a change-point). The large SU is indicative of a now irrelevant prior, which cannot 470 \nbe used to improve the localization performance anymore, indicating a reason to update the 471 \ninternal model. This was again similarly the case for both the AV and the A condition, 472 \nsuggesting a comparable mechanism for visual and auditory spatial inference (see below for a 473 \ndetailed discussion of the condition differences).  474 \nOur findings thus corroborate and extend the results by Krishnamurthy et al.21 who reported 475 \nsimilar effects following AV stimulation, by demonstrating Bayesian-like inference in 476 \nunimodal auditory settings and showing visual stimuli likely improved the sound localization 477 \nperformance by providing a more precise prior distribution. 478 \n 479 \nPrior Uncertainty is coded around stimulus onset, and associated with localization 480 \nperformance and response uncertainty 481 \nIn the time-domain, the pre-stimulus neural patterns associated with PU likely reflect late 482 \nERP responses to the previous stimulus, with their topography and latency suggesting a P3-483 \nlike component. The P3 is well documented to scale with surprisal and internal model 484 \nupdating8,18-20, which fits well with our observed positive association with subsequently 485 \nincreased PU. In the A condition, activity in this time-period was predictive of the behavioral 486 \nperformance, indicating that larger P3-like ERP responses (reflecting larger surprisal due to a 487 \nprediction error) to the previous stimulus were associated with larger estimation errors in the 488 \ncurrent trial. In the post-stimulus period, the significant clusters span the time-range of early 489 \nsensory processing including the P1 component. These scale negatively with PU, thus, the 490 \nstronger the prior, the larger the resulting sensory evoked ERP. Importantly, this early PU 491 \nregression pattern is still independent of the prediction error and any resulting surprisal, both 492 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nof which would be expected to appear later in the time period of a P3 ERP component and 493 \nscale positively with the ERP response.  494 \nThe above discussed pre-stimulus ERP likely underlies the theta-band time-frequency domain 495 \nresult, with which it shares a similar topography and time window. Additionally, low 496 \nfrequency oscillations have been associated with temporal expectancy of important upcoming 497 \nstimuli40. Considering the regular SOAs in our study (i.e. always 500 ms), their oscillatory 498 \npower in our results scale positively with PU, in line with stronger expectation of an 499 \nupcoming sensory event in times of weak prior information.  500 \nFurther, in line with numerous previous studies14,15, we found prior uncertainty reflected in 501 \nadjacent alpha- and beta-band activity patterns. Generally, the expectancy for an upcoming 502 \ntask-relevant stimulus is known to cause a decrease in alpha- and beta power, especially for 503 \nregular inter-stimulus intervals40. Thus, as PU increases, we would expect participants to 504 \nincrease the weight of sensory evidence rather than the prior, with the observed decrease in 505 \nalpha/beta power reflecting attentional preparation of the upcoming stimulus. Moreover, a 506 \npost-stimulus alpha/beta range cluster was both negatively associated with PU and response 507 \nuncertainty, suggesting a common neural pattern underlying the prior and the resulting 508 \nsubjective response uncertainty.  509 \nInterestingly, previous literature suggests a possible dissociation of neural patterns between 510 \npredictions and their precision, reflected in beta and alpha range oscillations, respectively13-15. 511 \nPresently, however, we found neural patterns associated with PU in multiple frequency bands 512 \nin pre-stimulus as well as post-stimulus period. A possible reason for this discrepancy is the 513 \napplied analysis pipeline. For instance, Sedley et al.15 partialized out the correlation between 514 \npredictor variables prior to their regression analysis, which might have canceled out 515 \ncorrelation in other frequency bands. Indeed, upon performing the same analysis without the 516 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\npartialization, they similarly observed an effect for prior precision in delta/theta, alpha and 517 \nbeta/gamma-bands. 518 \nFinally, we found a positive gamma cluster in the pre-stimulus time period for PU. 519 \nConsidering the short SOAs in our study (500ms), this association might be related to the 520 \nprevious stimulus processing reflecting facilitation of sensory processing and increased 521 \nweighting of the sensory likelihood in response to larger prior uncertainty and reduced prior 522 \nreliability. 523 \nHowever, the observed time-window (~250ms following the previous stimulus), the 524 \nbroadband nature, as well as its presence predominantly in the AV condition, indicate that the 525 \ngamma-response might be affected by microsaccadic muscular activity41. Although we 526 \ndiligently removed eye-related independent components, which can help to clear 527 \nmicrosaccadic artifacts from EEG data42, we are cautious to interpret the gamma response in 528 \nthis time-period as genuinely brain-related. 529 \n 530 \nSurprisal-related EEG activity evolves post-stimulus and is linked to behavioral error, 531 \nconfidence and prior bias 532 \nAs surprisal (SU) represents the deviation of sensory evidence from the prior, we 533 \nconsequently expected it to be represented in post-stimulus neural activity patterns. In the 534 \npast, EEG studies repeatedly found P3-like responses in response to surprising events 8,18-535 \n20, whose amplitude scales with the amount of surprisal and subsequent internal updating, 536 \nand which have been interpreted as the supporting evidence for the Bayesian brain 537 \nhypothesis43. In line with these results, SU in our data was associated with neural activity in 538 \nthe time range of the P3 ERP component, in both AV and A conditions. Importantly, this 539 \nadds to the previous literature by showing SU related responses in auditory spatial inference 540 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\ntasks, despite the auditory systems inferiority regarding localization, and thus demonstrates 541 \nthe ubiquity of the associated P3-related mechanism.  542 \nIn both the A and AV condition, larger amplitudes in early SU-related time-domain clusters 543 \n(~100-300 ms) were predictive of less prior bias in behavioral responses, indicating less use 544 \nof prior information following stronger neural correlates of surprisal. In addition, subjective 545 \nresponse uncertainty was likewise significantly positively associated with activity in two SU-546 \nrelated time-domain clusters, although only in the A condition. As to be expected, stronger 547 \nactivation of these patterns subsequently led to larger response uncertainty, on a trial-by-trial 548 \nbasis. 549 \nSurprisal has further been related to gamma-band oscillations, possibly reflecting the 550 \nprocessing of new sensory input14-16. Indeed, in the AV condition we found several clusters in 551 \nthe 70 – 90 Hz frequency range, associated with SU. While two of those falls into the time-552 \nrange of 200 – 300ms and are potentially driven by microsaccadic activity (as discussed 553 \nabove), three of the later ones between 400 and 500 ms are particularly interesting, as their 554 \nactivity correlates with subsequent response uncertainty and bias. More specifically, larger 555 \ngamma-band power in these clusters is associated with less surprisal and in subsequent 556 \nbehavior, larger prior bias and smaller response uncertainty.  557 \nIn addition to what previous studies have reported, our data also revealed SU to be reflected 558 \nin lower frequency activity. The activity patterns across space, time and frequency were very 559 \nsimilar between the AV and A conditions, with overall more pronounced power changes in 560 \nthe former. Starting from about 400ms after stimulus onset, alpha/beta-band power is 561 \nnegatively correlated with SU. High levels of surprisal indicate change-points which render 562 \nthe current prior irrelevant. Thus, with large SU levels alpha/beta range power decreases in 563 \norder to facilitate processing of the upcoming stimulus. In line with this interpretation, two 564 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nclusters in the AV condition’s alpha/beta range were predictive of the subsequent behavioral 565 \nestimation error and prior bias, respectively.  566 \nSeveral other clusters associated with surprisal were also predictive of behavioral outcomes. 567 \nInterestingly, activity in an early beta-band cluster in the AV condition scaled positively with 568 \nSU and subsequent localization accuracy (i.e. negatively with estimation error). The fact that 569 \nstronger neural correlates of SU would be associated with better performance is suggestive of 570 \nan attention effect, in that the appearance of a salient change-point leads to surprisal, but 571 \nconcurrently boosts attention and improves task performance.  572 \nThe fact that we did not find similar behaviorally relevant clusters in the A condition might 573 \nbe due to the lower signal-to-noise ratio as well as the lower stimulus intensity following an 574 \nA-only compared to a combined auditory and visual stimulus44.  575 \n 576 \nBehavioral and neural tokens of Bayesian inference are more pronounced following AV 577 \ncompared to A stimulation 578 \nOverall, the location estimation performance was better for the AV compared to the A 579 \ncondition, despite the identical probe sounds (both audio-only), in line with previous 580 \nstudies22,23. In part, this result likely reflects differences in the intrinsic sensory reliability of 581 \nvisual and auditory spatial information28,29. Additionally, A condition was particularly 582 \nchallenging due to the auditory-to-visual response mapping. Despite this difference, both 583 \nconditions showed response bias towards the prior stimulus locations, indicating participants 584 \nindeed kept track of the previous locations.  585 \nHowever, the higher sensory noise in the A-only condition led to larger PU and therefore less 586 \nprior bias during sound localization. In the AV condition, formation of the prior information 587 \nrelies on both senses and as a result, participants relied on their audio-visual prior more than 588 \nthe noisy audio-only sensory evidence provided by the probe sound. Similarly, a more precise 589 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nprior in the AV condition potentially led to larger SU values for the AV compared to the A 590 \ncondition.  591 \nOverall, our behavioral results indicate similar mechanism for integration of prior and 592 \nsensory likelihood following unimodal A and bimodal AV stimulation, that complies with 593 \nBayesian inference in a modality and task independent manner.  594 \nLikewise, the comparison of the neural activity associated with PU and SU between the 595 \nconditions revealed largely similar patterns. In both time- and time-frequency domain, the 596 \nobserved differences were most likely due to the larger and more extensive regression 597 \ncoefficient patterns for the AV compared to the A condition, due to stronger sensory 598 \nactivations and consequently a higher signal-to-noise ratio in the former.  599 \nFor PU, the comparison of the two conditions in the time domain revealed clusters in the pre-600 \nstimulus time period due to a more pronounced PU representation in the AV compared to the 601 \nA condition. In the time-frequency domain, the two conditions have spatially more distinct 602 \nPU patterns in theta range, reflected by a significant occipital cluster in comparison analysis, 603 \nthat likely stems from the additional visual prior representation in the AV condition.  604 \nComparison analysis revealed similar results for SU, mainly reflecting differences in strength 605 \nof the correlations between the two conditions. Additionally, these differences might partly 606 \nalso be driven by SU itself, which showed larger variation in the AV compared to the A 607 \ncondition. In the time domain, the differences are in the time range of ~200-300 ms due to 608 \nmore pronounced regression patterns in the AV compared to the A condition. Similarly, in 609 \nlower frequency range, the differences between the two conditions are mainly in the theta 610 \nrange and limited to occipital electrodes, possibly due to sensory specific processing of the 611 \nvisual stimulus in the AV condition. Finally, the difference in the gamma range clusters fall 612 \ninto the period of potential microsaccadic activity, and thus are likely driven by larger 613 \nartifacts in the AV condition due to the occurrence of an additional visual stimulus. 614 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nTaken together, these results suggest that auditory spatial localization in both AV and A 615 \nconditions is based on highly similar Bayesian inference mechanisms, which in the former 616 \nare additionally informed by the visual stimulus. Importantly, our data thus provide novel 617 \nevidence for auditory spatial localization according to Bayesian inference, while suggesting 618 \nthat the results following audio-visual priors previously reported by Krishnamurthy et al.21 619 \nare similarly applicable to true auditory-only settings. 620 \n 621 \nConclusions 622 \nPerception commonly relies on both sensory input and prior information. In the present study, 623 \nwe provide novel evidence that behavioral and neural responses during unimodal auditory 624 \nlocalization indeed conform with Bayesian inference principles. We demonstrate the impact 625 \nof dynamic changes in the environment on the weighting of prior knowledge and current 626 \nsensory evidence, and show that the resulting behavioral performance, model metrics and 627 \nneural patterns associated with PU and SU are in line with findings from other domains. 628 \nMoreover, these patterns are intensified yet structurally similar following additional visual 629 \npriors. 630 \nTaken together, despite the auditory system’s inferiority regarding spatial localization, our 631 \ndata suggest that it employs similar mechanisms as previously observed in visual processing 632 \nor more domain appropriate-tasks such as auditory pitch and temporal discrimination, 633 \nsupporting Bayesian inference as a general principle in human perceptual decision making. 634 \n 635 \nAcknowledgments 636 \nThis research was supported by an Austrian Science Fund (FWF) Young Independent 637 \nResearchers Group (Grant-DOI: 10.55776/ZK66) to Michelle Spierings, Ulrich Pomper, and 638 \nRobert Baumgartner. 639 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 640 \nAuthor Contributions 641 \nBurcu Bayram Data curation; formal analysis; visualization; writing-original draft 642 \nDavid Meijer Data curation; formal analysis; writing-original draft 643 \nRoberto Barumerli Formal analysis; writing- review and editing 644 \nMichelle Spierings Conceptualization; funding acquisition; project administration; 645 \nresources; writing- review and editing 646 \nRobert Baumgartner Conceptualization; funding acquisition; project administration; 647 \nresources; writing- review and editing 648 \nUlrich Pomper Conceptualization; funding acquisition; project administration; resources; 649 \nsupervision; writing-original draft 650 \n 651 \nConflict of Interest 652 \nThe authors declare no competing financial or non-financial interests. 653 \n 654 \nData Availability Statement 655 \nData are available upon request. 656 \n 657 \nReferences 658 \n 659 \n[1] Ma, W. J., & Jazayeri, M. (2014). Neural coding of uncertainty and probability. Annual 660 \nReview of Neuroscience, 37, 205–220. https://doi.org/10.1146/annurev-neuro-071013-014017 661 \n[2] Preuschoff, K., Mohr, P. N. C., & Hsu, M. (2013). Decision making under uncertainty. 662 \nFrontiers in Neuroscience, 7. https://doi.org/10.3389/fnins.2013.00218 663 \n[3] Nassar, M. R., Wilson, R. C., Heasly, B., & Gold, J. I. (2010). An Approximately 664 \nBayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing 665 \nEnvironment. Journal of Neuroscience, 30(37), 12366–12378. 666 \nhttps://doi.org/10.1523/JNEUROSCI.0822-10.2010 667 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n[4] O’Reilly, J. X. (2013). Making predictions in a changing world-inference, uncertainty, 668 \nand learning. Frontiers in Neuroscience, 7, 105. https://doi.org/10.3389/fnins.2013.00105 669 \n[5] Wilson, R., Nassar, M., & Gold, J. (2010). Bayesian Online Learning of the Hazard Rate 670 \nin Change-Point Problems. Neural Computation, 22, 2452–2476. 671 \nhttps://doi.org/10.1162/NECO_a_00007 672 \n[6] Bejjanki, V. R., Knill, D. C., & Aslin, R. N. (2016). Learning and inference using 673 \ncomplex generative models in a spatial localization task. Journal of Vision, 16(5), 9. 674 \nhttps://doi.org/10.1167/16.5.9 675 \n[7] Berniker, M., Voss, M., & Kording, K. (2010). Learning Priors for Bayesian 676 \nComputations in the Nervous System. PLoS ONE, 5(9), e12686. 677 \nhttps://doi.org/10.1371/journal.pone.0012686 678 \n[8] Nassar, M. R., Bruckner, R., & Frank, M. J. (2019). Statistical context dictates the 679 \nrelationship between feedback-related EEG signals and learning. ELife, 8, e46975. 680 \nhttps://doi.org/10.7554/eLife.46975 681 \n[9] Skerritt-Davis, B., & Elhilali, M. (2018). Detecting change in stochastic sound sequences. 682 \nPLOS Computational Biology, 14(5), e1006162. 683 \nhttps://doi.org/10.1371/journal.pcbi.1006162 684 \n[10] Kiryakova, R. K., Aston, S., Beierholm, U. R., & Nardini, M. (2020). Bayesian transfer 685 \nin a complex spatial localization task. Journal of Vision, 20(6), 17. 686 \nhttps://doi.org/10.1167/jov.20.6.17 687 \n[11] Stengård, E., & Berg, R. van den. (2019). Imperfect Bayesian inference in visual 688 \nperception. PLOS Computational Biology, 15(4), e1006465. 689 \nhttps://doi.org/10.1371/journal.pcbi.1006465 690 \n[12] Rahnev, D., & Denison, R. N. (2018). Suboptimality in perceptual decision making. The 691 \nBehavioral and Brain Sciences, 41, e223. https://doi.org/10.1017/S0140525X18000936 692 \n[13] Arnal, L. H., Wyart, V., & Giraud, A.-L. (2011). Transitions in neural oscillations reflect 693 \nprediction errors generated in audiovisual speech. Nature Neuroscience, 14(6), 797–801. 694 \nhttps://doi.org/10.1038/nn.2810 695 \n[14] Chao, Z. C., Huang, Y. T., & Wu, C.-T. (2022). A quantitative model reveals a 696 \nfrequency ordering of prediction and prediction-error signals in the human brain. 697 \nCommunications Biology, 5(1), 1076. https://doi.org/10.1038/s42003-022-04049-6 698 \n[15] Sedley, W., Gander, P. E., Kumar, S., Kovach, C. K., Oya, H., Kawasaki, H., Howard, 699 \nM. A., & Griffiths, T. D. (2016). Neural signatures of perceptual inference. ELife, 5, e11476. 700 \nhttps://doi.org/10.7554/eLife.11476 701 \n[16] Auksztulewicz, R., Friston, K. J., & Nobre, A. C. (2017). Task relevance modulates the 702 \nbehavioural and neural effects of sensory predictions. PLOS Biology, 15(12), e2003143. 703 \nhttps://doi.org/10.1371/journal.pbio.2003143 704 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n[17] Bastos, A. M., Lundqvist, M., Waite, A. S., Kopell, N., & Miller, E. K. (2020). Layer 705 \nand rhythm specificity for predictive routing. Proceedings of the National Academy of 706 \nSciences, 117(49), 31459-31469. 707 \n[18] Gómez, C. M., Arjona, A., Donnarumma, F., Maisto, D., Rodríguez-Martínez, E. I., & 708 \nPezzulo, G. (2019). Tracking the Time Course of Bayesian Inference With Event-Related 709 \nPotentials:A Study Using the Central Cue Posner Paradigm. Frontiers in Psychology, 10, 710 \n1424. https://doi.org/10.3389/fpsyg.2019.01424 711 \n[19] Kopp, B., & Lange, F. (2013). Electrophysiological indicators of surprise and entropy in 712 \ndynamic task-switching environments. Frontiers in Human Neuroscience, 7. 713 \nhttps://doi.org/10.3389/fnhum.2013.00300 714 \n[20] Visalli, A., Capizzi, M., Ambrosini, E., Kopp, B., & Vallesi, A. (2021). 715 \nElectroencephalographic correlates of temporal Bayesian belief updating and surprise. 716 \nNeuroImage, 231, 117867. https://doi.org/10.1016/j.neuroimage.2021.117867 717 \n[21] Krishnamurthy, K., Nassar, M. R., Sarode, S., & Gold, J. I. (2017). Arousal-related 718 \nadjustments of perceptual biases optimize perception in dynamic environments. Nature 719 \nHuman Behaviour, 1, 0107. https://doi.org/10.1038/s41562-017-0107 720 \n[22] King, A. J. (2009). Visual influences on auditory spatial learning. Philosophical 721 \nTransactions of the Royal Society B: Biological Sciences, 364(1515), 331–339. 722 \nhttps://doi.org/10.1098/rstb.2008.0230 723 \n[23] Odegaard, B., Wozny, D. R., & Shams, L. (2015). Biases in Visual, Auditory, and 724 \nAudiovisual Perception of Space. PLOS Computational Biology, 11(12), e1004649. 725 \nhttps://doi.org/10.1371/journal.pcbi.1004649 726 \n[24] Bolognini, N., Leor, F., Passamonti, C., Stein, B. E., & Làdavas, E. (2007). 727 \nMultisensory-Mediated Auditory Localization. Perception, 36(10), 1477–1485. 728 \nhttps://doi.org/10.1068/p5846 729 \n[25] Opoku-Baah, C., Schoenhaut, A. M., Vassall, S. G., Tovar, D. A., Ramachandran, R., & 730 \nWallace, M. T. (2021). Visual Influences on Auditory Behavioral, Neural, and Perceptual 731 \nProcesses: A Review. Journal of the Association for Research in Otolaryngology, 22(4), 732 \n365–386. https://doi.org/10.1007/s10162-021-00789-0 733 \n[26] Shams, L., Wozny, D. R., Kim, R., & Seitz, A. (2011). Influences of Multisensory 734 \nExperience on Subsequent Unisensory Processing. Frontiers in Psychology, 2. 735 \nhttps://doi.org/10.3389/fpsyg.2011.00264 736 \n[27] Cai, Y., Chen, G., Zhong, X., Yu, G., Mo, H., Jiang, J., Chen, X., Zhao, F., & Zheng, Y. 737 \n(2018). Influence of Audiovisual Training on Horizontal Sound Localization and Its Related 738 \nERP Response. Frontiers in Human Neuroscience, 12. 739 \nhttps://doi.org/10.3389/fnhum.2018.00423 740 \n[28] Alais, D., & Burr, D. (2004). The Ventriloquist Effect Results from Near-Optimal 741 \nBimodal Integration. Current Biology, 14(3), 257–262. 742 \nhttps://doi.org/10.1016/j.cub.2004.01.029 743 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n[29] Meijer, D., Veselič, S., Calafiore, C., & Noppeney, U. (2019). Integration of audiovisual 744 \nspatial signals is not consistent with maximum likelihood estimation. Cortex; a Journal 745 \nDevoted to the Study of the Nervous System and Behavior, 119, 74–88. 746 \nhttps://doi.org/10.1016/j.cortex.2019.03.026 747 \n[30] Kleiner, M., Brainard, D., & Pelli, D. (2007). What’s new in Psychtoolbox-3? 748 \nhttps://pure.mpg.de/rest/items/item_1790332/component/file_3136265/content 749 \n[31] Ignatiadis, K., Baier, D., Barumerli, R., Sziller, I., Tóth, B., & Baumgartner, R. (2024). 750 \nCortical signatures of auditory looming bias show cue-specific adaptation between newborns 751 \nand young adults. Communications Psychology, 2(1), 56. 752 \n[32] Majdak, P., Hollomey, C., & Baumgartner, R. (2022). AMT 1.x: A toolbox for 753 \nreproducible research in auditory modeling. Acta Acustica, 6, 19. 754 \nhttps://doi.org/10.1051/aacus/2022011 755 \n[33] Brooks, M. E., Kristensen, K., van Benthem, K. J., Magnusson, A., Berg, C. W., 756 \nNielsen, A., Skaug, H. J., Mächler, M., & Bolker, B. M. (2017). GlmmTMB Balances Speed 757 \nand Flexibility Among Packages for Zero-inflated Generalized Linear Mixed Modeling. 9. 758 \n[34] Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of 759 \nsingle-trial EEG dynamics including independent component analysis. Journal of 760 \nNeuroscience Methods, 134(1), 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009 761 \n[35] Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2011). FieldTrip: Open source 762 \nsoftware for advanced analysis of MEG, EEG, and invasive electrophysiological data. 763 \nComputational Intelligence and Neuroscience, 2011, 156869. 764 \nhttps://doi.org/10.1155/2011/156869 765 \n[36] de Cheveigné, A. (2020). ZapLine: A simple and effective method to remove power line 766 \nartifacts. NeuroImage, 207, 116356. https://doi.org/10.1016/j.neuroimage.2019.116356 767 \n[37] Winkler, I., Debener, S., Muller, K.-R., & Tangermann, M. (2015). On the influence of 768 \nhigh-pass filtering on ICA-based artifact reduction in EEG-ERP. 2015 37th Annual 769 \nInternational Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 770 \n4101–4105. https://doi.org/10.1109/EMBC.2015.7319296 771 \n[38] Pion-Tonachini, L., Kreutz-Delgado, K., & Makeig, S. (2019). ICLabel: An automated 772 \nelectroencephalographic independent component classifier, dataset, and website. 773 \nNeuroImage, 198, 181–197. https://doi.org/10.1016/j.neuroimage.2019.05.026 774 \n[39] Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-775 \ndata. Journal of Neuroscience Methods, 164(1), 177–190. 776 \nhttps://doi.org/10.1016/j.jneumeth.2007.03.024 777 \n[40] Pomper, U., Keil, J., Foxe, J. J., & Senkowski, D. (2015). Intersensory selective 778 \nattention and temporal orienting operate in parallel and are instantiated in spatially distinct 779 \nsensory and motor cortices: Human Intersensory and Temporal Attention. Human Brain 780 \nMapping, 36(8), 3246–3259. https://doi.org/10.1002/hbm.22845 781 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n[41] Yuval-Greenberg, S., Tomer, O., Keren, A. S., Nelken, I., & Deouell, L. Y. (2008). 782 \nTransient Induced Gamma-Band Response in EEG as a Manifestation of Miniature Saccades. 783 \nNeuron, 58(3), 429–441. https://doi.org/10.1016/j.neuron.2008.03.027 784 \n[42] Hipp, J. F., & Siegel, M. (2013). Dissociating neuronal gamma-band activity from 785 \ncranial and ocular muscle activity in EEG. Frontiers in Human Neuroscience, 7, 338. 786 \nhttps://doi.org/10.3389/fnhum.2013.00338 787 \n[43] Friston, K. (2012). The history of the future of the Bayesian brain. Neuroimage, 62–788 \n248(2), 1230–1233. https://doi.org/10.1016/j.neuroimage.2011.10.004 789 \n[44] Busch, N. A., Debener, S., Kranczioch, C., Engel, A. K., & Herrmann, C. S. (2004). Size 790 \nmatters: Effects of stimulus size, duration and eccentricity on the visual gamma-band 791 \nresponse. Clinical Neurophysiology: Official Journal of the International Federation of 792 \nClinical Neurophysiology, 115(8), 1810–1820. https://doi.org/10.1016/j.clinph.2004.03.015 793 \n 794 \nFigure legends 795 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 796 \n 797 \nFigure 1. Experimental design. Left part: An exemplary trial depicted for both audio-visual 798 \n(AV) and audio-only (A) conditions. Each trial starts with a fixation period followed by a 799 \nsequence of sounds. In the A condition, only auditory stimuli are presented. In the AV 800 \ncondition, the respective sound location is simultaneously shown via a line on the semi-arc 801 \n(for all sounds except for the final probe sound). At the end of the sequence, participants 802 \nindicate the location of the probe sound by rotating the response line on the semi-arc. 803 \nSubsequently, they mark an 80% confidence interval around the response line. Right part: 804 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nDepiction of the sampling of sound locations for an exemplary trial. Sound locations are 805 \nsampled from a Gaussian distribution with standard deviation of 10° (i.e. experimental noise). 806 \nAlong with each change-point sound (turquoise circle), a new mean sound location is 807 \nsampled. Four sounds are presented following the last change-point in the sequence and 808 \ntherefore, the trial has a stimulus-after-change-point (SAC) level of 4. 809 \n 810 \n 811 \nFigure 2. Behavioral and modeling results. A) Mean of the behavioral outcomes estimation 812 \nerror, prior bias, and response uncertainty, per sound-after-changepoint (SAC) level, for both 813 \naudio-visual and auditory conditions. Error bars indicate the standard error of the mean.  B) 814 \nMedian of the Prior Uncertainty1 (PU) and Surprisal (SU) values. Single subjects’ PU and SU 815 \nvalues are z-scored. Note, that the SEM is particularly small (mean = .59) for PU in the AV 816 \ncondition. This is due to the fact, that while we fitted the sensory noise individually for each 817 \nparticipant for the A condition, we fixed the sensory noise at one degree for the AV 818 \ncondition. Furthermore, we used the same set of trials for all participants, resulting in no 819 \nvariation across the trial sets. Hence, for a particular stimulus, our model predicts the same 820 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nPU for every participant. The only remaining variation stems from the random sensory noise 821 \nof the AV stimuli. 822 \n 823 \n 824 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 825 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nFigure 3. Event-related responses associated with Prior Uncertainty (PU) and Surprisal (SU). 826 \nLeft column: Time-domain representations of clusters (numbered) of regression coefficients 827 \nbetween EEG amplitudes and model latent variables, significant at the group-level. Data are 828 \nshown separately for the audio-visual (AV) and auditory (A) conditions, as well as for PU 829 \nand SU variables. Each line shows beta coefficients averaged over channels and time points 830 \nof each significant cluster, obtained via permutation testing. Significant time points are 831 \nmarked in bold colors. Right column: Topographical plots of each significant cluster, as 832 \nnumbered on the left side as. Significant channels are marked with asterisks. Behaviorally 833 \nrelevant clusters (see Figure 5A) are highlighted via a purple square in the background. Small 834 \nclusters (< 2 datapoints) are not shown, unless they are behaviorally relevant. 835 \n 836 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 837 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 838 \nFigure 4. A)  Time-frequency responses (lower frequency range, 4-30 Hz) associated with 839 \nPrior Uncertainty (PU) and Surprisal (SU). Left column: Time-frequency representations of 840 \nregression coefficients between EEG amplitudes and model latent variables, averaged over all 841 \nsignificant channels obtained via permutation testing. Significant clusters at the group-level 842 \nare highlighted and numbered. Data are shown separately for the audio-visual (AV) and 843 \nauditory (A) conditions, as well as for PU and SU variables. Right column: Topography plots 844 \nof each significant cluster, as numbered on the left side as. Significant channels of the 845 \nclusters are marked with asterisks. Behaviorally relevant clusters (see Figure 6B) are 846 \nhighlighted via a purple square in the background. B) Same as A, for higher frequency range 847 \n(i.e. 40-100 Hz). Small clusters (< 2 datapoints) are not shown, unless they are behaviorally 848 \nrelevant. 849 \n 850 \n 851 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 852 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nFigure 5. Differences in Prior Uncertainty (PU) and Surprisal (SU) related activity between 853 \nthe audio-visual (AV) and auditory (A) conditions. All data represent the difference of 854 \nregression coefficients (between the EEG data and the PU and SU latent variables, see 855 \nFigures 4 and 5) between the AV and A conditions. A) Left column: Time-domain 856 \nrepresentations of regression coefficient differences. Data are shown separately for the PU 857 \nand SU variables. Each line shows beta coefficient differences averaged over channels and 858 \ntime points of each significant cluster, obtained via permutation testing. Significant time 859 \npoints are marked in bold colors. Right column: Topographical plots of each significant 860 \ncluster, as numbered on the left side as. Significant channels are marked with asterisks.  861 \nB) Left column: Time-frequency (4 – 30 Hz) domain representations of regression coefficient 862 \ndifferences. Data are shown separately for the PU and SU variables. Significant clusters at the 863 \ngroup-level are highlighted and numbered. Data are shown separately for the PU and S 864 \nvariables. Right column: Topography plots of each significant cluster, as numbered on the 865 \nleft side as. Significant channels of the clusters are marked with asterisks. C) Same as B, for 866 \nthe 40-100 Hz time-frequency domain. Small clusters (< 2 datapoints) are not shown. 867 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 868 \n 869 \nFigure 6. Model latent variable components predict behavior. A) Results of the regression 870 \nanalysis between time-domain EEG data averaged over the dimensions of significant 871 \nregression clusters (see Figure 4) and the behavioral estimation error (EE), response 872 \nuncertainty (RU), and prior bias (PB) variables. Each circle in the violin plots represents 873 \nsingle subject beta coefficients. Grey bars represent the median, interquartile range and 1.5 874 \ntimes the interquartile range. Respective topoplots of the significant clusters are shown on top 875 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nof the violin plots (same as the highlighted clusters in Figures 4). B) Same as A), for the 4-30 876 \nHz time-frequency domain clusters (see Figure 5). C) Same as B), for the 40-100 Hz time-877 \nfrequency domain clusters (see Figure 6). 878 \n 879 \nTABLES 880 \n 881 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\nTable 1. Behavioral results\nStatistical results obtained using linear mixed-effect modeling.\nA. Estimation error\nFormula: Error ~ 1 + SAC * Modality * +(1 | Subjects)\nFixed effects coefficients Estimate SE z Pr(>|z|)\n(Intercept) -1.37 0.82 -1.69 0.09 .\nSAC 0.2 0.17 1.17 0.24\nModality=AV 2.56 0.89 2.88 <4e-3 **\nSAC:Modality=AV -0.46 0.24 -1.92 0.05\nB. Response uncertainty\nFormula: RU ~ 1 + SAC * Modality * +(1 | Subjects)\nFixed effects coefficients Estimate SE z Pr(>|z|)\n(Intercept) 49.52 2.44 20.29 <2e-16 ***\nSAC -0.09 0.14 -0.63 0.53\nModality=AV -1.06 0.76 -1.4 0.16\nSAC:Modality=AV -0.35 0.2 -1.71 0.09\nC. Prior bias\nFormula: Bias ~ 1 + SAC * Modality * +(1 | Subjects)\nFixed effects coefficients Estimate SE z Pr(>|z|)\n(Intercept) 0.12 0.05 2.64 8.22e-3 **\nSAC 0.02 0.01 3.38 7.2e-4 ***\nModality=AV 0.2 0.04 5.23 1.65e-07 ***\nSAC:Modality=AV <3e-4 0.01 0.02 0.98\nD. Prior uncertainty\nFormula: PU ~ 1 + SAC * Modality * +(1 | Subjects)\nFixed effects coefficients Estimate SE z Pr(>|z|)\n(Intercept) 16.57 0.3 54.37 <2e-16 ***\nSAC -0.37 0.02 -15.21 <2e-16 ***\nModality=AV -7.64 0.13 -59.52 <2e-16 ***\nSAC:Modality=AV -0.04 0.03 -1.14 0.25\nE. Surprisal\nFormula: SU ~ 1 + SAC * Modality * +(1 | Subjects)\nFixed effects coefficients Estimate SE z Pr(>|z|)\n(Intercept) 5.83 0.1 57.74 <2e-16 ***\nSAC -0.17 0.01 -13.86 <2e-16 ***\nModality=AV 0.3 0.07 4.49 7.02e-06 ***\nSAC:Modality=AV -0.01 0.08 -0.5 0.62\n 882 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint \n\n 883 \nTable 2. Cluster p-values obtained from regression analysis\nA. Time domain regression clusters\nC1 C2 C3 C4 C5 C6 C7\nAV PU 0.001 0.001 0.001 0.001\nA PU 0.001 0.001 0.001 0.001 0.017 0.006\nAV SU 0.012 0.005 0.001 0.001 0.007 0.013 0.011\nA SU 0.004 0.002 0.001 0.002 0.002 0.017 0.004\nB. Time-frequency domain lower frequency range (4-30 Hz) regression clusters\nC1 C2 C3 C4 C5 C6 C7\nAV PU 0.001 0.001 0.001 0.002\nA PU 0.001 0.023 0.024\nAV SU 0.014 0.018 0.001 0.001 0.001 0.001 0.006\nA SU 0.001 0.001 0.001 0.001 0.013\nC. Time-frequency domain higher frequency range (40-100 Hz) regression clusters\nC1 C2 C3 C4 C5 C6 C7\nAV PU 0.001\nAV SU 0.001 0.002 0.001 0.003 0.023 0.001 0.005\n 884 \nTable 3. Cluster p-values obtained from condition comparison analysis\nA. Time domain regression clusters\nC1 C2 C3 C4 C5 C6 C7\nAV-A PU 0.004 0.001 0.001 0.001 0.005 0.022\nAV-A SU 0.007 0.002 0.006 0.002 0.001 0.014\nB. Time-frequency domain lower frequency range (4-30 Hz) regression clusters\nC1 C2 C3 C4 C5 C6 C7\nAV-A PU 0.002 0.013\nAV-A SU 0.001 0.004 0.001 0.015\nC. Time-frequency domain higher frequency range (40-100 Hz) regression clusters\nC1 C2 C3 C4 C5 C6 C7\nAV-A SU 0.002 0.001 0.004\n 885 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted July 23, 2024. ; https://doi.org/10.1101/2024.07.22.604566doi: bioRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}