Feature interference as a neuronal basis for the behavioural cost of task uncertainty

13 Humans and animals have an impressive ability to juggle multiple tasks in a constantly changing 14 environment. This flexibility, however, leads to decreased performance under uncertain task 15 conditions. Here, we combined monkey electrophysiology, human psychophysics, and artificial 16 neural network modeling to investigate the neuronal mechanisms of this performance cost. We 17 developed a behavioural paradigm to measure and influence participants' decision -making and 18 perception in two distinct perceptual tasks. Our data revealed that both humans and monkeys, 19 unlike an artificial neural network trained for the same tasks, make less accurate perceptual 20 decisions when the task is uncertain. We generated a mechanistic hypothesis by comparing this 21 neural network trained to produce correct choices with another network trained to replicate the 22 participants’ choices. We hypothesized, and confirmed with further behavioural, physiological, 23 and causal experiments, that the cost of task flexibility comes from what we term task 24 interference. Under uncertain conditions, interference between different tasks causes errors 25 because it results in a stronger representation of irrelevant task features and entangled neuronal 26 representations of different features. Our results suggest a tantalizing, general hypothesis: that 27 cognitive capacity limitations, both in health and disease, stem from interference between neural 28 representations of different stimuli, tasks, or memories. 29 30

31 In the face of the inherent uncertainty and unexpected changes in natural environments, humans 32 and animals have evolved remarkable cognitive flexibility, enabling them to seamlessly switch 33 between tasks as the situation demands. However, this cognitive flexibility comes with a cost : 34 when we are uncertain which task should be performed (typically during and shortly before or 35 after task switching), we take longer to perform the task we choose and do so with lower accuracy 36 (1-4). 37 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint Numerous non-exclusive hypotheses have been proposed based on behavioural evidence to 1 explain the cost of task flexibility . These include limits on processing capacity (4), excessive 2 attention to irrelevant stimuli or features (5-8) and proactive interference between competing 3 tasks (1, 2, 9-12). These phenomenological explanations of behaviour do not directly address the 4 neural mechanisms that underly task switching costs and related limits on other forms of 5 cognition. 6 Here, we propose a novel neural mechanism that integrates and elucidates the roles and 7 connections among various hypotheses addressing cognitive flexibility cost . We used a 8 multidisciplinary approach including behavioural experiments in humans and rhesus monkeys, 9 multi-neuron, multi-area recordings and causal manipulations in monkeys, and artificial neural 10 network modeling to generate and test neural hypotheses. The core of our approach is our two-11 feature visual discrimination task, in which participants must discriminate one visual feature and 12 ignore the other, inferring the implicitly changing task rule that determines which feature is 13 relevant, based on their history of stimuli, choices and rewards. 14 Our findings suggest that the cost of task flexibility arises from stronger task interference under 15 uncertain conditions . Such interference is a joint product of retaining task -irrelevant visual 16 information and the entanglement of the neural representations of unrelated feature s. This 17 phenomenon likely underlies the cost of cognitive flexibility across species, systems, and health 18 states, offering promising avenues for advancing our understanding and addressing cognitive 19

in both basic science and translational research. 20 21

22 Task switching can have behavioural costs. 23 We designed a two-feature discrimination task that allows us to measure and manipulate 24 perception and task certainty, pushing participants to flexible (task uncertain) and engaged (task 25 certain) cognitive states(3) (Figure 1A). We trained 220 online human participants and two rhesus 26 monkeys (Macaca mulatta; both male, 12 and 9 kg) to discriminate changes in two visual features 27 of subsequently presented Gabor stimuli ( spatial frequency and orientation for human 28 participants; spatial frequency and location for monkey participants). Both features changed on 29 each trial near perceptual threshold ( average perceptual performance is 79% for monkeys, and 30 82% for humans), but only one was relevant. Participants indicated both their chosen task and 31 perceptual discrimination on every trial, and only correct discriminations of the relevant feature 32 were rewarded (Figure 1A-B). The relevant feature switched without a cue with a constant low 33 probability on each trial (2.5% for monkeys and 10% for humans). 34 The task structure meant that t he participants' certainty about which task they would be 35 rewarded for performing fluctuated throughout the session based on their history of stimuli, 36 choices, and rewards. A rewarded trial suggested a high likelihood of the same task on the next 37 trial (97.5% for monkeys, 90% for humans), thus leaving participants with higher task certainty . 38 An unrewarded trial, however, led to task uncertainty because participants received no explicit 39 feedback about the error type (i.e., whether the task had switched, they had made a perceptual 40 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint error, or both). For simplicity, we will refer to trials as having high or low task certainty based on 1 whether they were immediately preceded by a rewarded trial or an unrewarded trial . This 2 dichotomy proves to be an effective separation between high and low task certainty, supported 3 by task-related neuronal representations in the parietal cortex, and distinguished behavioural 4 patterns not solely attributable to recent rewards or trial position in a certain block (3). In the 5 following results , we will compare the trial condition under low task certainty (following an 6 unrewarded trial) vs. high task certainty (following a rewarded trial) to evaluate the cost for 7 flexibility. 8 9 10 FIG 1: Tasks and behavioural cost of flexibility. (A) Monkey s discriminated either the spatial 11 frequency (cyan) or location (magenta) of two sequentially presented Gabor stimuli while 12 ignoring changes in the irrelevant feature . The relevant feature is not explicitly cued. There 13 was a fixed, low probability of a task switch after each trial, leading to task blocks of variable 14 Perform frequency task Perform location task Task switching Hi gh Hi gh Lo w Task certainty Choice op ti ons Task Re wardTrial stimuli Perceptual Stim. 1 De lay Stim. 2 L FLFF Low task certaintyHigh task certainty F: frequency task L: location task Trials within a session C After a rewarded trial (High task certainty) After an unrewarded trial (Low task certainty) Perceptual performance on the chosen task 0.6 0.8 0.6 0.8 1 p = 0 .0 02 p = 0 .0 008 Orient. choices Freq. choices 1 0.6 0.8 1 Freq. choices Loc. choices monkey G F 0.6 0.8 1 D Task certainty influence on reaction time -600 -300 0 30 0 60 0 5 10 15 20Number of subjects Frequency task Orientation task Reaction time difference (ms) comparing trials with high vs. low task certainty p = 2×10-8 p = 9×10-7 0.1 0.2 0.3Proportion of conditions p = 3×10-14 p = 2×10-9 Location task Frequency task -80 -40 0 40 80 -0.8 -0.4 0 0.4 0.8 0 0.1Proportion of subjects Task confidence ~ perceptual confidence Orientation task Frequency task Correlation coefficient p = 5×10-17 p = 8×10-17 A B longer times with lo w task certainty longer times with hi gh task certainty Fre qu ency task Ori e n tat io n taskTask confidence Frequency discrimination confidence Orient at ion discrimination confidence Choice optionsTrial stimuli Frequency task Orient at ion task E p < 1 0-6 p < 1 0-6 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint length (cyan and magenta rectangles). Participants had to infer the relevant feature from 1 their history of choices and feedback. Participants were rewarded only if they made the 2 correct task choice and perceptual judgment (they indicated both on each trial by choosing 3 one of four possible targets that represent the choices schematized in the first column of the 4 table in A) . Therefore, task certainty was generally high following rewarded trials and low 5 following unrewarded trials. (B) Human participants performed the same task as the monkeys, 6 alternating between orientation discrimination and frequency discrimination . On a small 7 proportion of trials , some human participants were asked to report their confidence about 8 their task choice and perceptual discrimination of both features on a continuous scale . (C) 9 Behavioural cost of flexibility in perceptual performance for monkeys (top) and humans 10 (bottom). Each open symbol represents perceptual performance for a set of trials with 11 identical stimuli. Under low task certainty (following an unrewarded trial), both species show 12 worse perceptual performance under low task certainty (following an unrewarded trial; 13 ordinates) than under high task certainty (following a rewarded trial; abscissae; p=0.002 for 14 human orientation choices, p=0.0008 for human frequency choices; p<10-6 for monkey 15 location and frequency choices. Filled symbols denote mean performance. (D) Behavioural 16 cost of flexibility in reaction time. The upper (monkey) and lower (human ) histograms show 17 distributions of the differences between reaction times under high and low task certainty for 18 matched stimuli. The histograms are shifted to the left, indicating that participants reported 19 their perceptual discriminations more slowly under low task certainty than under high task 20 certainty. p<106 for both species and both tasks. (E) Behavioural cost of flexibility in 21 confidence. Cyan and magenta curves show the distribution of correlation coefficients 22 between task confidence and perceptual confidence on the chosen task for human 23 participants. p<106 for both tasks. 24 25 Both humans and monkeys demonstrate a cost of flexibility. Our participants perform the 26 perceptual task better (Figure 1C), faster (Figure 1D), and with greater confidence (Figure 1 E) 27 following a rewarded trial, when task certainty is high. 28 29 Recurrent neural networks trained on behavioural data can generate mechanistic hypotheses 30 To generate hypotheses about the neural mechanisms underlying this cost of flexibility, we 31 trained recurrent neural networks (RNNs) to perform the same two-feature discrimination task 32 and then analyzed the internal dynamics of the models. We used two training strategies to create 33 two distinct, comparable models. Both models received as inputs information about features of 34 the stimuli on the current trial and the history of stimuli, choices, and rewards over the past nine 35 trials (see Methods: RNN models and analyses) . The first model, the “correct choice network,” 36 was trained to produce the correct choice given those inputs (Figure 2A; see Methods ). The 37 correct choice network did not exhibit a cost of flexibility at any point during training (Figure 2B), 38 which indicates that the cost of flexibility is not somehow related to the structure or statistics of 39 our task ; nor does it necessarily appear during the intermediate stage of task learning . The 40 second model, the “monkey choice network,” was trained to predict the monkeys’ actual choices 41 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint (Figure 2A). Because of the way it was trained , unlike the correct choice network, the monkey 1 choice network exhibited the cost of flexibility characteristic of the behaviour of our monkeys 2 and human participants, which persisted from about halfway through training (Figure 2B and 2C). 3 To model reaction times, we applied a collapsing decision boundary to the four choice output 4 units such that the reaction time is given by the first timepoint in which any choice unit’s activity 5 crosses the boundary. The boundary parameters were chosen such that the mean reaction times 6 of the monkey choice network after rewarded and unrewarded trials were comparable to those 7 of the monkeys. We compared these reaction times with those of the correct choice network 8 using the same decision boundary (Figure 2D). The correct choice network was faster than the 9 monkey choice network both after rewarded trials and unrewarded trials. Notably, this 10 difference was greater after unrewarded trials, indicating that task certainty has a greater effect 11 on the time course of choice output activity in the monkey choice network. 12 13 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint FIG 2 : Generating mechanistic hypotheses using recurrent neural networks . (A) Model 1 schematic for RNNs designed to either predict the correct choice (correct choice network) or 2 predict the monkeys ’ choice on each trial (monkey choice network). (B) The difference in 3 perceptual accuracy (after rewarded trials – after unrewarded trials) throughout training for 4 both models. The two monkeys’ difference in perceptual accuracy was on average about 5%, 5 as indicated by the brown dashed line . The monkey choice network approaches this 6 difference, while the correct choice network remains around zero (shaded regions show 95% 7 confidence intervals from sliding window averaging ). (C) The monkey choice network (left) 8 displays a cost of flexibility after training, as shown by worse perceptual performance after 9 an un rewarded, low task certainty trial (location choices: p=2.4×10-14, frequency choices: 10 p=6.6×10-8). In contrast, the correct choice network (right) did not show a significant cost of 11 flexibility (location choices: p=0.43, frequency choices: p=0.14). Each point corresponds to a 12 set of accuracies calculated from 50 trials with identical feature change amounts . (D) The 13 monkey choice network displays a larger effect of task uncertainty on reaction time 14 compared to the correct choice network. The plot depicts histograms of reaction times when 15 the same collapsing decision boundary is applied to the outputs of both RNNs (see Methods). 16 Overall, the monkey choice network has longer reaction times than the correct choice 17 network given the same inputs (after unrewarded trials: p=1.7×10-271, after rewarded trials: 18 p<10-300). The difference between models is greater after unrewarded than after rewarded 19 trials (p= 3.7×10-36, Wilcoxon rank sum test. (E) At the end of the delay period between 20 stimulus presentations (4 00 ms after first stimulus presentation), information about the 21 irrelevant feature (e.g. location on trials when the model chose the spatial frequency task) is 22 encoded better following non-rewarded than rewarded trials (points are above the diagonal 23 for both models; p=1.49×10-8 for both). Each point represents a set of trials with an identical 24 first stimulus and shows the performance of a corresponding cross-validated linear decoder. 25 (F) The two features are represented more orthogonally in the correct choice network than 26 in the monkey choice network at the end of the delay period (400 ms after first stimulus 27 presentation). Histograms of the magnitude of the projection of the irrelevant feature axis 28 onto the relevant feature axis for the monkey choice and correct choice network s after 29 unrewarded and rewarded trials. The unit axis for each feature was computed using distance 30 covariance analysis and scaled by the corresponding distance covariance (see Methods for 31 details). All four distributions are significantly different ( Wilcoxon rank sum tests: monkey 32 choice network after non-reward > correct choice network after non-reward: p=6.1×10-106; 33 monkey choice network after reward > correct choice network after reward: p =3.8×10-142; 34 monkey choice network after reward > after non -reward: p=0.001, correct choice network 35 after non-reward > after reward: p =1.8×10-35). Triangle markers represent the medians. (G) 36 A schematic of how overrepresentation of the irrelevant feature when the task is uncertain 37 (depicted in E) combined with non-orthogonal representations of task variables (depicted in 38 F) leads to interference and thus a cost of flexibility. In an idealized correct choice network 39 (top), the irrelevant feature is more strongly encoded (longer magenta axis) under low task 40 certainty, however, it varies orthogonally to the relevant feature, thus avoiding interference. 41 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint In the monkey choice network (bottom), the feature axes are no longer orthogonal, leading 1 to more interference under low task certainty. 2 3 Task uncertainty leads to task interference 4 The observation that our monkey choice network , but not our correct choice network , 5 demonstrates costs of flexibility gives us a platform for generating hypotheses about the 6 mechanistic origins of this cost. We investigated the dynamics of the two networks and identified 7 two features that could in principle underlie the cost of flexibility. First, consistent with the 8 concepts of attentional and task -set inertia (1, 2, 5 -7, 9, 10, 12) , both models represented the 9 irrelevant feature better under low task certainty than under high task certainty ( orange data 10 points, Figure 2E). When the task was more certain (following rewarded trials) , the relevant 11 feature was encoded throughout the entire delay period, while information about the irrelevant 12 feature decayed nearly completely (Figure S2). This certainty-dependent ‘forgetting’ of irrelevant 13 information occurred even in the correct choice network (blue data points, Figure 2E), which did 14 not show a cost of flexibility, so it is on its own not a sufficient explanation of the cost of flexibility 15 in biological organisms. 16 The second feature of the models is not predicted by common accounts of task switch costs. In 17 the monkey choice network, the representations of the two visual features of the first stimulus 18 become non-orthogonal before the onset of the second stimulus presentation (Figures 2F, S3). 19 This tangling of the representations causes the monkey choice network to mix up information 20 about the two features (e.g. judgments of spatial frequency are influenced by the location of the 21 stimulus), which reduces accuracy. We can visualize the problem by identifying the axes that best 22 represent information about each visual feature in a space in which the activity of each unit in 23 the hidden layer is one dimension. These axes are the directions in population space along which 24 there is most variation in responses to different values of that feature. If information about the 25 two features is encoded independently , the axes best representing them will be orthogonal. 26 However, in the monkey choice network, those axes rotate such that by the end of the delay 27 period the y are no longer orthogonal (Figure s 2F, S3 (B)). This non -orthogonality means that 28 information about the two features is no longer independent, leading to ‘switch errors’ that 29 worsen perceptual performance. In contrast, the correct choice network maintains nearly 30 orthogonal feature axes, especially after unrewarded trials, such that their dot product remains 31 closer to zero (Figures 2F, S3). 32 We hypothesize that the cost of flexibility is the joint product of these two mechanisms (Figure 33 2G). The increased representation of irrelevant information when the task is uncertain is not a 34 problem when information about the two features is encoded independently (as in the correct 35 choice network). The independent representations make it straightforward to ignore irrelevant 36 information, even when it is robustly represented. However, when information about the two 37 features is not independent, as in the monkey choice network , the irrelevant information is 38 confounded with information about the relevant feature, worsening perceptual performance. 39 We tested predictions of this task interference hypothesis using new behavioural, neuronal, and 40 causal experiments designed to identify the types of errors associated with task uncertainty. 41 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint 1 Behavioural evidence for task interference 2 We first looked for behavioural evidence supporting the first part of our task interference 3 hypothesis, the information deemed irrelevant for the chosen task is more strongly retained 4 under low task certainty. On a small subset of trials, we asked human participants to report one 5 of the features (orientation or frequency) of the first stimulus after a delay (Figure 3A). Consistent 6 with the idea that irrelevant information is maintained longer in short -term memory when the 7 task is uncertain, participants had higher accuracy (smaller errors) when estimating the believed-8 irrelevant, but not the relevant, feature on low task certainty trials (Figure 3B). 9 Next, we looked for behavioural evidence supporting the hypothesis that the two features are 10 entangled when the task is uncertain . The monkey choice network predicted that under 11 uncertain task conditions, participants’ choices about one feature (e.g., spatial frequency) are 12 influenced by the other feature (e.g., location). Consistent with this hypothesis, the choices of 13 humans (Figure 3C upper panel) and monkeys (Figure 3C lower panel) were more strongly related 14 to the irrelevant stimulus feature following a non-reward (when the task was relatively uncertain) 15 than a reward (when the task was more certain). 16 17 18 FIG 3: Behavioural evidence for the task interference hypothesis. (A) In a subset of the trials, 19 we asked human participants to reproduce one of the features (spatial frequency or 20 orientation) of the first stimulus according to their memory following a delay. The plot depicts 21 the experimental design, including the slider used to reproduce the stimulus. (B) When 22 participants reported that a feature was behaviourally irrelevant, their recall error was better 23 (smaller) for that feature if the preceding trial had been incorrect (low task certainty, y-axis) 24 than correct (high task certainty , x-axis). The orange data points lie significantly below the 25 identity line (p=0.001), indicating that believed-irrelevant information was reported with 26 smaller errors under low task certainty . In contrast, there was no significant relationship 27 between task certainty and participants’ ability to remember the believed-relevant feature 28 (p>0.05), although the deviations from the identity line are not significantly different 29 between believed relevant and believed irrelevant features (Wilcoxon rank sum test, p>0.05). 30 1 2 1 2 Normalized feature recall error p = 0.1 p = 0.00 1 believed relevant believed irrelevant High task certainty Low task certainty What was the frequency of the first stimulus? 0.5~ 2s de lay Proportion of all monkey sessions / human participantsPerformance difference in predicting choice with irrelevant feature (%) (High task certainty - low task certainty) -250 0.1 0.2 0.3 25 p = 1×10-4 Task uncertain ty boo sts i rrelevant feature in fluence on perceptu al choi ces p = 2×10-14 -30 300 0.1 0.2 0.3 0 effects stronger with lo w task certainty effects stronger with hi gh task certainty B CA .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint (C) Perceptual choices are significantly more predictable from the irrelevant feature under 1 lower task certainty. For each monkey experiment session and human participant, we predict 2 the perceptual choices on one feature based on the changes in the other feature for trial-3 number matched conditions of low and high task certainty. Histograms (top: humans, bottom: 4 monkeys) show that across monkey experiment al sessions / human participants, the 5 distributions of differences in prediction performances under high and low task certainty 6 were significantly below zero (p=1×10-4 for humans, p=2×10-14 for monkeys), indicating that 7 irrelevant feature is more predictive of perceptual choices under task uncertainty. 8 9 Neuronal evidence for task interference 10 To test the predictions of the feature interference hypothesis on neural representations, we 11 recorded groups of neurons in monkey primary visual cortex (V1), which is known to encode the 12 visual features (frequency and location) that the monkeys discriminated. We analyzed the 13 neurons in a similar way as the units in our model, beginning by using linear regression to identify 14 the axes in V1 population activity space that best encode each visual feature. We found evidence 15 for task interference in both signal and noise. To look for entangled representations in the signal, 16 we tried to decode the frequency value from location encoding axis, and vice versa. Consistent 17 with our hypothesis, under low task certainty , the representations of spatial frequency and 18 location were more entangled , as indicated by better decoder performance under low task 19 certainty (Figure 4A). To investigate noise-based evidence for task interference, we investigated 20 the V1 population responses to the same stimulus across trials. The trial-to-trial fluctuations 21 between projections onto the spatial location and frequency axes were more correlated under 22 low task certainty (Figure 4B). 23 24 25 0.2 0.4 0.6 0.2 0.4 0.6 Trial by trial correl ati on between proj ectio ns on to th e location and frequ en cy axes High task certainty Low task certainty p = 1×10-4 0.5 0.8 Performance under high task certainty 0.5 0.8Performance under low task certainty Decoding relevant feature from irrelevant dimension p = 0 .0 03 p = 0 .0 2 V1 Freq . in Loc . dim. Loc . in Freq . dim. monkey G F 0.5 1 No n-micr ost imulat ion tr ia ls 0.5 1 Mic ros timulati on trial s (b ased o n loc at ion sele ct ivit y) Pr opor ti on of fr equency c hoi ce fr om f eat ure int er fer enc e Low t ask cert ainty (p =0.03 ) High ta sk certa inty (p =0.11) Task uncertainty boosts microstim. effects from feature interference p = 0 .0 09 -0.4 0.40 0.1 0.2Proportion of Conditions effects stronger with lo w task certainty effects stronger with hi gh task certainty 0 CA B .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint FIG 4: Neuronal and causal evidence for task interference from neural population recordings 1 and electrical microstimulation in monkey V1. (A) Stronger feature entanglement under low 2 task certainty. For each monkey recording session, we identified the dimensions in V1 3 population space along which we could best linearly decode each visual feature. Using linear 4 decoders, we assessed our ability to decode the relevant feature in the encoding dimension 5 of the irrelevant feature under high (abscissa) and low task certainty (ordinate). Across 6 sessions, for both features, the points are distributed significantly above the identity line 7 (p=0.003 for location classifiers in frequency dimension, p=0.02 for frequency classifiers in 8 location dimension), indicating increased feature entanglement under low task certainty. (B) 9 Projections of V1 population responses onto the spatial frequency and location axes are more 10 correlated under low task certainty, indicating that the two axes are less orthogonal . The 11 abscissa and ordinate of each data point show the absolute value of the correlation 12 coefficient relating projections onto the two axes for trials with identical stimuli under high 13 and low task certainty. We use the absolute value because there is no a priori prediction 14 about the sign of the correlation, only that nonzero correlation indicates non -orthogonality. 15 The mean of the distribution is above the identity line, indicating that decoding of the two 16 features is more correlated under low task certainty (p=1×10-4). (C) Causal tests of the task 17 interference hypothesis using V1 microstimulation. On a subset of randomly interleaved trials, 18 we microstimulated V1 neurons during the display of the second stimulus. Each point in the 19 scatterplot in the upper panel represents a set of trials with identical stimuli , displaying the 20 proportion of a given spatial frequency choices on trials with microstimulation (ordinate) or 21 without microstimulation (abscissa) . If the neuronal population representations of the two 22 features were independent, the points would align along the identity line , while d eviation 23 indicates feature interference. The observation that microstimulation has a bigger impact on 24 behaviour under low task certainty is evidence for task interference (Co mpare 25 microstimulation effects in high certainty: filled grey symbols, not significantly away from 26 identity line, p=0.11; vs. low task certainty: open grey symbols, above identity line, p=0.03 ). 27 Black symbols indicate the means of each task certainty condition. The bottom histogram 28 plots the differences between microstimulation effects under low and high task certainty (i.e. 29 how much the arrows in the scatter moved the data points away from the diagonal) . The 30 distribution across conditions reveals a significantly negative distribution (p=0.009) that 31 confirms stronger feature interference in V1 under low task certainty. 32 33 Causal evidence for task interference 34 Finally, we used a causal manipulation to test the prediction of the task interference hypothesis 35 that the representations of the two features are more entangled under low task certainty. If the 36 representations of the two features are orthogonal, then a causal manipulation that biases 37 judgments of one feature should not impact judgments of the other. If, on the other hand, the 38 representations of the two features are entangled, then biasing judgments of one feature should 39 necessarily impact judgments of the other in the way predicted by the entanglement. 40 We therefore used electrical microstimulation in V1 to bias judgments of one feature (location) 41 and measured the impact on judgments of the other feature (spatial frequency) under different 42 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint task certainty conditions. Thanks to the retinotopic organization of V1, we were able to induce a 1 bias in the perceived location change b y microstimulating V1 during the presentation of the 2 second stimulus. The bias was predicted by the tuning of the stimulated site: when the receptive 3 field recorded on the microstimulated channel was located to the right of the first stimulus, the 4 monkey reported rightward location shift s more frequently, and vice versa (Supplementary 5 Figure S4(A)). On trials when the monkey performed the spatial frequency task, microstimulation 6 (which affects essentially only location judgments; Fig. S4(B)) should not have influenced choices 7 if the representations were orthogonal. Consistent with the task interference hypothesis , 8 microstimulation had a bigger impact on spatial frequency judgments on trials with low than high 9 task certainty (Fig. 4C open symbols significantly deviate more from the diagonal line than filled 10 symbols), suggesting that the representations of location and spatial frequency were less 11 orthogonal under low certainty. 12 13

14 Humans and animals must flexibly switch between tasks to survive in an uncertain world . This 15 flexibility has long been known to come at a cost . The cost of flexibility has been explained by a 16 diverse set of ideas, which are mostly modeled and tested using behaviour (13-20). We aimed to 17 unify and mechanistically explain the extensive body of behavioural literature that has explored 18 this intriguing phenomenon. 19 We used a multidisciplinary approach to identify neural population reasons for the cost 20 associated with switching between tasks. By grounding our investigation in the neural 21 representations of tasks and their associated variables, we uncovered a task interference 22 neuronal mechanism that unifies previous findings and has implications for our understanding of 23 many cognitive processes. 24 Integrative approach reveals neuronal mechanism for complex behaviour 25 In our research, we presented a novel research paradigm combining artificial neural networks, 26 monkey electrophysiology, and human psychophysics , to understand the neuronal mechanism 27 for advanced cognition and complex behaviour. By constraining artificial neural networks using 28 actual behavioural observations, we generate data -driven hypotheses about the neuronal 29 dynamics that underlie the intricacies in real behaviour. Large -scale electrophysiology in 30 behaving monkeys allows for these hypotheses to be empirically tested at a single-neuron level, 31 providing a direct examination of the neural correlates of hypothesized cognitive processes. 32 Human psychophysics complements this, extending the relevance of our findings beyond a single 33 species and ensuring applicability to broader cognitive phenomena. 34 Combined, our approach holds potential to extract meaningful insights from complex data, a 35 critical challenge in an era when datasets become increasingly high dimensional (39). 36 Task interference may impact neural processes throughout the brain 37 Remarkably, we observed neural signatures of task interference even in V1, a primary sensory 38 area not typically associated with strong modulations by cognitive processes (21, 22) . This 39 suggests that the phenomenon of interfering neural representations is not confined to specific 40 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint brain regions but rather extends throughout the brain. Future research should investigate 1 whether task interference poses an even stronger limit in higher-order visual areas and prefrontal 2 cortex, which are more intimately linked with more advanced cognitive processing. 3 Dimensionality as a fundamental limit on cognition 4 Limited capacity is a repeatedly reported constraint on multiple c ognitive processes including 5 working memory (23), attention (24), decision-making (25), and learning (26). Our findings raise 6 the intriguing possibility that the dimensionality of a neural population representation is a 7 fundamental limit on cognition because it limits the behavioural repertoire that the population 8 can enable. The dimensionality limits the number of quantities (e.g. sensory features, number of 9 objects, task variables, timescales, etc.) that can be represented independently. I n theory, the 10 dimensionality of a neural population representation could approach the number of neurons in 11 that population, but the true dimensionality is typically much lower (27, 28). 12 Observations in diverse experimental systems are broadly consistent with the possibility that the 13 dimensionality of neural population representations limits cognitive capacity . For instance, 14 associating multiple simple features (such as the colour and surface texture of a banana) to aid 15 in perceiving a complex feature (such as the ripeness of the banana), might combine, or reduce 16 the dimensionality of, the representations of those associated features (29). However, reversing 17 such learned associations would in this scenario require the brain to increase the dimensionality 18 again, which could account for the challenges in reversal learning in operant conditioning in mice 19 (30), rule learning in children with bipolar disorder (31), and language learning in neurotypical 20 people (32). 21 Similarly, limitations in working memory capacity (23, 33) might be thought of as a limit on the 22 number of items that can be represented independently in a population of neurons. People can 23 overcome those limits by ‘chunking’ the memory of different items into larger, familiar units (e.g., 24 memorizing the digits of a credit card or phone number in groups of three or four (33, 34)). This 25 strategy may effectively reduce the neural dimensions needed to represent and maintain the 26 whole sequence. Interestingly, artificial neural networks exhibit human -like working memory 27 capacity limitations only after being pre -trained on large -scale natural image tasks (35), which 28 may indicate that the mechanisms adopted to process high -dimensional inputs compromise 29 performance on reduced laboratory tasks, perhaps due to an increase in interference between 30 stimuli. The possibility that limitations on diverse cognitive processes share a neuronal 31 mechanism offers a path toward understanding and alleviating those limitations. 32 Translational implications of task interference 33 Deficits in cognitive flexibility are an especially stark consequence of the many brain disorders, 34 including dementia, substance abuse, and developmental disorders , that are associated with 35 diminished cognitive capacity (36, 37) . If task interference indeed underlies many cognitive 36 limitations, our results offer hope for improving and repairing cognitive capacity. For example, 37 we recently demonstrated that methylphenidate (trade name Ritalin, a stimulant used to treat 38 disorders of attention; (38)) effectively increases the dimensionality of neural population 39 representations in the visual cortex and that this increase is correlated with improvement in a 40 visual task (38). This is a proof of principle supporting the idea that repurposing existing drugs to 41 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint address other cognitive abilities that might share a neural mechanism with their primary target 1 could be an effective way to improve cognition in health and disease. The suggestion in our V1 2

that task interference affects brain -wide mechanisms means that systemic drugs and/or 3 therapies that affect widespread neuromodulatory systems might be especially effective. 4 Together, our results hold promising implications for the future of basic science and translational 5 efforts to mechanistically understand, improve, and repair cognition. Our task interference 6 hypothesis provides a concrete set of testable predictions with implications for theoretical and 7 experimental work spanning many species and systems. 8 9

10 Behavioural task for non-human primates 11 We designed a two-interval, two-feature discrimination task with stochastic task switching that 12 allows us to measure and manipulate task certainty and perceptual decision-making (Xue et al, 13 2022). Briefly, two rhesus monkeys (both male, 12 and 9 kg) fixated a central spot while two 14 Gabor stimuli (~2° visual angle in diameter) were displayed in series (200 ms each), separated by 15 a variable delay (300 -500 ms). The second Gabor stimulus differed from the first in its spatial 16 location (shifted left or right) and spatial frequency (higher or lower), with independently 17 randomized change amounts in the two features. Small titrations of the change amounts in the 18 two features were made before each experiment to keep the perceptual performance of the 19 animals in both tasks at roughly 75%. After a 150 ms second delay, the fixation dot disappeared, 20 and the animals looked at one of four peripheral targets to indicate both the inferred relevant 21 feature and the direction of change in that feature (spatial frequency increase or decrease and 22 spatial location left -shift or right -shift). The relevant feature changed stochastically on 2.5% of 23 trials, and the monkeys were rewarded only if they correctly discriminated the relevant feature. 24 The visual display contained no information about the behavioural relevance of features, so the 25 animals needed to infer the relevant feature based on their stimulus, choice, and reward history. 26 The median number of trials in each daily session was 1086. All animal procedures were approved 27 by the Institutional Animal Care and Use Committees of the University of Pittsburgh and Carnegie 28 Mellon University, where the animal experiments were conducted. 29 Behavioural task for the human participants 30 We adapted our task for online human psychophysics (built with Gorilla experiment building tools 31 (40) and customized scripts ). We recruited a total of 220 adult h uman participants through 32 Prolific (prolific.co), who performed slightly different combinations of behavioural experiments. 33 In the main experiments, we used a version of the two-interval, two-feature discrimination task 34 with stochastic uncued task switching that we adapted to better suit human participants and 35 online experiments (Figure 1B). The timing was slightly different (500 ms initial fixation period, 36 300 ms stimulus presentations, and 500 ms delay period) . The two features to be discriminated 37 were orientation and spatial frequency. As in the monkey experiments, t he changes in the two 38 features were independently varied. Similar to the monkeys, t he change amounts were set at 39 each participant’s perceptual threshold, at which the participa nts performed perceptual 40 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint discriminations in both tasks at roughly 75% correct, determined by a psychophysical staircase 1 procedure before the main task session. Participants indicated their choice by clicking on one of 2 four buttons representing each feature and its change direction (indicated by a combination of 3 text and graphical icon) . As in the monkey experiments, the human participants received 4 feedback following each choice (a green tick indicated a correct report of the direction of change 5 in the task -relevant feature, and a red cross indicate d an incorrect perceptual judgment, task 6 choice, or both). As in the monkey experiments, the relevant feature switched stochastically, but 7 the frequency was different (with a probability of 0.1 on each trial for humans, 0.025 for 8 monkeys). The overall length of a typical session varied between 100 to 150 trials and around 15-9 20 minutes. 10 Task-belief and feature perception confidence report 11 Some task sessions required participants to report their confidence in their perceptual judgment 12 on a random 20% of trials. After reporting their decision on the main task but before receiving 13 feedback, participants were presented with three confidence sliders. Participants were 14 instructed to indicate their task-belief on the first slider, ranging from a strong belief that spatial 15 frequency was relevant to a strong belief that orientation was relevant. An intermediate value 16 indicated low task certainty. The second and third sliders were used to report the perceived 17 direction of change for each feature, where the absolute value reflected confidence in the 18 perceptual judgment. 19 Variable delay and feature recall question 20 In some sessions, we explored memory for believed -relevant and believed -irrelevant stimulus 21 features by interrupting a randomly selected 20% of trials after either 500 ms (short delay) or 22 2000 ms (long delay) to ask participants to recall a feature of the first Gabor. Participants 23 indicated the feature memory using a slider, facilitated by four reference Gabor patches that 24 spanned the possible range. We quantified estimation error as the difference between the actual 25 stimulus feature and the recalled feature. 26 RNN models and analyses 27 Training and architecture 28 We trained RNN models using custom code based on PsychRNN (Ehrlich et al., 2021) and 29 TensorFlow (Abadi et al., 2016). The models were governed by the following dynamical equations: 30 𝜏 dx = (-x + Wrec r + brec + Win u) dt + 𝜎rec √2𝜏 d𝜉 31 r = f(x) = max(0, x) 32 z = Wout r + bout 33 where x is the recurrent state, which when passed through the rectified linear unit (ReLU) 34 activation function gives the firing rate r, u is the input vector, and z is the output vector. Win, 35 Wrec, and Wout are the input, recurrent, and output weight matrices, and brec and bout are constant 36 biases added to the recurrent and output units. dt is the discrete time step, 𝜏 is the neuronal time 37 constant, and 𝜎rec scales recurrent noise, which is a Gaussian noise process (d𝜉). 38 Our monkey choice network (Figure 2A) was trained to predict the choices of two macaques 39 based on the same history of stimuli, choices, and rewards experienced by our monkey 40 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint participants. The RNN had two modules, which we term the ‘task module’ and the ‘perception 1 module.’ The two modules were fully connected in the recurrent layer but receive d different 2 inputs and map ped onto different outputs. The task module receive d the history of monkeys’ 3 choices, stimulus feature change amounts (signaling perceptual difficulty), and rewards for nine 4 preceding trials. These inputs were constant throughout the trial, based on the assumption that 5 task rule dynamics are slow compared to single -trial stimulus response dynamics. The task 6 module was trained to output the task that the monkey chose on the current trial as a one -hot 7 vector. The perception module received the spatial frequency (SF) and spatial location (SL) values 8 of the first stimulus and, after a variable delay (300 to 500 ms, uniform distribution) , of the 9 second stimulus. Units in the perception module displayed mixed selectivity for the two features. 10 The perception module was trained to output one of the four choices (SF increase, SF decrease, 11 SL decrease, SL increase) as a one-hot vector. 12 We compare this monkey choice network to a correct choice network (Figure 2A), which was 13 trained to output the correct choice (rather than the monkeys’ choices) given the perceptual 14 inputs on the current trial and its own trial history (feedback based on the model’s choices rather 15 than the monkeys’). Stimulus values were drawn from the same data used to train the monkey 16 choice network. After the models were trained, we used the monkeys’ trial history inputs for 17 both models so that we could directly compare trials (all model figures besides 2B). The correct 18 choice network trace in figure 2B was calculated directly from its training batches (200 trials per 19 epoch), and sliding window averaged over 100 epochs. The monkey choice network trace was 20 calculated post-hoc from weights saved during training (every 15 epochs) , and sliding window 21 averaged over 6 datapoints (90 epochs). We chose to analyze the monkey choice network at the 22 point during training with the highest monkey choice accuracy (see Figure S1), which was at 23 training epoch 1245 out of 1500. All RNNs were trained using the Adam optimizer, with L1 and 24 L2 regularization of weights and L2 regularization of firing rates. Note that to keep the model 25 architecture consistent with the monkey choice network, the correct choice network here has 26 key differences in the inputs from the model we used in our previous study (3). 27 Reaction time calculation 28 To calculate model reaction times, we applied a collapsing decision boundary of the form 29 𝑓(𝑡)  =  𝑎 ⋅ ,1 − !⋅# #$%/ to the choice output units of the RNNs, where 𝑡 is the time after choice 30 period onset. For figure 2D, the parameter values were a=1.5 ms, b=1.8, c=28 ms. 31 Distance covariance analysis for assessing orthogonality of feature axes 32 To quantify the (non-)orthogonality of feature axes in the models, we used distance covariance 33 analysis (DCA), a dimensionality reduction method that identifies linear projections that capture 34 interactions between variables by maximizing the correlational statistic distance covariance (41). 35 After reducing the dimensionality of the activity of all units in the hidden layer to the first 10 36 principal components (capturing >95% of the variance), we calculated t he distance covariance 37 statistic and corresponding axis separately for each unique set of trial history inputs, feature 38 (location and spatial frequency), and timepoint during the delay period, from 500 simulated trials 39 with different feature values. The projection of the irrelevant feature axis onto the relevant 40 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint feature axis was calculated by taking the dot product of the unit vectors and scaling it by the 1 smaller distance covariance (that of the believed-irrelevant feature) (Figures 2F and S3). 2 Electrophysiology for the monkey experiments 3 Different aspects of a subset of the electrophysiological data have been reported previously (Xue 4 et al, 2022). The visual stimuli were displayed on a linearized CRT monitor (1,024 × 768 pixels, 5 120-Hz refresh rate) placed 57 cm from the animal. We monitored eye position using an infrared 6 eye tracker (Eyelink 1000, SR Research) and used custom software (written in MATLAB using the 7 Psychophysics Toolbox, (25) to present stimuli and monitor behaviour. We recorded eye position 8 and pupil diameter (1,000 samples per s), neuronal responses (30,000 samples per s ), and the 9 signal from a photodiode to align neuronal responses to stimulus presentation times (30,000 10 samples per s) using hardware from Ripple. We recorded neuronal activity from chronically 11 implanted Utah arrays (Blackrock Microsystems; 48 electrodes in V1) during daily experimental 12 sessions for several months in each animal ( 90 sessions from monkey F and 68 sessions from 13 monkey G). We set the threshold for each channel at three times the standard deviation and used 14 threshold crossings as the multiunit activity on that unit. We positioned the stimuli to maximize 15 the overlap between potential stimulus locations and the joint receptive fields of V1 units . The 16 receptive fields were measured using separate mapping sessions, during which the monkeys 17 fixated their gaze on a dot, while Gabor stimuli were subsequently flashed at random locations 18 for 100ms each across a portion of the screen. 19 We included experimental sessions for analysis if the total number of completed trials was at 20 least 480 (which was ten times the number of channels on the multielectrode array). During 21 complete trials the monkeys successfully maintained fixation until they indicated their choice). 22 We analyzed the activity of V1 units during stimulus display periods, shifted by 34 ms neuronal 23 response latency (42). V1 units are included for analyses if the stimulus response is 25% larger 24 than baseline activity, measured 100 ms before stimulus onset, and 2) larger than 5 sp/s. These 25 procedures resulted in 89 sessions from Monkey F and 68 sessions from Monkey G; an average 26 46 V1 units, and a median 1086 completed trials per session. Other aspects of these data have 27 been reported in a previous study (3). 28 We did V1 microstimulation experiments during 6 sessions in one monkey (monkey F, Figure 4). 29 We were unable to conduct these experiments in the other monkey because the experiments 30 were interrupted by the COVID shutdown of the lab, after which there were no usable channels 31 on the array. During microstimulation sessions, we electrically stimulated one channel on the V1 32 array with 20, 30 μA, 200 Hz biphasic pulses between 50 and 150 ms after the onset of the second 33 Gabor stimulus on a randomly selected 50% of trials. 34 Microstimulation perturbed the monkey's reported location shift consistently based on the 35 relative locations of the V1 receptive field and the first Gabor location (Supplementary Figure 36 S4(A), but did not effectively manipulate the monkeys reported frequency change based on the 37 preferred frequency relative to the first Gabor frequency (Supplementary Figure S4(B). This 38 discrepancy may be attributed to the fact that microstimulation likely activates neurons through 39 direct axonal activation within a volume of tens of microns in diameter (43). While these 40 activated neurons may exhibit different frequency selectivity than the recorded neurons on the 41 same channel, they likely share similar receptive fields due to V1's retinotopic structure. We 42 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint assessed the level of interference as the degree to which the receptive field location relative to 1 the first Gabor location influenced the monkey’s frequency choices (which should be zero if the 2 representations of frequency and location are orthogonal) . We then compared the level of 3 interference induced by microstimulation following rewarded trials and following unrewarded 4 trials. 5 Statistical tests 6 All p-values reported in this study are from Wilcoxon signed rank tests unless otherwise specified. 7 8 9 Acknowledgments 10 We are grateful to John Maunsell, Douglas Ruff, Ramanujan Srinath, and Pouya Bashivan for 11 comments on this manuscript. This work was supported by the Simons Foundation (Simons 12 Collaboration on the Global Brain award 542961SPI to MRC), the National Eye Institute of the 13 National Institutes of Health (awards R01EY022930, R01EY034723, and RF1NS121913 to MRC). 14 15 Author contributions 16 C.X. and M.R.C. designed research; S.K.M. and C.X. trained recurrent neural nets and analyzed 17 the data; C.X. and L.E.K. performed monkey training and neuronal recording; C.X. analyzed the 18 neuronal data and performed the microstimulation experiments; R.C. and C.X. performed the 19 online human psychophysics experiments and data analyses; M.R.C. supervised the findings of 20 this work; C.X., S.K.M and M.R.C. wrote the paper. 21 22

23 1. Monsell S. Task switching. Trends Cogn Sci. 2003;7(3):134-40. 24 2. Kiesel A, Steinhauser M, Wendt M, Falkenstein M, Jost K, Philipp AM, et al. Control and 25 interference in task switching--a review. Psychol Bull. 2010;136(5):849-74. 26 3. Xue C, Kramer LE, Cohen MR. Dynamic task-belief is an integral part of decision-making. Neuron. 27 2022. 28 4. Koch I, Poljac E, Müller H, Kiesel A. Cognitive structure, flexibility, and plasticity in human 29 multitasking-An integrative review of dual-task and task-switching research. Psychol Bull. 30 2018;144(6):557-83. 31 5. Longman CS, Lavric A, Munteanu C, Monsell S. Attentional inertia and delayed orienting of 32 spatial attention in task-switching. J Exp Psychol Hum Percept Perform. 2014;40(4):1580-602. 33 6. Mayr U, Kuhns D, Rieter M. Eye movements reveal dynamics of task control. J Exp Psychol Gen. 34 2013;142(2):489-509. 35 7. Longman CS, Lavric A, Monsell S. More attention to attention? An eye-tracking investigation of 36 selection of perceptual attributes during a task switch. J Exp Psychol Learn Mem Cogn. 2013;39(4):1142-37 51. 38 8. Pashler H. Dual-task interference in simple tasks: data and theory. Psychol Bull. 39 1994;116(2):220-44. 40 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint 9. Allport A, Wylie G. Task-switching: Positive and negative priming of task-set. 1999. 1 10. Yeung N, Nystrom LE, Aronson JA, Cohen JD. Between-task competition and cognitive control in 2 task switching. J Neurosci. 2006;26(5):1429-38. 3 11. Waszak F, Hommel B, Allport A. Task-switching and long-term priming: role of episodic stimulus-4 task bindings in task-shift costs. Cogn Psychol. 2003;46(4):361-413. 5 12. Allport A, Styles EA, Hsieh S. Shifting Intentional Set: Exploring the Dynamic Control of Tasks. 6 1994. 7 13. Ardid S, Wang XJ. A tweaking principle for executive control: neuronal circuit mechanism for 8 rule-based task switching and conflict resolution. J Neurosci. 2013;33(50):19504-17. 9 14. Driscoll L, Shenoy K, Sussillo D. Flexible multitask computation in recurrent networks utilizes 10 shared dynamical motifs. bioRxiv. 2022:2022.08.15.503870. 11 15. Yang GR, Joglekar MR, Song HF, Newsome WT, Wang XJ. Task representations in neural 12 networks trained to perform many cognitive tasks. Nat Neurosci. 2019;22(2):297-306. 13 16. Mante V, Sussillo D, Shenoy KV, Newsome WT. Context-dependent computation by recurrent 14 dynamics in prefrontal cortex. Nature. 2013;503(7474):78-84. 15 17. Flesch T, Juechems K, Dumbalska T, Saxe A, Summerfield C. Orthogonal representations for 16 robust context-dependent task performance in brains and neural networks. Neuron. 2022;110(24):4212-17 9. 18 18. Riveland R, Pouget A. Generalization in Sensorimotor Networks Configured with Natural 19 Language Instructions. bioRxiv. 2023:2022.02.22.481293. 20 19. Deco G, Rolls ET. Attention and working memory: a dynamical model of neuronal activity in the 21 prefrontal cortex. Eur J Neurosci. 2003;18(8):2374-90. 22 20. Duan CA, Pagan M, Piet AT, Kopec CD, Akrami A, Riordan AJ, et al. Collicular circuits for flexible 23 sensorimotor routing. Nat Neurosci. 2021;24(8):1110-20. 24 21. Luck SJ, Chelazzi L, Hillyard SA, Desimone R. Neural mechanisms of spatial selective attention in 25 areas V1, V2, and V4 of macaque visual cortex. J Neurophysiol. 1997;77(1):24-42. 26 22. McAdams CJ, Reid RC. Attention modulates the responses of simple cells in monkey primary 27 visual cortex. J Neurosci. 2005;25(47):11023-33. 28 23. Cowan N. Working memory capacity: Classic edition: Psychology press; 2016. 29 24. Pylyshyn ZW, Storm RW. Tracking multiple independent targets: evidence for a parallel tracking 30 mechanism. Spat Vis. 1988;3(3):179-97. 31 25. Kang YH, Löffler A, Jeurissen D, Zylberberg A, Wolpert DM, Shadlen MN. Multiple decisions 32 about one object involve parallel sensory acquisition but time-multiplexed evidence incorporation. 33 bioRxiv. 2020:2020.10.15.341008. 34 26. Van Merrienboer JJ, Sweller J. Cognitive load theory and complex learning: Recent 35 developments and future directions. Educational psychology review. 2005;17:147-77. 36 27. Saxena S, Cunningham JP. Towards the neural population doctrine. Curr Opin Neurobiol. 37 2019;55:103-11. 38 28. Chung S, Abbott LF. Neural population geometry: An approach for understanding biological and 39 artificial neural networks. Curr Opin Neurobiol. 2021;70:137-44. 40 29. Libby A, Buschman TJ. Rotational dynamics reduce interference between sensory and memory 41 representations. Nat Neurosci. 2021;24(5):715-26. 42 30. Meister M. Learning, fast and slow. Curr Opin Neurobiol. 2022;75:102555. 43 31. Wegbreit E, Cushman GK, Weissman AB, Bojanek E, Kim KL, Leibenluft E, et al. Reversal-learning 44 deficits in childhood-onset bipolar disorder across the transition from childhood to young adulthood. J 45 Affect Disord. 2016;203:46-54. 46 32. Idemaru K, Holt LL. Word recognition reflects dimension-based statistical learning. J Exp Psychol 47 Hum Percept Perform. 2011;37(6):1939-56. 48 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint 33. MILLER GA. The magical number seven plus or minus two: some limits on our capacity for 1 processing information. Psychol Rev. 1956;63(2):81-97. 2 34. Thalmann M, Souza AS, Oberauer K. How does chunking help working memory? J Exp Psychol 3 Learn Mem Cogn. 2019;45(1):37-55. 4 35. Xie Y, Duan Y, Cheng A, Jiang P, Cueva CJ, Yang GR. Natural constraints explain working memory 5 capacity limitations in sensory-cognitive models. bioRxiv. 2023:2023.03.30.534982. 6 36. Ferreira CS, Maraver MJ, Hanslmayr S, Bajo T. Theta oscillations show impaired interference 7 detection in older adults during selective memory retrieval. Sci Rep. 2019;9(1):9977. 8 37. Dajani DR, Uddin LQ. Demystifying cognitive flexibility: Implications for clinical and 9 developmental neuroscience. Trends Neurosci. 2015;38(9):571-8. 10 38. Ni AM, Bowes BS, Ruff DA, Cohen MR. Methylphenidate as a causal test of translational and 11 basic neural coding hypotheses. Proc Natl Acad Sci U S A. 2022;119(17):e2120529119. 12 39. Fetsch CR. The importance of task design and behavioural control for understanding the neural 13 basis of cognitive functions. Curr Opin Neurobiol. 2016;37:16-22. 14 40. Anwyl-Irvine AL, Massonnié J, Flitton A, Kirkham N, Evershed JK. Gorilla in our midst: An online 15 behavioural experiment builder. Behav Res Methods. 2020;52(1):388-407. 16 41. Cowley B, Semedo J, Zandvakili A, Smith M, Kohn A, Yu B. Distance Covariance Analysis. In: Aarti 17 S, Jerry Z, editors. Proceedings of the 20th International Conference on Artificial Intelligence and 18 Statistics; Proceedings of Machine Learning Research: PMLR; 2017. p. 242--51. 19 42. Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S, Schall JD, et al. Signal timing 20 across the macaque visual system. J Neurophysiol. 1998;79(6):3272-8. 21 43. Histed MH, Bonin V, Reid RC. Direct activation of sparse, distributed populations of cortical 22 neurons by electrical microstimulation. Neuron. 2009;63(4):508-22. 23 24 Supplementary Figures 25 26 27 FIG S1: Accuracies of the monkey choice network (orange) and correct choice network (blue) 28 after training. Perceptual: percentage of correct perceptual judgments, regardless of the 29 chosen task (chance=50%); Correct choice: percentage of correct choices (chance=25%); 30 Correct task: percentage of correct task choices, regardless of the perceptual judgment 31 (chance=50%); Monkey choice: percentage of choices that match the monkeys’ choices 32 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint (chance=25%). Monkey task: percentage of task choices that match the monkeys’ chosen task, 1 regardless of the perceptual judgment (chance=50%). All differences between models are 2 statistically significant (two-sided z-tests, p<0.0005). 3 4 5 FIG S2: Feature (location, top row, and frequency, bottom row) decoder accuracies during a 500 6 ms interstimulus delay period for the monkey choice network (left column) and correct choice 7 network (right column) in four trial conditions. The conditions are as follows: 1) after a 8 rewarded trial, location task chosen (high location task certainty); 2) after an unrewarded trial, 9 location task chosen (low location task certainty); 3) after an unrewarded trial, frequency task 10 chosen (low frequency task certainty); 4) after a rewarded trial, frequency task chosen (high 11 frequency task certainty). In both models, the more a feature is believed to be irrelevant, the 12 more quickly information about that feature decays following stimulus offset. Each point 13 represents the mean performance of cross-validated linear decoders (same as Figure 2E), with 14 error bars showing +/- 1 standard deviation. A separate decoder was trained for each stimulus 15 condition (identical first stimulus) and timepoint. 16 17 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint 1 FIG S3: Feature axes become less orthogonal following stimulus offset in the monkey choice 2 network but not the correct choice network. Orthogonality is quantified by calculating the 3 projection of the irrelevant feature axis onto the relevant feature axis. (A) At the beginning of 4 the interstimulus delay period (0 ms after first stimulus offset), the monkey choice network 5 represents the two features (location and frequency) as orthogonally as the correct choice 6 network both after unrewarded trials (p=0.16) and after rewarded trials (p=0.67, Wilcoxon rank 7 sum tests). The median projections (triangle markers) are all below 0.02. This contrasts with 8 Figure 2F, which shows the same histograms at the end of the delay period (400 ms after first 9 stimulus offset). (B) A violin plot showing the time course of projections during a 500 ms 10 interstimulus delay period. Points represent the medians of the projection distributions; shaded 11 regions show kernel density estimates. Feature axes exhibit a partial ‘collapse’ in the monkey 12 choice network, but not the correct choice network, such that the projection of the irrelevant 13 feature axis onto the relevant feature axis increases significantly. 14 15 16 17 FIG S4: Electrical microstimulation induced location choice bias predicted by the receptive 18 field (RF) of nearby recorded neuron (A) but did not significantly induce frequency choice bias 19 predicted by the frequency selectivity of nearby recorded neuron (B) . We selectively 20 microstimulated V1 neurons during the second stimulus to observe the impact of 21 microstimulation on behavioural choices. Each point in the scatterplot represents a stimulus 22 condition. (A) In trials where the RF of the neuron found at the microstimulated site is to the 23 left side of the first stimulus, then we define location left-shift choice as the “microstimulation 24 favored location choice”; and vice versa for the location right-shift choice with RF to the right 25 side of the first stimulus. The scatter plot displays the behavioural effects of microstimulation 26 0 0.2 0.4 0.6 0.8 Non-microstimulation trials 0 0.2 0.4 0.6 0.8Microstimulation Proportion of microstimulation- favored lo catio n cho ice p = 0 .0 19 p = 0 .2 0 0.2 0.4 0.6 0.8 Non-microstimulation trials 0 0.2 0.4 0.6 0.8Microstimulation Proportion of microstimulation- favored freq uency ch oiceBA .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint on location choices by plotting the proportion of the microstimulation favored location 1 choices with microstimulation (ordinate of the points) against that without microstimulation 2 (abscissa of the point s). The dots lie significantly above the diagonal line, showing that 3 microstimulation biased the monkey’s location choices predicted by the RF location of nearby 4 recorded cells. (B) Similar figure convention to (A), except it shows the absence of a 5 systematic behavioural frequency choice bias predicted by the frequency selectivity of nearby 6 recorded cells 7 8 .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted March 6, 2024. ; https://doi.org/10.1101/2024.03.04.583375doi: bioRxiv preprint