GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk

GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk Eva-Madeleine Schmidt, Sara Bonati, Nils Köbis, Ivan Soraperra This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4611495/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 27 Sep, 2024 Read the published version in Scientific Reports → Version 1 posted 14 You are reading this latest preprint version Abstract Pre-trained large language models (LLMs) have garnered significant attention for their ability to generate human-like text and responses across various domains. This study delves into the social and strategic behavior of the commonly used LLM GPT-3.5 by investigating its suggestions in well-established behavioral economics paradigms. Specifically, we focus on social preferences, including altruism, reciprocity, and fairness, in the context of two classic economic games: the Dictator Game (DG) and the Ultimatum Game (UG). Our research aims to answer three overarching questions: ( 1 ) To what extent do GPT-3.5 suggestions reflect human social preferences? ( 2 ) How do socio-demographic features of the advisee and ( 3 ) technical parameters of the model influence the suggestions of GPT-3.5? We present detailed empirical evidence from extensive experiments with GPT-3.5, analyzing its responses to various game scenarios while manipulating the demographics of the advisee and the model temperature. Our findings reveal that, in the DG, model suggestions are more altruistic than in humans. We further show that it also picks up on more subtle aspects of human social preferences: fairness and reciprocity. This research contributes to the ongoing exploration of AI-driven systems' alignment with human behavior and social norms, providing valuable insights into the behavior of pre-trained LLMs and their implications for human-AI interactions. Additionally, our study offers a methodological benchmark for future research examining human-like characteristics and behaviors in language models. Physical sciences/Mathematics and computing/Computer science Biological sciences/Psychology Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Novel technical architectures such as transformers and extensive text corpora as training data have enabled recent breakthroughs in Natural language processing (NLP). With the advent of ChatGPT, this technological progress quickly became accessible to millions of users. By now, people can easily make use of multiple pre-trained large language models (pre-trained LLMs) to generate suggestions for various everyday activities, ranging from computer coding, automatic translations, and a plethora of writing tasks 1–3 . In view of the growing use of LLMs for everyday advice, concerns arise regarding their response patterns and their consequent impact on human behavior. Underlying these concerns: humans often follow AI advice 4,5 , even if it encourages people to break ethical rules 6 . The classic approach to understanding the output of computer models has been to take a look “under the hood” of the machine. However, this approach does not work with LLMs. The technical design of such models is often intransparent and highly complex, and therefore, generative AI models produce more unpredictable outputs 7 . One recent approach to gaining a better understanding of the performance of LLMs is observing their behavior in controlled experiments akin to how social scientists observe humans' behavior 8 . Thus, a growing number of studies have started to systematically probe LLMs' responses to different prompts 9,10 . For instance, socio-demographic prompting entails systematically sending multiple prompts with minor changes along socio-demographic features in the text 11 . Such studies have revealed varying degrees of resemblance to human-like behavior across domains such as ethical norms, logical reasoning tasks, personality facets, and moral judgments, highlighting similarities and divergences in LLM behavior compared to human responses 12–14 . For instance, several studies have sought to understand the political colorings of LLMs. One paper indicates that early versions of ChatGPT show a green-left-leaning political bias when prompted to answer questions about politics 15 . More recent studies document racial biases in AI advice 16 . Namely, some LLMs systematically suggest less prestigious jobs when prompts are written in African-American English, and defendants described in this dialect spoken by millions of Americans were more likely to receive a death penalty recommendation by an LLM compared to prompts written in “standard” English 16 . An additional concern arises from research showing that people by no means ignore AI-generated suggestions and advice but are, in fact, often altering their views, beliefs, and behavior based on it 17 . While AI advice has received attention in political and ethical domains, less is known about more fundamental social tendencies of LLMs. As inherently social beings, humans face countless social situations in their daily life 18–20 . Frequently, such situations present trade-offs between conflicting goals and norms. For instance, whether one should sacrifice one's own resources to help others or how much one should trust unknown others. Especially when people’s interests and motivations clash, they seek advice to decide what to do. Here, LLMs increasingly become trusted advisors 1,21 . AI advisors have the advantage of always being available and quick to offer advice for any given query. A systematic investigation of the type of advice LLMs give in such social situations is lacking. Therefore, whether LLMs mimic humans' notions of other-regarding concerns remains largely unknown. To gain insights into the advice provided by LLMs for some of the most basic social human behaviors, we draw on economic games. Instead of assessing what people say they would do , these economic games allow us to observe what people actually do when facing a decision with financial consequences for themselves and others. Therefore, economic games offer a clear and straightforward measure of how people trade off their and others' well-being in different social situations 22 . Due to this wide popularity of gauging human social behavior using simple decisions, economic games are also becoming an increasingly popular tool for studying machine behavior 23–27 . A recent line of research has used systematic prompting to understand LLMs' social preferences better. One study has used natural language descriptions of altruism and selfishness to introduce human-like behavior of LLMs in economic games 28,29 . Besides investigating the capacity of GPT-3.5 to manifest human-like social preferences, others have studied the preferences already incorporated in the model. Johnson & Obradovich 25 let GPT-3.5 play a Dictator Game, i.e., a task in which a decision-maker has to divide a financial resource between themselves and a passive recipient. The results show that GPT-3.5 allocates as much money to human partners as humans do, resembling behavior consistent with human altruism. Brookins and DeBacker 28 study LLM's behavior in the Dictator Game and prisoner dilemma. The latter describes a canonical task to assess cooperation. They argue that GPT-3.5 replicates human tendencies towards fairness and cooperation. The model exhibited elevated levels of altruism and cooperation in the Dictator Game and Prisoners’ Dilemma, respectively. Our paper adds to this line of research in two main ways. First, we assess the LLM's strategic and reciprocal tendencies when giving advice by using two economic games: the Dictator Game (DG) and the Ultimatum Game (UG). Both games are two-person games with a decision-maker deciding how to share an amount of money with a recipient. While the DG reflects a non-strategic decision as the recipient is merely passive, the UG adds a strategic component to this social situation. Namely, here, the sender proposes a division of the resources that the responder can either accept or reject, leaving both players with nothing. Therefore, we introduce the novel approach to compare LLM's behavior across two structurally similar but strategically distinctive games, allowing us to disentangle the model's sensitivity for the fairness of the outcomes and intentions. This way, we gain novel insights into whether LLMs social preferences are sensitive to intentions. Second, we manipulate the prompt and the technical details of the LLM to assess the robustness of the advice. On the one hand, we compare the LLM's suggestions for different demographics by manipulating the age and gender of the person receiving the advice. This approach allows us to explore whether suggestions are homogenous or different people would receive different advice. On the other hand, we manipulate the technical parameter of LLM's temperature. This parameter is crucial in determining the randomness and creativity of the LLM's output. A lower temperature leads to more deterministic and potentially repetitive responses, as the model prefers the most likely following words. Conversely, a higher temperature encourages the model to explore less likely options, injecting diversity and creativity into the responses. Since manipulations of temperature levels can produce "personality changes" of the model 29 , it is important to assess its impact on advices. While a rich collection of behavioral studies has documented how humans behave in economic games, research on LLMs' behavior in such tasks is in its infancy. Many behavioral studies have examined how people behave in the canonical games DG and the UG, allowing the aggregation of empirical insights in large meta-analyses (see meta-analyses on DG 30 and UG 31 ). A diverse overview exists of how demographic factors, such as age 30,32 and gender, shape behavior in economic games (DG 30 ; UG 33–35 ). Likewise no clear relationship between prompted gender and behavior in LLMs appears to exist – at least in terms of responses to the personality inventory 35 . However, evidence on how socio-demographic prompting of these features influences LLMs' advice remains scarce. Research Questions and Identification Strategy. Specifically, we aimed to answer three main research questions that we pre-registered on As.predicted (see https://aspredicted.org/RXX_NM8 ). The first question (Q1) assesses whether GPT-3.5 suggestions are sensitive to strategic considerations, thus following a human-like pattern, or are purely motivated by altruism. To test this notion of strategic sensitivity, we prompted GPT-3.5 to advise senders in the DG and UG on how much of their initial endowment they should send to a recipient (see more details in Methods). To test whether the LLM suggestions are sensitive to the risk of being rejected, we can compare the average amount suggested by the LLM in the UG and in the DG, where the risk is absent. If the amounts suggested in the UG significantly exceed those in the DG, the LLM reflects this strategic consideration in its advice. The second question (Q2) examines whether GPT-3.5 suggestions incorporate the concept of positive reciprocity, i.e., if GPT suggestions reward a previous kind action of the counterpart. To investigate whether the model suggests a generous response to a kind action, we prompted the GPT-3.5 with a scenario describing a previous interaction in which the receiver in the DG made a monetary gift to the sender. If suggestions are sensitive to positive reciprocity, the amounts suggested should be increasing in the size of the gift, and eventually higher than in the standard version of the DG where the kind action is not present. The third question (Q3) tests whether GPT-3.5’s suggestions incorporate costly punishment of bad intentions. Specifically, we explore the influence of negative reciprocity on the suggestion to reject unfair offers by studying whether suggestions to reject stem from the unfairness of the outcomes alone or involve a retaliatory response to unfair behavior. To do so, we compare the rejection suggestions made to a responder facing an unfair offer in the UG — for instance, receiving only 10% of the total amount while the sender retains 90% — with the suggestions given to an individual presented with an equivalent binary choice outside the UG context — i.e., the suggestion of what to choose between the option of keeping 10% of an amount of money for yourself and giving 90% to another person and the option of both players obtaining nothing. This design allows us to keep the (un)fairness of the outcomes constant while manipulating the way such (un)fairness originates. If suggestions are sensitive to outcome fairness, we expect to observe rejection suggestions to increase with the inequality of the final distribution. Moreover, if suggestions are driven by negative reciprocity, we expect to observe more suggestions to reject when the decision follows from an unfair proposal. Methods Technical Details. We used Python (version 3.10.8) to query the pre-trained OpenAI text-davinci-003 LLM (GPT-3.5) through the OpenAI API and to perform data pre-processing. We conducted the data analysis and visualization in R (version 4.3.1). Design The tasks. To answer our research question we employ the two games mentioned above: the Dictator Game (DG) and the Ultimatum Game (UG). The DG is a two-person game where a person (the dictator) unilaterally decides how much money to allocate to another person (the recipient). The DG is often used to study altruism and reciprocity 27,30,36 . The UG is also a two-person game where a person (the proposer) proposes a division of resources to a second person (the responder) who decides whether to accept or reject the offer. The UG is often used to study strategic negotiation and fairness considerations 37–39 . In this paper, we use the following variations of the games: Dictator Game (DG) : The dictator is given 10 Euros and can divide this money as they please between themselves and the recipient, who is passive and cannot make any decision. Dictator Game with reciprocity (DG-R) : The same as the DG, with the only difference that previous to the current interaction, the recipient gifted the dictator an amount of money that can take the following values: 1, 2, 3, 4, 5, and 10 euros. Ultimatum Game (UG) : The proposer is given 10 Euros and can propose a division of this money between themselves and the responder. In this case, the responder can choose whether to accept the proposal, obtaining the proposed share for themselves and leaving the rest of the money to the proposer, or to reject the proposal, leaving both of them with nothing. Binary Dictator Game (B-DG) : A variation of the previously described DG, where the choice of the dictator is restricted to two possible options: zero euros for both players or a specific split of the 10 euros. This game serves as the benchmark comparison for the responder's rejection rates in the UG. Prompts of the games . We designed game-specific prompts closely resembling human participant study descriptions to ensure comparability. GPT-3.5 was not portrayed as an independent agent; instead, prompts were used to seek model-generated suggestions for player behavior. This choice stemmed from two considerations: (a) the model cannot answer for itself, as its training data does not encompass knowledge about the qualities of GPT-3.5, a limitation previously highlighted 40 , and (b) asking for behavioral suggestions aligns with potential real-world use cases that could affect human behavior, as discussed above. We provide the exact prompts used in this study in the supplementary material (see supplementary material, Material B). To ensure that recommendations are based on a formally correct understanding of the games, allowing for a meaningful interpretation by humans, we asked the model to answer comprehension questions similar to those typically used for human participants in behavioral experiments (see supplementary material, Material C). We queried GPT-3.5 using prompts describing these tasks from the perspective of one of the players who is asking for suggestions on how to behave and asked to provide answers in a structured json format (see supplementary material, Material B). Specifically, we submitted prompts where: Dictators in the DG and proposers in the UG ask for suggestions about the amount X to send to the other player (X from 0 to 10); Responders in the UG ask for suggestions on whether to accept or reject an offer of Y euros (with Y in {1, 2, 3, 4, 5, 10}); Dictators in the DG with reciprocity ask for suggestions about the amount X to send to the recipient (X from 0 to 10) given that the recipient gifted them an amount Y before the interaction (with Y in {1, 2, 3, 4, 5, 10}); Dictators in the binary DG ask for suggestions on what to choose between the option to keep Y euros for them and give 10 - Y euros to the recipient (with Y in {1, 2, 3, 4, 5, 10}) and the option of both players obtaining zero euros. In Fig. 1, we illustrate how the replies to these prompts map onto and allow us to answer each research question. The supplementary material presents a more detailed overview of the experimental design (see supplementary material, Material A). Figure 1 Subordinated Questions to Research Question 1 (To what extent do GPT-3.5 suggestions reflect human social preferences?) and Experimental Design Note Q1 = Question 1, Q2 = Question 2, Q3 = Question 3, DG = Dictator Game, UG = Ultimatum Game. Prompt Manipulations. To assess robustness of the advices, we systematically manipulated the temperature of the language model, which is related to the creativity of the replies, and the demographic features of the person asking for suggestions. As for the temperature, we considered three levels: below default (temperature = 0.5); OpenAI default (temperature = 1); and above default (temperature = 1.5). As for the features of the person asking for advice, we included demographic information to some versions of the prompt. We systematically manipulated the demographics along two factors: the age of the person receiving the suggestion (with three levels: 18–30 yrs, 31–50 yrs, and 51–70 yrs) and the gender of the person receiving the suggestion (with three levels: female, male, and non-binary). For each prompt, we collected 1000 replies for each temperature level without including demographic information ( unprompted demographics ) and 1000 replies for each temperature level and combination of the demographic factors (see supplementary material, Material D for a more detailed breakdown of the variations). In the pre-registration, we planned to collect 10 responses for each prompt variation using a temperature of 0 to assess the model responses when LLM predictions are deterministic. We failed to collect such responses due to a coding mistake, and by the time we realized that such data was missing, OpenAI already updated the model. We therefore decided not to collect such information. Analysis & Results The pre-processing steps are detailed in the supplementary material (Material E). The pre-registration of the analyses can be found under the following link: https://aspredicted.org/RXX_NM8 . For all the questions, we performed separate tests for each temperature level and a single test collapsing the temperature. The main analysis collapsed the answers for the different levels of demographics. To additionally test whether and how age and gender influence the responses of the model we plotted the dependent variables grouped by age and gender for each question and conducting subsequent regression analyses. The analysis included 599,244 valid responses out of 600,000 prompts submitted (see supplementary material, Material F for an overview of the number of observations per game and temperature). Invalid responses are the ones for which the model either provided a value in the json response that did not comply with the type requested or it did not provide a value in the json response and it was not possible to infer the value from the text (see supplementary material, Material E for the details). Q1: Are GPT altruistic suggestions sensitive to strategic considerations? To answer (Q1) whether altruistic suggestions are sensitive to strategic considerations or purely driven by altruism, we compared the suggested amount sent in the DG and the suggested amount sent in the UG. Figure 2 shows the average suggestion in the two cases separated by temperature level. Note Average suggestions of GPT-3.5 in the Dictator Game (DG) were higher than those in the Ultimatum Game (UG). The p -values associated with each comparison are indicated as follows: * p < 0.05, ** p < 0.01, *** p < 0.001. When pooling responses across the different levels of the demographic and temperature factors, we observe considerably more altruistic suggestions in DG Sender ( M = 6.44, SD = 1.57) than in UG Sender ( M = 4.13, SD = 1.55), t (59973) = 180.77, p < .001, Cohen's d = 1.47 (95% CI [1.46; 1.49]) and results are consistent across different temperature levels (see Table 1 ). This result indicates that the model’s suggestions do not reflect human behavior when a risk of rejection exists. Contrary to typical results in the behavioral science literature, we observe substantially lower suggested amounts to send when the risk is present than when the risk is not present 41 . Specifically, the amounts suggested by the LLM in the DG are substantially more generous than those of human participants. The amounts suggested by the LLM in the UG are slightly lower than in behavioral experiments with human participants. When looking at the effect of the demographic features of the person receiving the advice, we find robust effects across age and gender. In all cases, the model suggestions are less generous when risk is not present (see supplementary material, Material G). The effect size is also comparable across demographics, except for older advisees, i.e., age 51–70, who receive more extreme advice than the other age groups. Table 1 Results of Welch Two Sample t-test DG Sender UG Sender CI M SD M SD p lower upper All 6.44 1.57 4.13 1.55 t (59973) = 180.77 < .001 2.28 2.33 Temp = 0.5 6.57 1.38 3.84 0.99 t ( 18169 ) = 160.89 < .001 2.70 2.76 Temp = 1.0 6.45 1.52 4.16 1.58 t (19962) = 104.32 < .001 2.24 2.33 Temp = 1.5 6.30 1.77 4.40 1.90 t (19885) = 73.01 < .001 1.85 1.95 Note. Suggestions of GPT-3.5 in the Dictator Game (DG) were higher than those in the Ultimatum Game (UG) across all temperature levels. When compared to human behavior, we find more altruistic suggestions in the DG, suggesting that the model's recommendations do not align with human behavior when there is a risk of rejection. Q2: Do GPT suggestions incorporate positive reciprocity? To answer (Q2) whether the model’s suggestions reward the kind actions of the counterpart, we look at the suggested amount in the DG Reciprocity conditions. We assess reciprocity in two ways: first, we look at whether suggestions are increasing in the size of the gift received from the counterpart, and second, we compare these suggestions to the suggested amount in the DG Sender, where the kind action is absent. Figure 3 shows the average amount sent in the DG Sender and the DG Reciprocity for the different kindness levels, i.e., 1, 2, 3, 4, 5, and 10 euros. The figure shows that (i) suggestions are indeed increasing with the kindness of the other player's action, and (ii) suggestions in the presence of a kind action are more generous than in the absence of a kind action only when kindness is substantial. Note Suggestions increase with the kindness of the other player's action (DG-1 to DG-10). Suggestions in the presence of a kind action are more generous than in the absence of a kind action (DG Sender) only when kindness is substantial (DG-10). DG = Dictator Game, DG- n refer to DG Reciprocity with n depicting the amount of money a receiver previously gifted to the dictator. Linear regressions with the suggested amount as the dependent variable and with both the kindness of the action and a dummy variable capturing the DG Sender as the explanatory variables statistically support these observations (see Table 2 ). The estimated parameter for the “Kindness” variable is positive, indicating that suggestions increase with the amount received from the counterpart. Moreover, the estimated parameter for the variable “DG Sender” is significantly greater than zero, indicating that suggestions are higher when kindness is absent compared to when kindness is present. Overall, the results support the existence of positive reciprocity in the LLM’s suggestions as suggested generosity is increasing with the kindness of the other participant. However, we find contradictory results when comparing these suggestions to those without a generous action. As the figure and the regressions show, these results are robust across temperature levels of the model. When looking at robustness across demographic, we observe consistent patterns for all the demographic features (see supplementary material, Material H). Table 2 OLS regression with Amount Sent as the Dependent Variable All Temp = 0.5 Temp = 1.0 Temp = 1.5 Kindness 0.448*** (0.001) 0.517*** (0.002) 0.450*** (0.002) 0.376*** (0.003) DG Sender 1.734*** (0.012) 2.031*** (0.017) 1.763*** (0.021) 1.407*** (0.025) Constant 4.704*** (0.007) 4.539*** (0.010) 4.684*** (0.012) 4.890*** (0.014) Observations 209,879 70,000 69,996 69,883 R² 0.331 0.517 0.344 0.201 Adjusted R² 0.331 0.517 0.344 0.201 Note. In DG with reciprocity, suggestions increase with the amount received from the counterpart. Suggestions are higher when kindness is absent (DG Sender) compared to when kindness is present. The results support the existence of positive reciprocity in the LLM’s suggestions, but we find contradictory results when comparing these suggestions to those without a generous action. DG Sender = dummy variable capturing suggestions in the Dictator Game, Kindness = continuous variables capturing the kindness of the other participant action. *p < 0.05; **p < 0.01; ***p < 0.001 Q3: Do GPT suggestions incorporate costly punishment for unfair actions? To assess the influence of negative reciprocity on the suggestion to reject unfair offers (Q3), we compare GPT suggestions in UG Reciever and in the DG Binary. Figure 4 shows the rate of rejection suggestion for different allocations in the two cases, split by the temperature of the model. The figure highlights various patterns. First, suggestions to reject unfair allocations in binary DG are virtually non-existent for all temperature levels. This result implies that GPT suggestions are not averse to outcome inequality, even when inequality is substantial (1 euro for the decision maker vs 9 euros for the other player). Second, suggestions to reject unfair proposals in UG strongly correlate with the model's temperature. Similar to binary DG, suggestions to reject are virtually non-existent at low temperature levels. For higher temperature levels, however, the model starts to suggest rejections of unfair outcomes proposed by the other player. Third, such suggestions are weakly increasing with the level of unfairness. Overall, these results indicate that “cold” GPT models (low temperature) make very rational and self-interested suggestions. Conversely, “warmer” models (high temperature) suggest retaliatory punishment for unfair actions. While these patterns align with human emotional reactions to intentions, rejection rates seldomly exceed 10%. This result is in notable contrast with human behavior, where, on average, approximately 16% of UG offers are rejected 42 . Note DG = Dictator game, UG = Ultimatum game. The ordered pairs of values on the x-axis indicate the split of the 10 euros, with the first value representing the amount for the dictator / proposer and the second value representing the amount for the recipient / responder. These observations receive statistical support from the regression results in Table 3 . The table reports the results of linear probability models with a dummy variable indicating the rejection, i.e., the choice of the option that gives both agents zero euros, as the dependent variable. We included the variable “Fairness”, capturing the amount for the decision maker; the indicator variable “UG Receiver”, capturing choices made in the Ultimatum Game context; and their interaction, as explanatory variables. Estimated parameters in the Table are scaled to show the effects in percentage points. Regression model (2; temperature = 0.5) shows that rejections are virtually non-existent and do not change substantially with the fairness of the outcome and with the intention — e.g., UG Receiver increases rejection suggestions of the most unfair offer by only one-quarter of a percentage point. Models (3; temperature = 1) and (4; temperature = 1.5) show substantially higher suggestions to reject when unfairness is generated by another person’s decision (the likelihood of rejection of the most unfair offer is 3.9 and 9.5 percentage points higher in the context of the UG for the medium and high temperature, respectively) and a strong relationship between rejection suggestions and the fairness of the allocation in the UG context (Compared to the DG context, for the medium and high-temperature suggestions to reject in the UG decrease by 0.8 and 1.8 percentage point for each extra euro allocated to the decision-maker). When looking at these patterns across different demographic features of the person receiving the suggestion, we find qualitatively robust results (see supplementary material, Material I). At the same time, some differences in the intensity of negative reciprocity emerge, with rejection suggestions decreasing with age and slightly higher for the non-binary gender category. We further repeated the regression using logit and probit models (see supplementary material, Material J). In contrast to the OLS regression, these show that fairness increases the likelihood of rejection and that UG Receiver variable decreases rejection likelihood. This indicates that the relationship between fairness, the UG context, and rejection behavior is complex and that results should be interpreted with caution. Table 3 OLS Regressions with Rejection Rates as the Dependent Variable All Temp = 0.5 Temp = 1.0 Temp = 1.5 Fairness -0.039*** (0.006) 0.000 (0.000) -0.010* (0.004) -0.107*** (0.017) UG - Receiver 4.535*** (0.090) 0.214*** (0.031) 3.890*** (0.139) 9.512*** (0.224) Fairness × UG - Receiver -0.876*** (0.031) -0.043*** (0.010) -0.796*** (0.046) -1.791*** (0.077) Constant 0.184*** (0.018) 0.000 (0.000) 0.032* (0.014) 0.523*** (0.052) Observations 299,578 100,000 99,949 99,629 R² 0.019 0.001 0.017 0.039 Adjusted R² 0.019 0.001 0.017 0.039 Note. Rejection likelihood increases significantly with unfair offers and varies with temperature settings. UG – Receiver = dummy variable capturing suggestions to the receiver in the Ultimatum Game, Fairness = amount of money to the receiver; * p < 0.05; ** p < 0.01; *** p < 0.001 Further data exploration and answers to the validation checks In the supplementary material (Material K), we provide the analysis of the GPT’s suggestions regarding rejections of very generous offers, i.e., offers the proposer sends the whole 10€ to the responder. We included these prompts as a stress test for situations that rarely occur in human interactions. Suggestions in DG Binary showed strong tendencies towards inequality aversion, with suggestions to reject in more than 35% of the cases. This result starkly contrasts with the virtually non-existent aversion to inequality observed for unfair offers. Moreover, suggestions to reject in UG Receiver, where intentions are present, are rarely made – a result that is consistent with humans rarely rejecting generous offers. Control Questions. As for the answers to the control questions, these can help interpret the coherence of GPTs’ suggestions. Starting with the model answers to the control questions for the DG, the model consistently yielded accurate responses across all ten instances of prompting. In the case of UG Sender, however, invalid responses were observed for the question: "If person A proposes to send 2 Euros to person B and person B rejects, how much money does person A have in the end?". Here, the model consistently answered 10€ instead of 0€. This response suggests that the model has trouble correctly dealing with the more complicated strategic setting. Interestingly, the answer would have been accurate within the DG context, suggesting that the model suggestions may not fully differentiate the two contexts. Within the UG Receiver, we observed several inconsistencies. Specifically, in response to the question: "If person A proposes to send 2 Euros to person B and person B accepts how much money does person B have in the end?", the model provided the incorrect answer of 7€ eight times and the correct answer of 2€ only twice. Additionally, for the question: "If person A proposes to send 2 Euros to person B and person B rejects how much money does person B have in the end?”, the response was incorrect in all instances. Also, the model does not correctly deal with prompts entailing strategic interactions in this case. In these cases, analyzing suggestions is nonetheless informative because when asking for advice, people would probably not be exposed to this type of inconsistency in interpreting the games. Discussion This study focuses on understanding the fine-grained aspects of social preferences in GPT-3.5 suggestions. Our study builds on existing research by investigating GPT-3.5's alignment with human-like social preferences by testing them using well-established paradigms for social behavior. We extend previous work beyond altruism by examining the nuances of reciprocity and costly punishment. Moreover, we systematically analyzed the influence of demographic and technical parameters with the aim of discerning the extent to which GPT-3.5 suggestions reflect human social preferences and to identify factors that shape its responses. As the first main research question we assessed whether GPT-3.5's suggestions reflect strategic considerations related to the risk of rejection, comparing the offers in the DG to those in the UG. Surprisingly, we observed that DG offers exceeded UG offers. This finding contradicts the typical pattern observed in human participants, wherein UG offers tend to be generally higher, driven by the inherent risk of rejection and the associated potential for a 0–0 outcome 41,43,44 . This finding is largely consistent across gender and age of the advisee – with deviations only for advisees aged 51–70 years. Hence, LLM suggestions substantially deviate from typical human behavior in these different social settings. Namely, suggested mean offers in the DG were, on average, much higher (64.4%) than previously observed human behavior (~ 30%) 38 . This difference underscores the model's notably generous and seemingly altruistic disposition compared to human decision-making. These results do not align with previous findings documenting that model suggestions largely resembled humans' sharing rates in the DG 39 . Instead, they confirm results by Brookins & DeBacker 42 , showing that UG offers aligned more closely with typical human behavior (4€ in UG 45 ; in model = 4.13). GPT-3.5's suggestions demonstrate a level of generosity that surpasses typical human behavior. Speculatively, there could be a disparity between our written advocacy for the importance of altruism and our actual behavior (we do not practice what we preach ). This tendency towards generosity in the training data could influence the model's predisposition towards more altruistic suggestions the DG. It could further be speculated that an altruistic tendency was deliberately introduced by OpenAI, for example, in the context of reinforcement learning from human feedback, to render the model generally more amicable and prosocial in its suggestions. This is in line with recent work indicating that the newer version of GPT-3.5 (GPT-4) prioritizes the well-being of others over individual well-being (using a similar methodological approach to ours 46 ). This does not, however, explain why UG offers resemble typical human behavior. Additionally, the disparity between the LLM's and human offers in the DG contradicts the assumption that research of this nature may inadvertently yield more insights into the survey responses of the reference population rather than unveiling the inherent characteristics of the language models themselves 47,48 . As a second main question, besides LLMs capturing general notions of strategic risks, our study explored its sensitivity to reciprocity. Therefore, we compared the suggested offers in a DG following a benevolent action of the recipient to the suggested offers in the standard DG Sender, where a benevolent action towards the dictators before the game was absent. Overall, we find mixed evidence of positive reciprocity in the model suggestions. We find that suggested offers increase with the niceness of the other participant's gesture, supporting that GPT-3.5’s suggestions reflect human-like reciprocity 42 . However, the fact that in the absence of a generous action (DG Sender), suggestions are more generous than for low levels of niceness (DG Reciprocity) raises some doubts that warrant further consideration regarding the interpretation of these results. The disparities in responses between DG with reciprocity and DG Sender may be attributed to contextual differences. In DG with reciprocity, where the recipient's action influences the suggestions, the model demonstrates sensitivity to the perceived generosity of the other participant. In contrast, in DG Sender, which lacks knowledge about the other participant, the LLM indicates a general predisposition towards high altruism, as commented above. This suggests that the model's behavior may vary based on the contextual information available. Notably, the observed pattern aligns with human behavior in sequential DGs. It is, thus, possible that GPT-3.5 interpreted the prompt as a sequential DG. As a third question, we assessed whether the model’s suggestions capture the costly punishment of unfair offers. We did so by comparing two settings: rejections of unfair outcomes when these are the results of the action of another person (UG Receiver) versus when they are simply allocations one can choose from (Binary DG). Overall, our results indicate that LLM’s suggestions to reject reflect the unfairness of the action. Rejection rates increase with the relative unfairness in the UG Receiver setting. Interestingly, we find no support for inequity aversion as rejection rates remained close to zero in all conditions in the binary DG setting. Our findings generally reveal that rejection suggestions occur less frequently than in human behavior. Namely, rejection rates reached a maximum of 10%, starkly contrasting the 40% rejection rates observed in human participants in the UG 49 . Moreover, LLM’s suggestions to reject decreases with lower temperature levels. This novel result suggests that lower temperature levels lead to decisions commonly considered “rational” in a game-theoretic sense, with suggestions aligned with the maximization of one own monetary payoff. We find a resemblance between the dependence of rejection rates on the perceived unfairness of offers in GPT-3.5 suggestions and the expected patterns exhibited by human counterparts 50 . Methodological considerations The novel approach to letting the LLM complete comprehension check items provides another indication of LLM inconsistencies. The results revealed that the model exhibited limitations in grasping more intricate implications within the UG. However, understanding such nuances may not be a prerequisite for real-life applications. Our analysis indicates that the model's responses remain consistent with what we would expect from a human-generated answer, highlighting the potential implications of model behavior on human behavior, irrespective of an internal understanding of the scenarios on the side of the LLM. On a technical note, while the relationship between temperature and rejection rates can be given an emotional interpretation, with suggestions at lower temperatures being cold (i.e., not emotional) and rationally self-interested and suggestions at higher temperatures being war (i.e., emotional) there could be a mechanical explanation for this pattern. Since higher temperatures make the output of the model noisier, in cases where the model is forced to provide only one out of two tokens and the confidence is low enough, one can observe that the rate of the two tokens gets closer to 50% when increasing the temperature. Still, while a technical explanation for the effect can be provided, human recipients of the advice may ascribe emotionality to the underlying motives of the suggestion. On a methodological note, we believe that our research provides a foundation for future investigations and offers a benchmark for methodological approaches examining human-like characteristics and behaviors in LLMs. As AI systems continue to play an increasingly significant role in various applications, comprehending their behavior and the factors that influence it becomes paramount to ensure responsible and ethical AI interactions. Conclusion So, are LLMs good advisors in strategic situations? The answer, as so often, is it depends. For some settings, GPT-3.5 captures nuances of social situations. Overall, we find mixed results regarding the internal consistency of GPT-3.5 suggestions, which in some cases align with well-known behavioral patterns observed in human subjects and in others don’t. On the one hand, we find a remarkable degree of internal consistency within single decision settings, with altruism increasing in the niceness of the counterpart's action and with rejection rates in UG being dependent on the fairness of the offer. On the other hand, we document inconsistencies across decision settings, with offers in the presence of risk of rejection being lower than offers without risk of rejection. This result raises a warning regarding the use of single tasks to probe LLM and suggests adopting a multi-task approach, which permits a better assessment of consistency. Our finding that GPT-3-5 does not capture the strategic differences between both settings adds to the (growing) list of contexts where LLMs seem to fall short of human-like performance. Hence, blindly following suggestions by the LLM in strategic situations can also backfire. Declarations Competing interests The authors declare that they have no competing interests. Author Contribution EMS: Conceptualization, Methodology, Data Curation, Data Analysis, Writing – Original Draft, Writing – Review & Editing; SB: Software, Validation, Writing – Review & Editing; NK: Conceptualization, Methodology, Data Curation, Writing – Review & Editing; IS: Supervision, Data Analysis, Writing – Review & Editing Acknowledgements All authors were supported by the Max Planck Society. Eva-Madeleine Schmidt was supported by the Max Planck School of Cognition. Data Availability The datasets generated and/or analysed during the current study are available in the OSF repository, https://osf.io/shq8x/?view_only=52b2aa6871384fc4978dc1ec3100b575. References Hutson, M. Robo-writers: the rise and risks of language-generating AI. Nature 591, 22–25 (2021). Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z. & Trautsch, A. A large-scale comparison of human-written versus ChatGPT-generated essays. Sci. Rep. 13, 1–11 (2023). Köbis, N. C. & Mossink, L. D. Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Comput. Human Behav. 114, 106553 (2021). Böhm, R., Jörling, M., Reiter, L. & Fuchs, C. People devalue generative AI’s competence but not its advice in addressing societal and personal challenges. Commun Psychol 1, 1–10 (2023). Krügel, S., Ostermaier, A. & Uhl, M. ChatGPT’s inconsistent moral advice influences users’ judgment. Sci. Rep. 13, 1–5 (2023). Leib, M., Köbis, N., Rilke, R. M., Hagens, M. & Irlenbusch, B. Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty. The Economic Journal (2023). Heaven, W. D. Large language models can do jaw-dropping things. But nobody knows exactly why. Technology review (2024). Rahwan, I. et al. Machine behaviour. Nature 568, 477–486 (2019). Johnson, T. & Obradovich, N. Evidence of behavior consistent with self-interest and altruism in an artificially intelligent agent. Brookins, P. & DeBacker, J. M. Playing Games With GPT: What Can We Learn About a Large Language Model From Canonical Strategic Games? SSRN Journal (2023) doi: 10.2139/ssrn.4493398 . Beck, T., Schuff, H., Lauscher, A. & Gurevych, I. Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting. arXiv [cs.CL] 2589–2615 (2023). Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A. & Kersting, K. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nat Mach Intell 4, 258–268 (2022). Binz, M. & Schulz, E. Using cognitive psychology to understand GPT-3. Proc. Natl. Acad. Sci. U.S.A. 120, e2218523120 (2023). Miotto, M., Rossberg, N. & Kleinberg, B. Who is GPT-3? An Exploration of Personality, Values and Demographics. arXiv [cs.CL] (2022). Hartmann, J., Schwenzow, J. & Witte, M. The political ideology of conversational AI: Converging evidence on ChatGPT’s pro-environmental, left-libertarian orientation. arXiv [cs.CL] (2023). Hofmann, V., Kalluri, P. R., Jurafsky, D. & King, S. Dialect prejudice predicts AI decisions about people’s character, employability, and criminality. arXiv [cs.CL] (2024). Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L. & Naaman, M. Co-Writing with Opinionated Language Models Affects Users’ Views. arXiv [cs.HC] (2023). Weiss, A. et al. Trust in everyday life. J. Pers. Soc. Psychol. 121, 95–114 (2021). Columbus, S., Molho, C., Righetti, F. & Balliet, D. Interdependence and cooperation in daily life. J. Pers. Soc. Psychol. 120, 626–650 (2021). Hofmann, W., Wisneski, D. C., Brandt, M. J. & Skitka, L. J. Morality in everyday life. Science 345, 1340–1343 (2014). Strong, J. ‘AI reads human emotions. Should it? MIT Technology Review (2020). https:// online.ucpress.edu/collabra/article/7/1/19004/116331/Economic-Games-An-Introduction-and-Guide-for . Crandall, J. W. et al. Cooperating with machines. Nat. Commun. 9, 233 (2018). Werner, T. Algorithmic and human collusion. Köbis, N. et al. Artificial Intelligence can facilitate selfish decisions by altering the appearance of interaction partners. arXiv [cs.AI] (2023). Koster, R. et al. Human-centred mechanism design with Democratic AI. Nat Hum Behav (2022) doi: 10.1038/s41562-022-01383-x . von Schenk, A., Klockmann, V. & Köbis, N. Social Preferences Toward Humans and Machines: A Systematic Experiment on the Role of Machine Payoffs. Perspect. Psychol. Sci. 17456916231194948 (2023) doi: 10.1177/17456916231194949 . Guo, F. GPT in Game Theory Experiments. Preprint at http://arxiv.org/abs/2305.05516 (2023). Phelps, S. & Russell, Y. I. Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics. Preprint at http://arxiv.org/abs/2305.07970 (2023). Engel, C. Dictator games: a meta study. Exp. Econ. 14, 583–610 (2011). Cooper, D. J. & Dutcher, E. G. The dynamics of responder behavior in ultimatum games: a meta-study. Exp Econ 14, 519–546 (2011). Johnson, N. D. & Mislin, A. A. Trust games: A meta-analysis. Journal of Economic Psychology 32, 865–889 (2011). Eckel, C. C. & Grossman, P. J. CHIVALRY AND SOLIDARITY IN ULTIMATUM GAMES. ECONOMIC INQUIRY . Sarlo, M., Lotto, L., Palomba, D., Scozzari, S. & Rumiati, R. Framing the Ultimatum Game: Gender differences and autonomic responses. Int J Psychol 48, 263–271 (2013). Fornwagner, H., Grosskopf, B., Lauf, A., Schöller, V. & Städter, S. On the robustness of gender differences in economic behavior. Sci Rep 12, 21549 (2022). Klimecki, O. M., Mayer, S. V., Jusyte, A., Scheeff, J. & Schönenberg, M. Empathy promotes altruistic behavior in economic interactions. Sci. Rep. 6, 1–5 (2016). Nowak, M. A., Page, K. M. & Sigmund, K. Fairness versus reason in the ultimatum game. Science 289, 1773–1775 (2000). Oosterbeek, H., Sloof, R. & van de Kuilen, G. Cultural differences in ultimatum game experiments: Evidence from a meta-analysis. Exp. Econ. 7, 171–188 (2004). Brañas-Garza, P., Espín, A. M., Exadaktylos, F. & Herrmann, B. Fair and unfair punishers coexist in the Ultimatum Game. Sci. Rep. 4, 6025 (2014). Sobieszek, A. & Price, T. Playing Games with Ais: The Limits of GPT-3 and Similar Large Language Models. Minds & Machines 32, 341–364 (2022). Cochard, F., Le Gallo, J., Georgantzis, N. & Tisserand, J.-C. Social preferences across different populations: Meta-analyses on the ultimatum game and dictator game. Journal of Behavioral and Experimental Economics 90, 101613 (2021). Oosterbeek, H., Sloof, R. & van de Kuilen, G. Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis. Experimental Economics 7, 171–188 (2004). Forsythe, R., Horowitz, J. L., Savin, N. E. & Sefton, M. Fairness in simple bargaining experiments. Games and Economic Behavior 6, 347–369 (1994). Bechler, C., Green, L. & Myerson, J. Proportion offered in the Dictator and Ultimatum Games decreases with amount and social distance. Behav Processes 115, 149–155 (2015). Rutinowski, J. et al. The Self-Perception and Political Biases of ChatGPT. Human Behavior and Emerging Technologies 2024, 1–9 (2024). Dominguez-Olmedo, R., Hardt, M. & Mendler-Dünner, C. Questioning the Survey Responses of Large Language Models. Preprint at http://arxiv.org/abs/2306.07951 (2023). Diekmann, A. The Power of Reciprocity: Fairness, Reciprocity, and Stakes in Variants of the Dictator Game. The Journal of Conflict Resolution 48, 487–505 (2004). Herne, K., Lappalainen, O. & Kestilä-Kekkonen, E. Experimental Comparison of Direct, General, and Indirect Reciprocity. JOURNAL OF SOCIO-ECONOMICS 45, 38–46 (2013). Yamagishi, T. et al. The private rejection of unfair offers and emotional commitment. PNAS Proceedings of the National Academy of Sciences of the United States of America 106, 11520–11523 (2009). Charness, G. & Rabin, M. Understanding social preferences with simple tests. Q. J. Econ. 117, 817–869 (2002). Additional Declarations No competing interests reported. Supplementary Files 202406socprefLLMsupplementarymaterial.docx Cite Share Download PDF Status: Published Journal Publication published 27 Sep, 2024 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 28 Jul, 2024 Reviews received at journal 23 Jul, 2024 Reviews received at journal 21 Jul, 2024 Reviews received at journal 19 Jul, 2024 Reviews received at journal 13 Jul, 2024 Reviewers agreed at journal 10 Jul, 2024 Reviewers agreed at journal 09 Jul, 2024 Reviewers agreed at journal 08 Jul, 2024 Reviewers agreed at journal 26 Jun, 2024 Reviewers invited by journal 24 Jun, 2024 Editor assigned by journal 24 Jun, 2024 Editor invited by journal 24 Jun, 2024 Submission checks completed at journal 24 Jun, 2024 First submitted to journal 20 Jun, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4611495","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":326565411,"identity":"10b39ae8-4075-4642-a01e-241f6595c10e","order_by":0,"name":"Eva-Madeleine Schmidt","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFUlEQVRIie2RMUvEMBTHXwi0SyRrSkW/QqRwNxxcv0pK4W6Jep9AC0JvdC3c9+hceaviJgU7HA6ZFVcR47XqIcFzFMxveATe+/HPSwA8nr9JACApcHtCgAYgJMXQUT8qIio+FPorBaiQ78O98tlxK5yjeWgXk7Pk7uYKF9AdjC9ouSa6mxZh3riUqJqNEy3nYnR/orACk+wjWUpSm7xgxhkjWxbEWqJVbN17xayipBSkxhyEli4lvb3ulWRlKwM831JOH50poHtFxr2ixKBMbYpzfdHORnZ4HlXDLkeblKw2KmDGeTF+iSbWLxPOV8f4bF/sUIRLI57qLuVhvnbGfMG2zgqarNwx/02xf5ruNjwej+ef8AbOMl9I20a/EQAAAABJRU5ErkJggg==","orcid":"","institution":"Max Planck Institute for Human Development","correspondingAuthor":true,"prefix":"","firstName":"Eva-Madeleine","middleName":"","lastName":"Schmidt","suffix":""},{"id":326565412,"identity":"bb1748c0-6589-4058-8d87-9a11eb07063c","order_by":1,"name":"Sara Bonati","email":"","orcid":"","institution":"Max Planck Institute for Human Development","correspondingAuthor":false,"prefix":"","firstName":"Sara","middleName":"","lastName":"Bonati","suffix":""},{"id":326565413,"identity":"3d37bed0-4ea5-45a4-ae17-1a8a6384481b","order_by":2,"name":"Nils Köbis","email":"","orcid":"","institution":"Max Planck Institute for Human Development","correspondingAuthor":false,"prefix":"","firstName":"Nils","middleName":"","lastName":"Köbis","suffix":""},{"id":326565414,"identity":"82cb71e2-cb00-4ec1-99ac-a37b8465068b","order_by":3,"name":"Ivan Soraperra","email":"","orcid":"","institution":"Max Planck Institute for Human Development","correspondingAuthor":false,"prefix":"","firstName":"Ivan","middleName":"","lastName":"Soraperra","suffix":""}],"badges":[],"createdAt":"2024-06-20 11:40:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4611495/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4611495/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-024-73306-x","type":"published","date":"2024-09-27T15:57:56+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":60339396,"identity":"ceb83d17-22fb-4610-bee1-b622272027f7","added_by":"auto","created_at":"2024-07-15 17:58:06","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":313615,"visible":true,"origin":"","legend":"\u003cp\u003eSubordinated Questions to Research Question 1 (To what extent do GPT-3.5 suggestions reflect human social preferences?) and Experimental Design\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eNote\u003c/em\u003e. Q1 = Question 1, Q2 = Question 2, Q3 = Question 3, DG = Dictator Game, UG = Ultimatum Game.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4611495/v1/f3abbc2334a9e70d81025ead.png"},{"id":60339395,"identity":"b59d9471-87fd-4927-a109-e1e20efb565d","added_by":"auto","created_at":"2024-07-15 17:58:06","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":46052,"visible":true,"origin":"","legend":"\u003cp\u003eAverage suggested amount sent in DG Sender and UG Sender by Temperature Level\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eNote. \u003c/em\u003eAverage suggestions of GPT-3.5 in the Dictator Game (DG) were higher than those in the Ultimatum Game (UG). The \u003cem\u003ep\u003c/em\u003e-values associated with each comparison are indicated as follows: *\u003cem\u003ep\u003c/em\u003e \u0026lt; 0.05, **\u003cem\u003ep\u003c/em\u003e \u0026lt; 0.01, ***\u003cem\u003ep\u003c/em\u003e \u0026lt; 0.001.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-4611495/v1/e8ee3e8a97a4b41c589a39e8.png"},{"id":60339399,"identity":"50c410d5-dfb9-4f07-83c7-1794ce6aacdc","added_by":"auto","created_at":"2024-07-15 17:58:06","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":14184,"visible":true,"origin":"","legend":"\u003cp\u003eMean suggested Amount to send in DG Sender and DG Reciprocity by Temperature\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eNote\u003c/em\u003e. Suggestions increase with the kindness of the other player's action (DG-1 to DG-10). Suggestions in the presence of a kind action are more generous than in the absence of a kind action (DG Sender) only when kindness is substantial (DG-10). DG = Dictator Game, DG-\u003cem\u003en\u003c/em\u003e refer to DG Reciprocity with \u003cem\u003en\u003c/em\u003edepicting the amount of money a receiver previously gifted to the dictator.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-4611495/v1/0bf8fabae20fb6c5f963b265.png"},{"id":60339398,"identity":"14c99aa5-6c3f-4662-bbe0-f1248d544c10","added_by":"auto","created_at":"2024-07-15 17:58:06","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":13885,"visible":true,"origin":"","legend":"\u003cp\u003eRejection Rates in DG Binary and UG Receiver per Temperature\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eNote. \u003c/em\u003eDG = Dictator game, UG = Ultimatum game. The ordered pairs of values on the x-axis indicate the split of the 10 euros, with the first value representing the amount for the dictator / proposer and the second value representing the amount for the recipient / responder.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-4611495/v1/1b82635cf2d24c9cf20aacd4.png"},{"id":65628079,"identity":"f423da2e-184f-4f05-8200-0a2fd044f4bb","added_by":"auto","created_at":"2024-09-30 16:17:43","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1059526,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4611495/v1/cd89f513-5fc0-497f-adaf-22737face86b.pdf"},{"id":60339670,"identity":"6595e113-bd55-482c-aac1-a6d2c6c71111","added_by":"auto","created_at":"2024-07-15 18:06:06","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":171294,"visible":true,"origin":"","legend":"","description":"","filename":"202406socprefLLMsupplementarymaterial.docx","url":"https://assets-eu.researchsquare.com/files/rs-4611495/v1/498022faff0a1b9b25d930c7.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk","fulltext":[{"header":"Introduction","content":"\u003cp\u003eNovel technical architectures such as transformers and extensive text corpora as training data have enabled recent breakthroughs in Natural language processing (NLP). With the advent of ChatGPT, this technological progress quickly became accessible to millions of users. By now, people can easily make use of multiple pre-trained large language models (pre-trained LLMs) to generate suggestions for various everyday activities, ranging from computer coding, automatic translations, and a plethora of writing tasks\u003csup\u003e1\u0026ndash;3\u003c/sup\u003e. In view of the growing use of LLMs for everyday advice, concerns arise regarding their response patterns and their consequent impact on human behavior. Underlying these concerns: humans often follow AI advice\u003csup\u003e4,5\u003c/sup\u003e, even if it encourages people to break ethical rules\u003csup\u003e6\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe classic approach to understanding the output of computer models has been to take a look \u0026ldquo;under the hood\u0026rdquo; of the machine. However, this approach does not work with LLMs. The technical design of such models is often intransparent and highly complex, and therefore, generative AI models produce more unpredictable outputs\u003csup\u003e7\u003c/sup\u003e. One recent approach to gaining a better understanding of the performance of LLMs is observing their behavior in controlled experiments akin to how social scientists observe humans' behavior\u003csup\u003e8\u003c/sup\u003e. Thus, a growing number of studies have started to systematically probe LLMs' responses to different prompts\u003csup\u003e9,10\u003c/sup\u003e. For instance, socio-demographic prompting entails systematically sending multiple prompts with minor changes along socio-demographic features in the text\u003csup\u003e11\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eSuch studies have revealed varying degrees of resemblance to human-like behavior across domains such as ethical norms, logical reasoning tasks, personality facets, and moral judgments, highlighting similarities and divergences in LLM behavior compared to human responses\u003csup\u003e12\u0026ndash;14\u003c/sup\u003e. For instance, several studies have sought to understand the political colorings of LLMs. One paper indicates that early versions of ChatGPT show a green-left-leaning political bias when prompted to answer questions about politics\u003csup\u003e15\u003c/sup\u003e. More recent studies document racial biases in AI advice\u003csup\u003e16\u003c/sup\u003e. Namely, some LLMs systematically suggest less prestigious jobs when prompts are written in African-American English, and defendants described in this dialect spoken by millions of Americans were more likely to receive a death penalty recommendation by an LLM compared to prompts written in \u0026ldquo;standard\u0026rdquo; English\u003csup\u003e16\u003c/sup\u003e. An additional concern arises from research showing that people by no means ignore AI-generated suggestions and advice but are, in fact, often altering their views, beliefs, and behavior based on it\u003csup\u003e17\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eWhile AI advice has received attention in political and ethical domains, less is known about more fundamental social tendencies of LLMs. As inherently social beings, humans face countless social situations in their daily life\u003csup\u003e18\u0026ndash;20\u003c/sup\u003e. Frequently, such situations present trade-offs between conflicting goals and norms. For instance, whether one should sacrifice one's own resources to help others or how much one should trust unknown others. Especially when people\u0026rsquo;s interests and motivations clash, they seek advice to decide what to do. Here, LLMs increasingly become trusted advisors\u003csup\u003e1,21\u003c/sup\u003e. AI advisors have the advantage of always being available and quick to offer advice for any given query. A systematic investigation of the type of advice LLMs give in such social situations is lacking. Therefore, whether LLMs mimic humans' notions of other-regarding concerns remains largely unknown.\u003c/p\u003e \u003cp\u003eTo gain insights into the advice provided by LLMs for some of the most basic social human behaviors, we draw on economic games. Instead of assessing what people \u003cem\u003esay they would do\u003c/em\u003e, these economic games allow us to observe what people \u003cem\u003eactually do\u003c/em\u003e when facing a decision with financial consequences for themselves and others. Therefore, economic games offer a clear and straightforward measure of how people trade off their and others' well-being in different social situations\u003csup\u003e22\u003c/sup\u003e. Due to this wide popularity of gauging human social behavior using simple decisions, economic games are also becoming an increasingly popular tool for studying machine behavior \u003csup\u003e23\u0026ndash;27\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eA recent line of research has used systematic prompting to understand LLMs' social preferences better. One study has used natural language descriptions of altruism and selfishness to introduce human-like behavior of LLMs in economic games\u003csup\u003e28,29\u003c/sup\u003e. Besides investigating the capacity of GPT-3.5 to manifest human-like social preferences, others have studied the preferences already incorporated in the model. Johnson \u0026amp; Obradovich\u003csup\u003e25\u003c/sup\u003e let GPT-3.5 play a Dictator Game, i.e., a task in which a decision-maker has to divide a financial resource between themselves and a passive recipient. The results show that GPT-3.5 allocates as much money to human partners as humans do, resembling behavior consistent with human altruism. Brookins and DeBacker\u003csup\u003e28\u003c/sup\u003e study LLM's behavior in the Dictator Game and prisoner dilemma. The latter describes a canonical task to assess cooperation. They argue that GPT-3.5 replicates human tendencies towards fairness and cooperation. The model exhibited elevated levels of altruism and cooperation in the Dictator Game and Prisoners\u0026rsquo; Dilemma, respectively.\u003c/p\u003e \u003cp\u003eOur paper adds to this line of research in two main ways. First, we assess the LLM's strategic and reciprocal tendencies when giving advice by using two economic games: the Dictator Game (DG) and the Ultimatum Game (UG). Both games are two-person games with a decision-maker deciding how to share an amount of money with a recipient. While the DG reflects a non-strategic decision as the recipient is merely passive, the UG adds a strategic component to this social situation. Namely, here, the sender \u003cem\u003eproposes\u003c/em\u003e a division of the resources that the responder can either accept or reject, leaving both players with nothing. Therefore, we introduce the novel approach to compare LLM's behavior across two structurally similar but strategically distinctive games, allowing us to disentangle the model's sensitivity for the fairness of the outcomes and intentions. This way, we gain novel insights into whether LLMs social preferences are sensitive to intentions.\u003c/p\u003e \u003cp\u003eSecond, we manipulate the prompt and the technical details of the LLM to assess the robustness of the advice. On the one hand, we compare the LLM's suggestions for different demographics by manipulating the age and gender of the person receiving the advice. This approach allows us to explore whether suggestions are homogenous or different people would receive different advice. On the other hand, we manipulate the technical parameter of LLM's temperature. This parameter is crucial in determining the randomness and creativity of the LLM's output. A lower temperature leads to more deterministic and potentially repetitive responses, as the model prefers the most likely following words. Conversely, a higher temperature encourages the model to explore less likely options, injecting diversity and creativity into the responses. Since manipulations of temperature levels can produce \"personality changes\" of the model\u003csup\u003e29\u003c/sup\u003e, it is important to assess its impact on advices.\u003c/p\u003e \u003cp\u003eWhile a rich collection of behavioral studies has documented how humans behave in economic games, research on LLMs' behavior in such tasks is in its infancy. Many behavioral studies have examined how people behave in the canonical games DG and the UG, allowing the aggregation of empirical insights in large meta-analyses (see meta-analyses on DG\u003csup\u003e30\u003c/sup\u003e and UG\u003csup\u003e31\u003c/sup\u003e). A diverse overview exists of how demographic factors, such as age\u003csup\u003e30,32\u003c/sup\u003e and gender, shape behavior in economic games (DG\u003csup\u003e30\u003c/sup\u003e; UG\u003csup\u003e33\u0026ndash;35\u003c/sup\u003e). Likewise no clear relationship between prompted gender and behavior in LLMs appears to exist \u0026ndash; at least in terms of responses to the personality inventory\u003csup\u003e35\u003c/sup\u003e. However, evidence on how socio-demographic prompting of these features influences LLMs' advice remains scarce.\u003c/p\u003e \u003cp\u003e \u003cb\u003eResearch Questions and Identification Strategy.\u003c/b\u003e Specifically, we aimed to answer three main research questions that we pre-registered on As.predicted (see \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://aspredicted.org/RXX_NM8\u003c/span\u003e\u003cspan address=\"https://aspredicted.org/RXX_NM8\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe first question (Q1) assesses whether GPT-3.5 suggestions are sensitive to strategic considerations, thus following a human-like pattern, or are purely motivated by altruism. To test this notion of strategic sensitivity, we prompted GPT-3.5 to advise senders in the DG and UG on how much of their initial endowment they should send to a recipient (see more details in Methods). To test whether the LLM suggestions are sensitive to the risk of being rejected, we can compare the average amount suggested by the LLM in the UG and in the DG, where the risk is absent. If the amounts suggested in the UG significantly exceed those in the DG, the LLM reflects this strategic consideration in its advice.\u003c/p\u003e \u003cp\u003eThe second question (Q2) examines whether GPT-3.5 suggestions incorporate the concept of positive reciprocity, i.e., if GPT suggestions reward a previous kind action of the counterpart. To investigate whether the model suggests a generous response to a kind action, we prompted the GPT-3.5 with a scenario describing a previous interaction in which the receiver in the DG made a monetary gift to the sender. If suggestions are sensitive to positive reciprocity, the amounts suggested should be increasing in the size of the gift, and eventually higher than in the standard version of the DG where the kind action is not present.\u003c/p\u003e \u003cp\u003eThe third question (Q3) tests whether GPT-3.5\u0026rsquo;s suggestions incorporate costly punishment of bad intentions. Specifically, we explore the influence of negative reciprocity on the suggestion to reject unfair offers by studying whether suggestions to reject stem from the unfairness of the outcomes alone or involve a retaliatory response to unfair behavior. To do so, we compare the rejection suggestions made to a responder facing an unfair offer in the UG \u0026mdash; for instance, receiving only 10% of the total amount while the sender retains 90% \u0026mdash; with the suggestions given to an individual presented with an equivalent binary choice outside the UG context \u0026mdash; i.e., the suggestion of what to choose between the option of keeping 10% of an amount of money for yourself and giving 90% to another person and the option of both players obtaining nothing. This design allows us to keep the (un)fairness of the outcomes constant while manipulating the way such (un)fairness originates. If suggestions are sensitive to outcome fairness, we expect to observe rejection suggestions to increase with the inequality of the final distribution. Moreover, if suggestions are driven by negative reciprocity, we expect to observe more suggestions to reject when the decision follows from an unfair proposal.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e \u003cb\u003eTechnical Details.\u003c/b\u003e We used Python (version 3.10.8) to query the pre-trained OpenAI text-davinci-003 LLM (GPT-3.5) through the OpenAI API and to perform data pre-processing. We conducted the data analysis and visualization in R (version 4.3.1).\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eDesign\u003c/h2\u003e \u003cp\u003e \u003cb\u003eThe tasks.\u003c/b\u003e To answer our research question we employ the two games mentioned above: the Dictator Game (DG) and the Ultimatum Game (UG). The DG is a two-person game where a person (the dictator) unilaterally decides how much money to allocate to another person (the recipient). The DG is often used to study altruism and reciprocity\u003csup\u003e27,30,36\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe UG is also a two-person game where a person (the proposer) proposes a division of resources to a second person (the responder) who decides whether to accept or reject the offer. The UG is often used to study strategic negotiation and fairness considerations\u003csup\u003e37\u0026ndash;39\u003c/sup\u003e. In this paper, we use the following variations of the games:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eDictator Game (DG)\u003c/b\u003e: The dictator is given 10 Euros and can divide this money as they please between themselves and the recipient, who is passive and cannot make any decision.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eDictator Game with reciprocity (DG-R)\u003c/b\u003e: The same as the DG, with the only difference that previous to the current interaction, the recipient gifted the dictator an amount of money that can take the following values: 1, 2, 3, 4, 5, and 10 euros.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eUltimatum Game (UG)\u003c/b\u003e: The proposer is given 10 Euros and can propose a division of this money between themselves and the responder. In this case, the responder can choose whether to accept the proposal, obtaining the proposed share for themselves and leaving the rest of the money to the proposer, or to reject the proposal, leaving both of them with nothing.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eBinary Dictator Game (B-DG)\u003c/b\u003e: A variation of the previously described DG, where the choice of the dictator is restricted to two possible options: zero euros for both players or a specific split of the 10 euros. This game serves as the benchmark comparison for the responder's rejection rates in the UG.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003ePrompts of the games\u003c/b\u003e. We designed game-specific prompts closely resembling human participant study descriptions to ensure comparability. GPT-3.5 was not portrayed as an independent agent; instead, prompts were used to seek model-generated suggestions for player behavior. This choice stemmed from two considerations: (a) the model cannot answer for itself, as its training data does not encompass knowledge about the qualities of GPT-3.5, a limitation previously highlighted\u003csup\u003e40\u003c/sup\u003e, and (b) asking for behavioral suggestions aligns with potential real-world use cases that could affect human behavior, as discussed above. We provide the exact prompts used in this study in the supplementary material (see supplementary material, Material B). To ensure that recommendations are based on a formally correct understanding of the games, allowing for a meaningful interpretation by humans, we asked the model to answer comprehension questions similar to those typically used for human participants in behavioral experiments (see supplementary material, Material C).\u003c/p\u003e \u003cp\u003eWe queried GPT-3.5 using prompts describing these tasks from the perspective of one of the players who is asking for suggestions on how to behave and asked to provide answers in a structured json format (see supplementary material, Material B). Specifically, we submitted prompts where:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDictators in the DG and proposers in the UG ask for suggestions about the amount X to send to the other player (X from 0 to 10);\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eResponders in the UG ask for suggestions on whether to accept or reject an offer of Y euros (with Y in {1, 2, 3, 4, 5, 10});\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDictators in the DG with reciprocity ask for suggestions about the amount X to send to the recipient (X from 0 to 10) given that the recipient gifted them an amount Y before the interaction (with Y in {1, 2, 3, 4, 5, 10});\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDictators in the binary DG ask for suggestions on what to choose between the option to keep Y euros for them and give 10 - Y euros to the recipient (with Y in {1, 2, 3, 4, 5, 10}) and the option of both players obtaining zero euros.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eIn Fig.\u0026nbsp;1, we illustrate how the replies to these prompts map onto and allow us to answer each research question. The supplementary material presents a more detailed overview of the experimental design (see supplementary material, Material A).\u003c/p\u003e \u003cp\u003e \u003cb\u003eFigure 1\u003c/b\u003e Subordinated Questions to Research Question 1 (To what extent do GPT-3.5 suggestions reflect human social preferences?) and Experimental Design\u003c/p\u003e \u003cp\u003e \u003cstrong\u003e\u003cdiv description=\"\" class=\"Drawing\" id=\"1\" name=\"image5.png\"\u003e\u003c/div\u003eNote\u003c/strong\u003e \u003cp\u003eQ1\u0026thinsp;=\u0026thinsp;Question 1, Q2\u0026thinsp;=\u0026thinsp;Question 2, Q3\u0026thinsp;=\u0026thinsp;Question 3, DG\u0026thinsp;=\u0026thinsp;Dictator Game, UG\u0026thinsp;=\u0026thinsp;Ultimatum Game.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003ePrompt Manipulations.\u003c/b\u003e To assess robustness of the advices, we systematically manipulated the temperature of the language model, which is related to the creativity of the replies, and the demographic features of the person asking for suggestions. As for the temperature, we considered three levels: below default (temperature\u0026thinsp;=\u0026thinsp;0.5); OpenAI default (temperature\u0026thinsp;=\u0026thinsp;1); and above default (temperature\u0026thinsp;=\u0026thinsp;1.5). As for the features of the person asking for advice, we included demographic information to some versions of the prompt. We systematically manipulated the demographics along two factors: the age of the person receiving the suggestion (with three levels: 18\u0026ndash;30 yrs, 31\u0026ndash;50 yrs, and 51\u0026ndash;70 yrs) and the gender of the person receiving the suggestion (with three levels: female, male, and non-binary). For each prompt, we collected 1000 replies for each temperature level without including demographic information (\u003cem\u003eunprompted demographics\u003c/em\u003e) and 1000 replies for each temperature level and combination of the demographic factors (see supplementary material, Material D for a more detailed breakdown of the variations).\u003c/p\u003e \u003cp\u003eIn the pre-registration, we planned to collect 10 responses for each prompt variation using a temperature of 0 to assess the model responses when LLM predictions are deterministic. We failed to collect such responses due to a coding mistake, and by the time we realized that such data was missing, OpenAI already updated the model. We therefore decided not to collect such information.\u003c/p\u003e \u003c/div\u003e"},{"header":"Analysis \u0026 Results","content":"\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\u003cp\u003eThe pre-processing steps are detailed in the supplementary material (Material E). The pre-registration of the analyses can be found under the following link: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://aspredicted.org/RXX_NM8\u003c/span\u003e\u003cspan address=\"https://aspredicted.org/RXX_NM8\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. For all the questions, we performed separate tests for each temperature level and a single test collapsing the temperature. The main analysis collapsed the answers for the different levels of demographics. To additionally test whether and how age and gender influence the responses of the model we plotted the dependent variables grouped by age and gender for each question and conducting subsequent regression analyses.\u003c/p\u003e \u003cp\u003eThe analysis included 599,244 valid responses out of 600,000 prompts submitted (see supplementary material, Material F for an overview of the number of observations per game and temperature). Invalid responses are the ones for which the model either provided a value in the json response that did not comply with the type requested or it did not provide a value in the json response and it was not possible to infer the value from the text (see supplementary material, Material E for the details).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eQ1: Are GPT altruistic suggestions sensitive to strategic considerations?\u003c/h2\u003e \u003cp\u003eTo answer (Q1) whether altruistic suggestions are sensitive to strategic considerations or purely driven by altruism, we compared the suggested amount sent in the DG and the suggested amount sent in the UG. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the average suggestion in the two cases separated by temperature level.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eNote\u003c/strong\u003e \u003cp\u003eAverage suggestions of GPT-3.5 in the Dictator Game (DG) were higher than those in the Ultimatum Game (UG). The \u003cem\u003ep\u003c/em\u003e-values associated with each comparison are indicated as follows: *\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.05, **\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.01, ***\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eWhen pooling responses across the different levels of the demographic and temperature factors, we observe considerably more altruistic suggestions in DG Sender (\u003cem\u003eM\u003c/em\u003e\u0026thinsp;=\u0026thinsp;6.44, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1.57) than in UG Sender (\u003cem\u003eM\u003c/em\u003e\u0026thinsp;=\u0026thinsp;4.13, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1.55), \u003cem\u003et\u003c/em\u003e(59973)\u0026thinsp;=\u0026thinsp;180.77, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;.001, \u003cem\u003eCohen's d\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1.47 (95% CI [1.46; 1.49]) and results are consistent across different temperature levels (see Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). This result indicates that the model\u0026rsquo;s suggestions do not reflect human behavior when a risk of rejection exists. Contrary to typical results in the behavioral science literature, we observe substantially lower suggested amounts to send when the risk is present than when the risk is not present \u003csup\u003e41\u003c/sup\u003e. Specifically, the amounts suggested by the LLM in the DG are substantially more generous than those of human participants. The amounts suggested by the LLM in the UG are slightly lower than in behavioral experiments with human participants.\u003c/p\u003e \u003cp\u003eWhen looking at the effect of the demographic features of the person receiving the advice, we find robust effects across age and gender. In all cases, the model suggestions are less generous when risk is not present (see supplementary material, Material G). The effect size is also comparable across demographics, except for older advisees, i.e., age 51\u0026ndash;70, who receive more extreme advice than the other age groups.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eResults of Welch Two Sample t-test\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"9\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eDG Sender\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003eUG Sender\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c9\" namest=\"c8\"\u003e \u003cp\u003e\u003cem\u003eCI\u003c/em\u003e\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cem\u003eSD\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003eSD\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003elower\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eupper\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAll\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cem\u003et\u003c/em\u003e(59973)\u0026thinsp;=\u0026thinsp;180.77\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;.001\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e2.28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e2.33\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;0.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3.84\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cem\u003et\u003c/em\u003e(\u003cem\u003e18169\u003c/em\u003e)\u0026thinsp;=\u0026thinsp;160.89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;.001\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e2.70\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e2.76\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;1.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cem\u003et\u003c/em\u003e(19962)\u0026thinsp;=\u0026thinsp;104.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;.001\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e2.24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e2.33\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;1.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6.30\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.77\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.90\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cem\u003et\u003c/em\u003e(19885)\u0026thinsp;=\u0026thinsp;73.01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;.001\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e1.95\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"9\"\u003e\u003cem\u003eNote.\u003c/em\u003e Suggestions of GPT-3.5 in the Dictator Game (DG) were higher than those in the Ultimatum Game (UG) across all temperature levels. When compared to human behavior, we find more altruistic suggestions in the DG, suggesting that the model's recommendations do not align with human behavior when there is a risk of rejection.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eQ2: Do GPT suggestions incorporate positive reciprocity?\u003c/h2\u003e \u003cp\u003eTo answer (Q2) whether the model\u0026rsquo;s suggestions reward the kind actions of the counterpart, we look at the suggested amount in the DG Reciprocity conditions. We assess reciprocity in two ways: first, we look at whether suggestions are increasing in the size of the gift received from the counterpart, and second, we compare these suggestions to the suggested amount in the DG Sender, where the kind action is absent. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the average amount sent in the DG Sender and the DG Reciprocity for the different kindness levels, i.e., 1, 2, 3, 4, 5, and 10 euros. The figure shows that (i) suggestions are indeed increasing with the kindness of the other player's action, and (ii) suggestions in the presence of a kind action are more generous than in the absence of a kind action only when kindness is substantial.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eNote\u003c/strong\u003e \u003cp\u003eSuggestions increase with the kindness of the other player's action (DG-1 to DG-10). Suggestions in the presence of a kind action are more generous than in the absence of a kind action (DG Sender) only when kindness is substantial (DG-10). DG\u0026thinsp;=\u0026thinsp;Dictator Game, DG-\u003cem\u003en\u003c/em\u003e refer to DG Reciprocity with \u003cem\u003en\u003c/em\u003e depicting the amount of money a receiver previously gifted to the dictator.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eLinear regressions with the suggested amount as the dependent variable and with both the kindness of the action and a dummy variable capturing the DG Sender as the explanatory variables statistically support these observations (see Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). The estimated parameter for the \u0026ldquo;Kindness\u0026rdquo; variable is positive, indicating that suggestions increase with the amount received from the counterpart. Moreover, the estimated parameter for the variable \u0026ldquo;DG Sender\u0026rdquo; is significantly greater than zero, indicating that suggestions are higher when kindness is absent compared to when kindness is present. Overall, the results support the existence of positive reciprocity in the LLM\u0026rsquo;s suggestions as suggested generosity is increasing with the kindness of the other participant. However, we find contradictory results when comparing these suggestions to those without a generous action. As the figure and the regressions show, these results are robust across temperature levels of the model. When looking at robustness across demographic, we observe consistent patterns for all the demographic features (see supplementary material, Material H).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eOLS regression with Amount Sent as the Dependent Variable\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAll\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;0.5\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;1.0\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;1.5\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eKindness\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.448***\u003c/p\u003e \u003cp\u003e(0.001)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.517***\u003c/p\u003e \u003cp\u003e(0.002)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.450***\u003c/p\u003e \u003cp\u003e(0.002)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.376***\u003c/p\u003e \u003cp\u003e(0.003)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDG Sender\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.734***\u003c/p\u003e \u003cp\u003e(0.012)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.031***\u003c/p\u003e \u003cp\u003e(0.017)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.763***\u003c/p\u003e \u003cp\u003e(0.021)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.407***\u003c/p\u003e \u003cp\u003e(0.025)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eConstant\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.704***\u003c/p\u003e \u003cp\u003e(0.007)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.539***\u003c/p\u003e \u003cp\u003e(0.010)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.684***\u003c/p\u003e \u003cp\u003e(0.012)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e4.890***\u003c/p\u003e \u003cp\u003e(0.014)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eObservations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e209,879\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e70,000\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e69,996\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e69,883\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u0026sup2;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.331\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.517\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.344\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.201\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdjusted R\u0026sup2;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.331\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.517\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.344\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.201\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"5\"\u003e\u003cem\u003eNote.\u003c/em\u003e In DG with reciprocity, suggestions increase with the amount received from the counterpart. Suggestions are higher when kindness is absent (DG Sender) compared to when kindness is present. The results support the existence of positive reciprocity in the LLM\u0026rsquo;s suggestions, but we find contradictory results when comparing these suggestions to those without a generous action. DG Sender\u0026thinsp;=\u0026thinsp;dummy variable capturing suggestions in the Dictator Game, Kindness\u0026thinsp;=\u0026thinsp;continuous variables capturing the kindness of the other participant action. *p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; **p\u0026thinsp;\u0026lt;\u0026thinsp;0.01; ***p\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eQ3: Do GPT suggestions incorporate costly punishment for unfair actions?\u003c/h2\u003e \u003cp\u003eTo assess the influence of negative reciprocity on the suggestion to reject unfair offers (Q3), we compare GPT suggestions in UG Reciever and in the DG Binary. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e4\u003c/span\u003e shows the rate of rejection suggestion for different allocations in the two cases, split by the temperature of the model. The figure highlights various patterns. First, suggestions to reject unfair allocations in binary DG are virtually non-existent for all temperature levels. This result implies that GPT suggestions are not averse to outcome inequality, even when inequality is substantial (1 euro for the decision maker vs 9 euros for the other player).\u003c/p\u003e \u003cp\u003eSecond, suggestions to reject unfair proposals in UG strongly correlate with the model's temperature. Similar to binary DG, suggestions to reject are virtually non-existent at low temperature levels. For higher temperature levels, however, the model starts to suggest rejections of unfair outcomes proposed by the other player. Third, such suggestions are weakly increasing with the level of unfairness. Overall, these results indicate that \u0026ldquo;cold\u0026rdquo; GPT models (low temperature) make very rational and self-interested suggestions. Conversely, \u0026ldquo;warmer\u0026rdquo; models (high temperature) suggest retaliatory punishment for unfair actions. While these patterns align with human emotional reactions to intentions, rejection rates seldomly exceed 10%. This result is in notable contrast with human behavior, where, on average, approximately 16% of UG offers are rejected\u003csup\u003e42\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eNote\u003c/strong\u003e \u003cp\u003eDG\u0026thinsp;=\u0026thinsp;Dictator game, UG\u0026thinsp;=\u0026thinsp;Ultimatum game. The ordered pairs of values on the x-axis indicate the split of the 10 euros, with the first value representing the amount for the dictator / proposer and the second value representing the amount for the recipient / responder.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eThese observations receive statistical support from the regression results in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. The table reports the results of linear probability models with a dummy variable indicating the rejection, i.e., the choice of the option that gives both agents zero euros, as the dependent variable. We included the variable \u0026ldquo;Fairness\u0026rdquo;, capturing the amount for the decision maker; the indicator variable \u0026ldquo;UG Receiver\u0026rdquo;, capturing choices made in the Ultimatum Game context; and their interaction, as explanatory variables. Estimated parameters in the Table are scaled to show the effects in percentage points. Regression model (2; temperature\u0026thinsp;=\u0026thinsp;0.5) shows that rejections are virtually non-existent and do not change substantially with the fairness of the outcome and with the intention \u0026mdash; e.g., UG Receiver increases rejection suggestions of the most unfair offer by only one-quarter of a percentage point. Models (3; temperature\u0026thinsp;=\u0026thinsp;1) and (4; temperature\u0026thinsp;=\u0026thinsp;1.5) show substantially higher suggestions to reject when unfairness is generated by another person\u0026rsquo;s decision (the likelihood of rejection of the most unfair offer is 3.9 and 9.5 percentage points higher in the context of the UG for the medium and high temperature, respectively) and a strong relationship between rejection suggestions and the fairness of the allocation in the UG context (Compared to the DG context, for the medium and high-temperature suggestions to reject in the UG decrease by 0.8 and 1.8 percentage point for each extra euro allocated to the decision-maker). When looking at these patterns across different demographic features of the person receiving the suggestion, we find qualitatively robust results (see supplementary material, Material I). At the same time, some differences in the intensity of negative reciprocity emerge, with rejection suggestions decreasing with age and slightly higher for the non-binary gender category.\u003c/p\u003e \u003cp\u003eWe further repeated the regression using logit and probit models (see supplementary material, Material J). In contrast to the OLS regression, these show that fairness increases the likelihood of rejection and that UG Receiver variable decreases rejection likelihood. This indicates that the relationship between fairness, the UG context, and rejection behavior is complex and that results should be interpreted with caution.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eOLS Regressions with Rejection Rates as the Dependent Variable\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAll\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;0.5\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;1.0\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTemp\u0026thinsp;=\u0026thinsp;1.5\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFairness\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-0.039*** (0.006)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.000\u003c/p\u003e \u003cp\u003e(0.000)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-0.010*\u003c/p\u003e \u003cp\u003e(0.004)\u003c/p\u003e\u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-0.107***\u003c/p\u003e \u003cp\u003e(0.017)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eUG - Receiver\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.535***\u003c/p\u003e \u003cp\u003e(0.090)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.214***\u003c/p\u003e \u003cp\u003e(0.031)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3.890***\u003c/p\u003e \u003cp\u003e(0.139)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e9.512***\u003c/p\u003e \u003cp\u003e(0.224)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFairness \u0026times; UG - Receiver\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-0.876*** (0.031)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e-0.043*** (0.010)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-0.796*** (0.046)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-1.791*** (0.077)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eConstant\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.184***\u003c/p\u003e \u003cp\u003e(0.018)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.000\u003c/p\u003e \u003cp\u003e(0.000)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.032*\u003c/p\u003e \u003cp\u003e(0.014)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.523***\u003c/p\u003e \u003cp\u003e(0.052)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eObservations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e299,578\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100,000\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e99,949\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e99,629\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u0026sup2;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.001\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.017\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.039\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdjusted R\u0026sup2;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.001\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.017\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.039\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"5\"\u003e\u003cem\u003eNote.\u003c/em\u003e Rejection likelihood increases significantly with unfair offers and varies with temperature settings. UG \u0026ndash; Receiver\u0026thinsp;=\u0026thinsp;dummy variable capturing suggestions to the receiver in the Ultimatum Game, Fairness\u0026thinsp;=\u0026thinsp;amount of money to the receiver; *\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.05; **\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.01; ***\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eFurther data exploration and answers to the validation checks\u003c/h2\u003e \u003cp\u003eIn the supplementary material (Material K), we provide the analysis of the GPT\u0026rsquo;s suggestions regarding rejections of very generous offers, i.e., offers the proposer sends the whole 10\u0026euro; to the responder. We included these prompts as a stress test for situations that rarely occur in human interactions. Suggestions in DG Binary showed strong tendencies towards inequality aversion, with suggestions to reject in more than 35% of the cases. This result starkly contrasts with the virtually non-existent aversion to inequality observed for unfair offers. Moreover, suggestions to reject in UG Receiver, where intentions are present, are rarely made \u0026ndash; a result that is consistent with humans rarely rejecting generous offers.\u003c/p\u003e \u003cp\u003e \u003cb\u003eControl Questions.\u003c/b\u003e As for the answers to the control questions, these can help interpret the coherence of GPTs\u0026rsquo; suggestions. Starting with the model answers to the control questions for the DG, the model consistently yielded accurate responses across all ten instances of prompting. In the case of UG Sender, however, invalid responses were observed for the question: \"If person A proposes to send 2 Euros to person B and person B rejects, how much money does person A have in the end?\". Here, the model consistently answered 10\u0026euro; instead of 0\u0026euro;. This response suggests that the model has trouble correctly dealing with the more complicated strategic setting. Interestingly, the answer would have been accurate within the DG context, suggesting that the model suggestions may not fully differentiate the two contexts.\u003c/p\u003e \u003cp\u003eWithin the UG Receiver, we observed several inconsistencies. Specifically, in response to the question: \"If person A proposes to send 2 Euros to person B and person B accepts how much money does person B have in the end?\", the model provided the incorrect answer of 7\u0026euro; eight times and the correct answer of 2\u0026euro; only twice. Additionally, for the question: \"If person A proposes to send 2 Euros to person B and person B rejects how much money does person B have in the end?\u0026rdquo;, the response was incorrect in all instances. Also, the model does not correctly deal with prompts entailing strategic interactions in this case. In these cases, analyzing suggestions is nonetheless informative because when asking for advice, people would probably not be exposed to this type of inconsistency in interpreting the games.\u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study focuses on understanding the fine-grained aspects of social preferences in GPT-3.5 suggestions. Our study builds on existing research by investigating GPT-3.5's alignment with human-like social preferences by testing them using well-established paradigms for social behavior. We extend previous work beyond altruism by examining the nuances of reciprocity and costly punishment. Moreover, we systematically analyzed the influence of demographic and technical parameters with the aim of discerning the extent to which GPT-3.5 suggestions reflect human social preferences and to identify factors that shape its responses.\u003c/p\u003e \u003cp\u003eAs the first main research question we assessed whether GPT-3.5's suggestions reflect strategic considerations related to the risk of rejection, comparing the offers in the DG to those in the UG. Surprisingly, we observed that DG offers exceeded UG offers. This finding contradicts the typical pattern observed in human participants, wherein UG offers tend to be generally higher, driven by the inherent risk of rejection and the associated potential for a 0\u0026ndash;0 outcome\u003csup\u003e41,43,44\u003c/sup\u003e. This finding is largely consistent across gender and age of the advisee \u0026ndash; with deviations only for advisees aged 51\u0026ndash;70 years. Hence, LLM suggestions substantially deviate from typical human behavior in these different social settings. Namely, suggested mean offers in the DG were, on average, much higher (64.4%) than previously observed human behavior (~\u0026thinsp;30%)\u003csup\u003e38\u003c/sup\u003e. This difference underscores the model's notably generous and seemingly altruistic disposition compared to human decision-making. These results do not align with previous findings documenting that model suggestions largely resembled humans' sharing rates in the DG\u003csup\u003e39\u003c/sup\u003e. Instead, they confirm results by Brookins \u0026amp; DeBacker\u003csup\u003e42\u003c/sup\u003e, showing that UG offers aligned more closely with typical human behavior (4\u0026euro; in UG\u003csup\u003e45\u003c/sup\u003e; in model\u0026thinsp;=\u0026thinsp;4.13).\u003c/p\u003e \u003cp\u003eGPT-3.5's suggestions demonstrate a level of generosity that surpasses typical human behavior. Speculatively, there could be a disparity between our written advocacy for the importance of altruism and our actual behavior (we do not \u003cem\u003epractice what we preach\u003c/em\u003e). This tendency towards generosity in the training data could influence the model's predisposition towards more altruistic suggestions the DG. It could further be speculated that an altruistic tendency was deliberately introduced by OpenAI, for example, in the context of reinforcement learning from human feedback, to render the model generally more amicable and prosocial in its suggestions. This is in line with recent work indicating that the newer version of GPT-3.5 (GPT-4) prioritizes the well-being of others over individual well-being (using a similar methodological approach to ours\u003csup\u003e46\u003c/sup\u003e). This does not, however, explain why UG offers resemble typical human behavior. Additionally, the disparity between the LLM's and human offers in the DG contradicts the assumption that research of this nature may inadvertently yield more insights into the survey responses of the reference population rather than unveiling the inherent characteristics of the language models themselves\u003csup\u003e47,48\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eAs a second main question, besides LLMs capturing general notions of strategic risks, our study explored its sensitivity to reciprocity. Therefore, we compared the suggested offers in a DG following a benevolent action of the recipient to the suggested offers in the standard DG Sender, where a benevolent action towards the dictators before the game was absent. Overall, we find mixed evidence of positive reciprocity in the model suggestions. We find that suggested offers increase with the niceness of the other participant's gesture, supporting that GPT-3.5\u0026rsquo;s suggestions reflect human-like reciprocity\u003csup\u003e42\u003c/sup\u003e. However, the fact that in the absence of a generous action (DG Sender), suggestions are more generous than for low levels of niceness (DG Reciprocity) raises some doubts that warrant further consideration regarding the interpretation of these results.\u003c/p\u003e \u003cp\u003eThe disparities in responses between DG with reciprocity and DG Sender may be attributed to contextual differences. In DG with reciprocity, where the recipient's action influences the suggestions, the model demonstrates sensitivity to the perceived generosity of the other participant. In contrast, in DG Sender, which lacks knowledge about the other participant, the LLM indicates a general predisposition towards high altruism, as commented above. This suggests that the model's behavior may vary based on the contextual information available. Notably, the observed pattern aligns with human behavior in sequential DGs. It is, thus, possible that GPT-3.5 interpreted the prompt as a sequential DG.\u003c/p\u003e \u003cp\u003eAs a third question, we assessed whether the model\u0026rsquo;s suggestions capture the costly punishment of unfair offers. We did so by comparing two settings: rejections of unfair outcomes when these are the results of the action of another person (UG Receiver) versus when they are simply allocations one can choose from (Binary DG). Overall, our results indicate that LLM\u0026rsquo;s suggestions to reject reflect the unfairness of the action. Rejection rates increase with the relative unfairness in the UG Receiver setting. Interestingly, we find no support for inequity aversion as rejection rates remained close to zero in all conditions in the binary DG setting.\u003c/p\u003e \u003cp\u003eOur findings generally reveal that rejection suggestions occur less frequently than in human behavior. Namely, rejection rates reached a maximum of 10%, starkly contrasting the 40% rejection rates observed in human participants in the UG\u003csup\u003e49\u003c/sup\u003e. Moreover, LLM\u0026rsquo;s suggestions to reject decreases with lower temperature levels. This novel result suggests that lower temperature levels lead to decisions commonly considered \u0026ldquo;rational\u0026rdquo; in a game-theoretic sense, with suggestions aligned with the maximization of one own monetary payoff. We find a resemblance between the dependence of rejection rates on the perceived unfairness of offers in GPT-3.5 suggestions and the expected patterns exhibited by human counterparts\u003csup\u003e50\u003c/sup\u003e.\u003c/p\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eMethodological considerations\u003c/h2\u003e \u003cp\u003eThe novel approach to letting the LLM complete comprehension check items provides another indication of LLM inconsistencies. The results revealed that the model exhibited limitations in grasping more intricate implications within the UG. However, understanding such nuances may not be a prerequisite for real-life applications. Our analysis indicates that the model's responses remain consistent with what we would expect from a human-generated answer, highlighting the potential implications of model behavior on human behavior, irrespective of an internal understanding of the scenarios on the side of the LLM.\u003c/p\u003e \u003cp\u003eOn a technical note, while the relationship between temperature and rejection rates can be given an emotional interpretation, with suggestions at lower temperatures being cold (i.e., not emotional) and rationally self-interested and suggestions at higher temperatures being war (i.e., emotional) there could be a mechanical explanation for this pattern. Since higher temperatures make the output of the model noisier, in cases where the model is forced to provide only one out of two tokens and the confidence is low enough, one can observe that the rate of the two tokens gets closer to 50% when increasing the temperature. Still, while a technical explanation for the effect can be provided, human recipients of the advice may ascribe emotionality to the underlying motives of the suggestion.\u003c/p\u003e \u003cp\u003eOn a methodological note, we believe that our research provides a foundation for future investigations and offers a benchmark for methodological approaches examining human-like characteristics and behaviors in LLMs. As AI systems continue to play an increasingly significant role in various applications, comprehending their behavior and the factors that influence it becomes paramount to ensure responsible and ethical AI interactions.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eSo, are LLMs good advisors in strategic situations? The answer, as so often, is it depends. For some settings, GPT-3.5 captures nuances of social situations. Overall, we find mixed results regarding the internal consistency of GPT-3.5 suggestions, which in some cases align with well-known behavioral patterns observed in human subjects and in others don\u0026rsquo;t. On the one hand, we find a remarkable degree of internal consistency within single decision settings, with altruism increasing in the niceness of the counterpart's action and with rejection rates in UG being dependent on the fairness of the offer. On the other hand, we document inconsistencies across decision settings, with offers in the presence of risk of rejection being lower than offers without risk of rejection. This result raises a warning regarding the use of single tasks to probe LLM and suggests adopting a multi-task approach, which permits a better assessment of consistency. Our finding that GPT-3-5 does not capture the strategic differences between both settings adds to the (growing) list of contexts where LLMs seem to fall short of human-like performance. Hence, blindly following suggestions by the LLM in strategic situations can also backfire.\u003c/p\u003e "},{"header":"Declarations","content":"\u003cp\u003e \u003ch2\u003eCompeting interests\u003c/h2\u003e \u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eEMS: Conceptualization, Methodology, Data Curation, Data Analysis, Writing \u0026ndash; Original Draft, Writing \u0026ndash; Review \u0026amp; Editing; SB: Software, Validation, Writing \u0026ndash; Review \u0026amp; Editing; NK: Conceptualization, Methodology, Data Curation, Writing \u0026ndash; Review \u0026amp; Editing; IS: Supervision, Data Analysis, Writing \u0026ndash; Review \u0026amp; Editing\u003c/p\u003e\u003ch2\u003eAcknowledgements\u003c/h2\u003e \u003cp\u003eAll authors were supported by the Max Planck Society. Eva-Madeleine Schmidt was supported by the Max Planck School of Cognition.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe datasets generated and/or analysed during the current study are available in the OSF repository, https://osf.io/shq8x/?view_only=52b2aa6871384fc4978dc1ec3100b575.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eHutson, M. Robo-writers: the rise and risks of language-generating AI. Nature 591, 22\u0026ndash;25 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHerbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z. \u0026amp; Trautsch, A. A large-scale comparison of human-written versus ChatGPT-generated essays. Sci. Rep. 13, 1\u0026ndash;11 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eK\u0026ouml;bis, N. C. \u0026amp; Mossink, L. D. Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Comput. Human Behav. 114, 106553 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eB\u0026ouml;hm, R., J\u0026ouml;rling, M., Reiter, L. \u0026amp; Fuchs, C. People devalue generative AI\u0026rsquo;s competence but not its advice in addressing societal and personal challenges. Commun Psychol 1, 1\u0026ndash;10 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKr\u0026uuml;gel, S., Ostermaier, A. \u0026amp; Uhl, M. ChatGPT\u0026rsquo;s inconsistent moral advice influences users\u0026rsquo; judgment. Sci. Rep. 13, 1\u0026ndash;5 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLeib, M., K\u0026ouml;bis, N., Rilke, R. M., Hagens, M. \u0026amp; Irlenbusch, B. Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty. The Economic Journal (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHeaven, W. D. Large language models can do jaw-dropping things. But nobody knows exactly why. \u003cem\u003eTechnology review\u003c/em\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRahwan, I. \u003cem\u003eet al.\u003c/em\u003e Machine behaviour. Nature 568, 477\u0026ndash;486 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJohnson, T. \u0026amp; Obradovich, N. Evidence of behavior consistent with self-interest and altruism in an artificially intelligent agent.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrookins, P. \u0026amp; DeBacker, J. M. Playing Games With GPT: What Can We Learn About a Large Language Model From Canonical Strategic Games? SSRN Journal (2023) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.2139/ssrn.4493398\u003c/span\u003e\u003cspan address=\"10.2139/ssrn.4493398\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBeck, T., Schuff, H., Lauscher, A. \u0026amp; Gurevych, I. Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting. \u003cem\u003earXiv [cs.CL]\u003c/em\u003e 2589\u0026ndash;2615 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchramowski, P., Turan, C., Andersen, N., Rothkopf, C. A. \u0026amp; Kersting, K. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nat Mach Intell 4, 258\u0026ndash;268 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBinz, M. \u0026amp; Schulz, E. Using cognitive psychology to understand GPT-3. \u003cem\u003eProc. Natl. Acad. Sci. U.S.A.\u003c/em\u003e 120, e2218523120 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMiotto, M., Rossberg, N. \u0026amp; Kleinberg, B. Who is GPT-3? An Exploration of Personality, Values and Demographics. \u003cem\u003earXiv [cs.CL]\u003c/em\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHartmann, J., Schwenzow, J. \u0026amp; Witte, M. The political ideology of conversational AI: Converging evidence on ChatGPT\u0026rsquo;s pro-environmental, left-libertarian orientation. \u003cem\u003earXiv [cs.CL]\u003c/em\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHofmann, V., Kalluri, P. R., Jurafsky, D. \u0026amp; King, S. Dialect prejudice predicts AI decisions about people\u0026rsquo;s character, employability, and criminality. \u003cem\u003earXiv [cs.CL]\u003c/em\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJakesch, M., Bhat, A., Buschek, D., Zalmanson, L. \u0026amp; Naaman, M. Co-Writing with Opinionated Language Models Affects Users\u0026rsquo; Views. \u003cem\u003earXiv [cs.HC]\u003c/em\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWeiss, A. \u003cem\u003eet al.\u003c/em\u003e Trust in everyday life. J. Pers. Soc. Psychol. 121, 95\u0026ndash;114 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eColumbus, S., Molho, C., Righetti, F. \u0026amp; Balliet, D. Interdependence and cooperation in daily life. J. Pers. Soc. Psychol. 120, 626\u0026ndash;650 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHofmann, W., Wisneski, D. C., Brandt, M. J. \u0026amp; Skitka, L. J. Morality in everyday life. Science 345, 1340\u0026ndash;1343 (2014).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStrong, J. \u0026lsquo;AI reads human emotions. Should it? MIT Technology Review (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ehttps://\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eonline.ucpress.edu/collabra/article/7/1/19004/116331/Economic-Games-An-Introduction-and-Guide-for\u003c/span\u003e\u003cspan address=\"http://online.ucpress.edu/collabra/article/7/1/19004/116331/Economic-Games-An-Introduction-and-Guide-for\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCrandall, J. W. \u003cem\u003eet al.\u003c/em\u003e Cooperating with machines. Nat. Commun. 9, 233 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWerner, T. Algorithmic and human collusion.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eK\u0026ouml;bis, N. \u003cem\u003eet al.\u003c/em\u003e Artificial Intelligence can facilitate selfish decisions by altering the appearance of interaction partners. \u003cem\u003earXiv [cs.AI]\u003c/em\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKoster, R. \u003cem\u003eet al.\u003c/em\u003e Human-centred mechanism design with Democratic AI. Nat Hum Behav (2022) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41562-022-01383-x\u003c/span\u003e\u003cspan address=\"10.1038/s41562-022-01383-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003evon Schenk, A., Klockmann, V. \u0026amp; K\u0026ouml;bis, N. Social Preferences Toward Humans and Machines: A Systematic Experiment on the Role of Machine Payoffs. Perspect. Psychol. Sci. 17456916231194948 (2023) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/17456916231194949\u003c/span\u003e\u003cspan address=\"10.1177/17456916231194949\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuo, F. GPT in Game Theory Experiments. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2305.05516\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2305.05516\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePhelps, S. \u0026amp; Russell, Y. I. Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2305.07970\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2305.07970\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEngel, C. Dictator games: a meta study. Exp. Econ. 14, 583\u0026ndash;610 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCooper, D. J. \u0026amp; Dutcher, E. G. The dynamics of responder behavior in ultimatum games: a meta-study. Exp Econ 14, 519\u0026ndash;546 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJohnson, N. D. \u0026amp; Mislin, A. A. Trust games: A meta-analysis. Journal of Economic Psychology 32, 865\u0026ndash;889 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEckel, C. C. \u0026amp; Grossman, P. J. CHIVALRY AND SOLIDARITY IN ULTIMATUM GAMES. \u003cem\u003eECONOMIC INQUIRY\u003c/em\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSarlo, M., Lotto, L., Palomba, D., Scozzari, S. \u0026amp; Rumiati, R. Framing the Ultimatum Game: Gender differences and autonomic responses. Int J Psychol 48, 263\u0026ndash;271 (2013).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFornwagner, H., Grosskopf, B., Lauf, A., Sch\u0026ouml;ller, V. \u0026amp; St\u0026auml;dter, S. On the robustness of gender differences in economic behavior. Sci Rep 12, 21549 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKlimecki, O. M., Mayer, S. V., Jusyte, A., Scheeff, J. \u0026amp; Sch\u0026ouml;nenberg, M. Empathy promotes altruistic behavior in economic interactions. Sci. Rep. 6, 1\u0026ndash;5 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNowak, M. A., Page, K. M. \u0026amp; Sigmund, K. Fairness versus reason in the ultimatum game. Science 289, 1773\u0026ndash;1775 (2000).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOosterbeek, H., Sloof, R. \u0026amp; van de Kuilen, G. Cultural differences in ultimatum game experiments: Evidence from a meta-analysis. Exp. Econ. 7, 171\u0026ndash;188 (2004).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBra\u0026ntilde;as-Garza, P., Esp\u0026iacute;n, A. M., Exadaktylos, F. \u0026amp; Herrmann, B. Fair and unfair punishers coexist in the Ultimatum Game. Sci. Rep. 4, 6025 (2014).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSobieszek, A. \u0026amp; Price, T. Playing Games with Ais: The Limits of GPT-3 and Similar Large Language Models. Minds \u0026amp; Machines 32, 341\u0026ndash;364 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCochard, F., Le Gallo, J., Georgantzis, N. \u0026amp; Tisserand, J.-C. Social preferences across different populations: Meta-analyses on the ultimatum game and dictator game. Journal of Behavioral and Experimental Economics 90, 101613 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOosterbeek, H., Sloof, R. \u0026amp; van de Kuilen, G. Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis. Experimental Economics 7, 171\u0026ndash;188 (2004).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eForsythe, R., Horowitz, J. L., Savin, N. E. \u0026amp; Sefton, M. Fairness in simple bargaining experiments. Games and Economic Behavior 6, 347\u0026ndash;369 (1994).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBechler, C., Green, L. \u0026amp; Myerson, J. Proportion offered in the Dictator and Ultimatum Games decreases with amount and social distance. Behav Processes 115, 149\u0026ndash;155 (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRutinowski, J. \u003cem\u003eet al.\u003c/em\u003e The Self-Perception and Political Biases of ChatGPT. \u003cem\u003eHuman Behavior and Emerging Technologies\u003c/em\u003e 2024, 1\u0026ndash;9 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDominguez-Olmedo, R., Hardt, M. \u0026amp; Mendler-D\u0026uuml;nner, C. Questioning the Survey Responses of Large Language Models. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2306.07951\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2306.07951\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDiekmann, A. The Power of Reciprocity: Fairness, Reciprocity, and Stakes in Variants of the Dictator Game. The Journal of Conflict Resolution 48, 487\u0026ndash;505 (2004).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHerne, K., Lappalainen, O. \u0026amp; Kestil\u0026auml;-Kekkonen, E. Experimental Comparison of Direct, General, and Indirect Reciprocity. JOURNAL OF SOCIO-ECONOMICS 45, 38\u0026ndash;46 (2013).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYamagishi, T. \u003cem\u003eet al.\u003c/em\u003e The private rejection of unfair offers and emotional commitment. PNAS Proceedings of the National Academy of Sciences of the United States of America 106, 11520\u0026ndash;11523 (2009).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCharness, G. \u0026amp; Rabin, M. Understanding social preferences with simple tests. Q. J. Econ. 117, 817\u0026ndash;869 (2002).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4611495/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4611495/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003ePre-trained large language models (LLMs) have garnered significant attention for their ability to generate human-like text and responses across various domains. This study delves into the social and strategic behavior of the commonly used LLM GPT-3.5 by investigating its suggestions in well-established behavioral economics paradigms. Specifically, we focus on social preferences, including altruism, reciprocity, and fairness, in the context of two classic economic games: the Dictator Game (DG) and the Ultimatum Game (UG). Our research aims to answer three overarching questions: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) To what extent do GPT-3.5 suggestions reflect human social preferences? (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) How do socio-demographic features of the advisee and (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) technical parameters of the model influence the suggestions of GPT-3.5? We present detailed empirical evidence from extensive experiments with GPT-3.5, analyzing its responses to various game scenarios while manipulating the demographics of the advisee and the model temperature. Our findings reveal that, in the DG, model suggestions are more altruistic than in humans. We further show that it also picks up on more subtle aspects of human social preferences: fairness and reciprocity. This research contributes to the ongoing exploration of AI-driven systems' alignment with human behavior and social norms, providing valuable insights into the behavior of pre-trained LLMs and their implications for human-AI interactions. Additionally, our study offers a methodological benchmark for future research examining human-like characteristics and behaviors in language models.\u003c/p\u003e","manuscriptTitle":"GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-15 17:58:01","doi":"10.21203/rs.3.rs-4611495/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-07-28T06:37:33+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-23T04:38:23+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-22T00:24:30+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-20T02:38:57+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-13T17:40:45+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"203766828489520380038889122167133812084","date":"2024-07-10T16:23:36+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"29595435049011102293119674229377836882","date":"2024-07-09T16:56:06+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"50722446807683795244235636930462290800","date":"2024-07-08T14:09:21+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"92195878303050373720135430166450252652","date":"2024-06-26T23:09:27+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-06-24T23:07:19+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-06-24T22:29:21+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-06-24T18:13:16+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-06-24T08:56:09+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2024-06-20T11:39:05+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"080a17e2-2b1d-454c-aca8-bee58d55d83e","owner":[],"postedDate":"July 15th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":34555693,"name":"Physical sciences/Mathematics and computing/Computer science"},{"id":34555694,"name":"Biological sciences/Psychology"}],"tags":[],"updatedAt":"2024-09-30T16:11:17+00:00","versionOfRecord":{"articleIdentity":"rs-4611495","link":"https://doi.org/10.1038/s41598-024-73306-x","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2024-09-27 15:57:56","publishedOnDateReadable":"September 27th, 2024"},"versionCreatedAt":"2024-07-15 17:58:01","video":"","vorDoi":"10.1038/s41598-024-73306-x","vorDoiUrl":"https://doi.org/10.1038/s41598-024-73306-x","workflowStages":[]},"version":"v1","identity":"rs-4611495","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4611495","identity":"rs-4611495","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}