Investigating the Perceived Unpredictability of Artificial Intelligence: The Role of Temperature and Pre-Prompting

With the growing prevalence of systems that use generative models, understanding how humans perceive these systems is of utmost importance. AI systems are widely characterised as “unpredictable,” potentially eroding user trust, safety, and reliable adoption. Developers routinely tune the temperature hyperparameter of foundation models to manage this risk. It is unclear how the temperature hyperparameter and pre-prompt influence the unpredictability of Foundation Models (FM). This study examined human-AI interactions within a novel experimental context. 141 participants were randomly assigned to varying temperature or pre-prompted conditions across OpenAI and Anthropic models to complete a Wordgame Task using a generalisable AI interface. Results revealed that temperature robustly modulates objective metrics related to unpredictability, such as perplexity and information entropy of output. Contrary to pre-registered hypotheses, human-rated unpredictability was not influenced by temperature. However, manipulating pre-prompt settings yielded strong, convergent relationships between both human-rated unpredictability and other unpredictability metrics across models. Post-hoc generation of FM unpredictability ratings captured statistical regularities with human ratings but revealed more nuanced mapping to the objective predictability of interaction output. Overall, backend prompt engineering is an effective method for shaping human perceptions of AI predictability, not temperature.