Can ChatGPT Solve Undergraduate Exams from Logistics Studies? An Investigation

The performance of Large Language Models, such as ChatGPT, generally increases with each further model release. In this paper, we investigate whether and how well different ChatGPT models solve the exams of three different logistics undergraduate courses. We contribute to the discussion of ChatGPT’s existing logistics knowledge, particularly in the field of warehousing. Both, the free version (GPT-4o mini) and the chargeable version (GPT-4o) completed three different logistics exams using three different prompting techniques (with and without role assignment as logistics expert or student). The o1-preview model was also used (without role assignment) for six attempts. The tests were repeated three times. A total of 60 tests were completed and compared to in-class results of logistics students. The results show that a total of 46 tests were passed. The best attempt solved 93% of an exam correctly. Compared to the students from the respective semester, ChatGPT outperforms students in one exam. On the other two exams the students perform better on average.