A survey of Large Language Model use in a hospital, research, and teaching campus

Background The use of Large Language Models (LLMs) has exploded since November 2022 but there is sparse evidence regarding LLM use in health, medical and research contexts.

To summarise the current uses of and attitudes towards LLMs across the clinical, research and teaching contexts in our campus. Design We administered a survey about LLM uses and attitudes. We conducted summary quantitative analysis and inductive qualitative analysis of free text responses. Setting In August-September 2023, we circulated the survey amongst all staff and students across our campus (approximately n=7500), a fully integrated paediatric academic hospital and research institute. Participants We received 281 anonymous survey responses. Main outcome measures We asked about participants’ knowledge of LLMs, their current use of LLMs in professional or learning contexts, and perspectives on possible future uses, opportunities, and risks of LLM use.

Over 90% of respondents have heard of LLM tools and about two-thirds have used them in their work on our campus. Respondents reported using LLMs for a range of uses, including for generating or editing text and exploring ideas. Many, but not necessarily all, respondents seem aware of the limitations and potential risks of LLMs, including privacy and security risks. Various respondents expressed enthusiasm about opportunities of LLM use, including increased efficiency.

Our findings show LLM tools are already widely used on our campus. Guidelines and governance are needed to keep up with practice. We have developed recommendations for the use of LLMs on our campus using insights from this survey. What is known The known: The use of Large Language Models (LLMs) has increased rapidly since the introduction of ChatGPT in November 2022. The new: Most survey respondents are aware of, if not using, LLMs in their work across our hospital, research, and university campus. Diverse uses were reported, including generating or editing text and exploring ideas. There were varying attitudes towards LLMs. Perceived risks included privacy and security risks. A key perceived opportunity was increased efficiency. The implications: LLM tools are already widely used on our campus, highlighting the need for guidelines and governance to keep up with practice. Competing Interest Statement The authors have declared no competing interest. Funding Statement LG was supported by an Australian Government Research Training Program (RTP) Scholarship and MCRI PhD Top Up Scholarship. Research at the Murdoch Children's Research Institute (MCRI) was supported by the Victorian Government's Operational Infrastructure Support Program. The funding organizations are independent of all researchers and were not involved in any of the study design, the collection, analysis, and interpretation of data, the writing of the report or the decision to submit the manuscript for publication. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: This quality improvement project received quality assurance approval from the Royal Children's Hospital Research Ethics & Governance Office (100638). I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data Availability The survey used for data collection is available in Appendix 1. The data that support the findings of this study are available upon reasonable request from the corresponding author, LG. The data are not publicly available due to privacy of the participants.