Development of a Core Outcome Set for Neurological Disorders (COS-Neuro): an AI-Assisted Thematic Framework Analysis

Background Neurological disorders affect approximately 3 billion people globally, yet clinical trial success is often hindered by poorly selected outcome measures, impacting trial design, compliance, and interpretation. Over the past 25 years, Core Outcome Sets (COS) have emerged as standardized tools to enhance outcome selection, ensuring comparability across studies and reflecting the priorities of both researchers and patients. Despite the success of COS initiatives in other fields, their development in neurology remains limited, leaving many trialists without disease-specific guidance.

This study aimed to develop a COS framework for neurological disorders with the assistance of artificial intelligence (AI) by analysing the frequency and scope of outcomes previously reported in existing COS to identify common themes applicable to neurological research.

COS-Neuro was developed using AI-assisted thematic framework analysis, complemented by expert review. A modified five-step thematic analysis was conducted without pre-determined codes: Dataset Gathering – Data was collected from the COMET database, and COS domains for neurological disorders were coded. Prompt Design & Testing – Large language models (LLMs), including ChatGPT 3.5, Google Gemini 1.5 Flash and Meta Llama-2-70b, were trialled, and prompts refined based on their outputs. Thematic Analysis – LLMs categorised domains into core areas. Human Refinement – Experts reviewed LLM-generated core areas and selected those most appropriate for further interpretation. Clinical Validation – Experts validated the domains, core areas, and concepts. This approach integrated AI with expert oversight to develop a standardised COS framework for neurological disorders.

Utilising LLMs, particularly ChatGPT, a robust conceptual framework for COS in neurological disorders was developed, based on the existing 112 existing COS. Through adaptation of the OMERACT model, the final framework comprised four concepts, 13 core areas, and 75 domains, as determined by expert consensus.

COS-Neuro establishes AI-assisted framework for developing COS in neurological disorders. This project provides a foundational resource for future COS research and serves a reference for designing trials in areas where established COS are lacking. Furthermore, it sets a precedent for the integration of AI in qualitative analysis in medicine, demonstrating the scalability of approaches like OMERACT for the development of ‘COS of COS’ across various specialties. Competing Interest Statement The authors have declared no competing interest. Funding Statement The author(s) received no specific funding for this work. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: N/A I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data Availability All relevant data are within the manuscript and its Supporting Information files.