Do physician associates and medical graduates have comparable knowledge? A re-analysis of progress test data

Do physician associates and medical graduates have comparable knowledge? A re-analysis of progress test data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Do physician associates and medical graduates have comparable knowledge? A re-analysis of progress test data Hugh Logan Ellis, Liam Dunnell This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7114807/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Objective To re-evaluate the claim that physician associate (PA) and medical graduates have "comparable knowledge" by analysing the correlation patterns of their performance on a standardised multiple choice question test, rather than relying solely on mean scores. Design Secondary cross-sectional analysis of publicly available data from a single UK institution. Participants Progress test results from 96 PA students (across two stages), 1,195 medical students (stages 1–5), and 65 Foundation Year 1 (FY1) doctors. Main Outcome Measures Pearson correlation coefficients of question-by-question performance between groups, supplemented with mean scores on an 88-item multiple-choice test. Results Despite similar mean scores between second-year PAs and fourth-year medical students, their question-by-question performance showed no correlation. PA performance was also uncorrelated with that of final-year medical students (r = 0.045) and FY1 doctors (r = 0.008), in stark contrast to the strong correlation observed between medical students and FY1 doctors (r = 0.927). These divergent patterns led to extreme performance differences on specific questions. For example, on one question (M3433) PAs scored 89% while final-year medical students scored 5%. Conversely, on another (M3497), PAs scored 2% while medical students scored 95%. Conclusion Relying on mean scores alone to assess knowledge equivalence is misleading. Correlation analysis reveals that the PAs and medical graduates in this cohort possess fundamentally different knowledge structures, demonstrating distinct patterns of strengths and weaknesses. These data do not support the claim that the two professions have comparable knowledge bases. Other Public Policy Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Aging populations, rising demand for care, and increasing costs have placed substantial pressure on health systems in the UK and internationally( 1 ). Doctors are expensive to train and take many years to qualify. In response, the NHS introduced new healthcare roles, including physician associates (PAs), during the early 2000s, with university training programmes starting in 2008( 2 ). Recent workforce plans have proposed expanding the PA role, with the NHS Long-Term Workforce Plan aiming to train 10,000 PAs by 2036/37( 3 ). However, these proposals alongside high-profile concerns voiced in the medical community—have sparked growing debate and public scrutiny about PA training and role expansion ( 4 , 5 ). In response, the government has commissioned an independent review to examine the safety and appropriate scope of PA practice( 6 ) and, the UK government has confirmed that the Workforce Plan will undergo a refresh in summer 2025, aligning with the upcoming 10-Year Health Plan to ensure the NHS workforce model remains current ( 7 ). In this context, a preprint entitled "Physician Associate graduates have comparable knowledge to medical graduates" was published during the review period by Drew-Hill et al.( 8 ) This analysis compared the progress test performance of 96 PA students, 1,195 medical students, and 65 FY1 doctors at Plymouth University, concluding that graduating PA students performed at a level "consistent with" Stage 4 medical students, and that many met or exceeded the performance of new FY1 doctors. The authors interpreted this as evidence that PA graduates possess "comparable knowledge" to medical graduates at the point of graduation. Motivated by curiosity about these findings, we sought to better understand the Drew-Hill et al. data, to look beyond mean scores and at the patterns of question-specific performance across groups. Methods Data Source Drew-Hill et al. made their progress test data from Plymouth University publicly available. Each test originally included 125 multiple-choice questions (MCQ), but due to changes between years, only 88 questions were identical across all groups and included in the analysis. These 88 questions were answered by medical students (years 1–5), PA students (years 1–2), and Foundation Year 1 (FY1) doctors. The tests used negative marking, and the questions were selected to match the official PA curriculum. Medical students took the assessment at the start of each year, while PA students took it at the end. Statistical Analysis We used R (version 4.5.0) with the tidyverse package to analyze the data( 9 , 10 ). We replicated the original cross-sectional analysis by calculating effect sizes (Cohen's d) to compare mean scores between PA Stage 2, Medical Stage 5, and FY1 doctors. Our main analysis examined how similarly different groups answered individual questions. We calculated the average score for each question within each group, then computed Pearson correlations between groups to measure whether they showed similar patterns of correct and incorrect answers. To test whether correlations differed significantly between group comparisons, we used Fisher's z-transformation . Where we found marked differences in performance, we conducted an exploratory analysis examining questions with distinct performance patterns. We identified two contrasting patterns: ( 1 ) questions showing progressive improvement through medical training, and ( 2 ) questions where PAs demonstrated strong performance despite limited progression in medical students. While these analyses were post-hoc and exploratory, they provide insight into how training pathways may develop different knowledge profiles. For pattern 1, we identified questions meeting all of the following criteria: Medical Stage 1 performance 60%, improvement > 30 percentage points, and consistent year-on-year progression. For pattern 2, we identified questions where PA Stage 2 scored > 60%, medical students showed < 30% improvement from Stage 1 to 5, and both Medical Stage 5 and FY1 doctors scored < 40%. Ethical Considerations This secondary analysis used only publicly available, anonymised data. No ethical approval was required. We followed the STROBE guidelines for reporting observational studies( 11 ). Results Overall Performance Patterns Analysis of all 88 questions revealed substantial differences in question-specific performance between PAs and medical students/doctors. PA Stage 2 (n = 42) students achieved lower mean scores than both Medical Stage 5 students (n = 166) (49.92% vs 60.27%, Cohen's d = 1.15) and FY1 doctors (n = 65) (49.92% vs 57.45%, Cohen's d = 0.73). Drew-Hill et al. described these differences as "small-to-medium at most" and emphasised the non-significant difference between PA Stage 2 and Medical Stage 4 (p = 0.370). However, standard interpretation of effect sizes( 12 ) suggests these differences are substantial. For example, a Cohen's d of 1.15 indicates that the average medical graduate performed better than approximately 87% of PA graduates on the assessment. The authors also reported that PA scores increased by 15.11% per stage, compared to 11.87% for medical students, and presented this as evidence of the "intensive nature" of PA programmes. However, this more rapid growth rate still leaves PA graduates well behind medical graduates in absolute terms. Different Patterns of Performance Despite similar overall mean scores between PA Stage 2 (49.92%) and Medical Stage 4 (n = 180) (51.63%), our correlation analysis showed almost no relationship between how the two groups performed on individual questions (r = 0.016, 95% CI: -0.194 to 0.225). Only 23.9% of questions showed a difference of less than 10 percentage points between PAs and medical students. There was also essentially no correlation between PA Stage 2 and either Medical Stage 5 (r = 0.045, 95% CI: -0.166 to 0.252) or FY1 doctors (r = 0.008, 95% CI: -0.202 to 0.217). In contrast, Medical Stage 5 students and FY1 doctors had a very strong correlation in their question-by-question performance (r = 0.927, 95% CI: 0.890 to 0.952). Fisher's z-tests confirmed these correlation differences were statistically significant (both p < 0.0001). Progressive Improvement Questions Of the 88 questions analysed, 30 (34.1%) met our criteria for assessing progressive exam performance. On these questions, Medical Stage 1 students scored 13%, while Medical Stage 5 students achieved 82% and FY1 doctors scored 77%. In contrast, PA Stage 2 students scored 47%. The difference between PA Stage 2 and Medical Stage 5 students was highly significant (p < 0.001, t-test) with a very large effect size (Cohen's d = 1.73). Even compared to FY1 doctors, PAs showed substantial differences (Cohen's d = 1.36). Correlations within these 30 questions showed the same pattern: PA Stage 2 performance showed almost no relationship with either Medical Stage 5 (r = -0.037, 95% CI: -0.392 to 0.328) or FY1 doctors (r = -0.067, 95% CI: -0.417 to 0.301), while Medical Stage 5 and FY1 doctors remained correlated (r = 0.75, 95% CI: 0.534 to 0.874). Characteristics of Questions with Better PA Performance We identified six questions where PA Stage 2 students substantially outperformed both Medical Stage 5 students and FY1 doctors by > 20 percentage points. On these questions, medical students showed minimal progression across training stages. For example, on question M0087 Medical Stage 1 students scored 0%, Medical Stage 5 scored 19%, while PA Stage 2 students achieved 89%. Similar patterns appeared across all six questions, with medical student performance typically improving by less than 20 percentage points from Stage 1 to Stage 5, while PA students consistently scored above 60%. FY1 doctors also performed poorly on these questions, averaging 15% compared to PA Stage 2's 76%. Discussion Principal Findings Our re-analysis of the Drew-Hill et al. data reveals a critical insight that is missed when relying solely on mean exam scores: PA and medical students exhibit fundamentally different patterns of exam performance. While the average scores between second-year PAs and fourth-year medical students were similar, the near-zero correlation in their question-by-question performance (r = 0.016) suggests there is no meaningful relationship in what they get right or wrong. In simple terms, knowing that a PA student answered a question correctly gives no indication as to whether a medical student would have done the same. When comparing question-by-question performance of final-year medical students and FY1 doctors, there is an extremely strong correlation (r = 0.927), demonstrating a consistent pattern in answering exam questions. Of note, the FY1 doctors represented graduates from 20 different UK medical schools, suggesting patterns in answering remain similar across medical schools. The absence of any such relationship between PAs suggests a potential difference in preparation for these examinations. The reason for this is beyond the scope of the data but we theorise this could be due to differences in their educational training, clinical experiences, exam techniques or exam design. Understanding the reasons for these differences would be a valuable area of more work. Implications for Policy and Practice These findings have significant implications for UK health policy, particularly the ongoing expansion of the PA role( 3 ). This expansion is primarily justified as a solution to medical workforce shortages and a way to enhance team-based care( 13 ). The expansion of practice for any professional should be appropriately scrutinised and based on evidence. Concerns have been raised by others over the paucity of evidence examining the clinical competence of PAs( 4 ). This has led to a wider debate about the comparability of their training and skills ( 14 , 15 ). Our analysis, using the only publicly available comparative dataset, challenges Drew-Hill et al’s claims of “comparable knowledge”. It suggests that PA MCQ exam performance is not the same as those who have been through medical school. These data suggest their training may develop a different profile of performance. This is not a criticism of the PA profession itself, but a critique of the evidence provided to inform workforce policy. If these data are generalisable, it challenges the assumption they might be interchangeable with medical graduates, even in supervised roles. Limitations This study has several important limitations. First, the data come from a single university, and its generalisability to all UK PA and medical programmes is uncertain. Second, the PA student sample size was relatively small, which can affect the stability of the findings. Third, as we did not have access to the exam questions, we cannot comment on their clinical importance, question quality or determine the reasons for differing answer patterns. Fourth, our analysis of "progressive knowledge" questions used post-hoc criteria to identify performance patterns. The thresholds chosen were not pre-specified and different criteria might yield different results. Fifth, this is a cross-sectional analysis of performance on an MCQ exam; while exam performance may serve as one indicator of knowledge, it represents a limited measure of what is inherently a complex and multifaceted construct ( 16 ). The knowledge that these exams do assess, likely represents an assessment of student learning, rather than in vivo behaviours and ultimately the results of these on patient care ( 17 ). Therefore, exam performance may only weakly correlate with clinical competence, professionalism, and patient outcomes, and in isolation does not measure all the capabilities required for effective practice( 18 ). Finally, both authors are medically qualified, which may introduce potential bias in interpreting differences between medical and PA training outcomes. This analysis was only possible because the original authors shared their data. Similar transparency from other institutions would help build a more complete picture of PA and medical training outcomes across the UK, aiding policymakers. Conclusion Comparing mean exam scores is an inadequate method for assessing knowledge between different healthcare professionals, and even trying to show equivalence of something as multifaceted as knowledge using only MCQs is likely to be flawed. That said, our correlation analysis demonstrates that PAs and medical graduates actually have fundamentally different profiles of exam question-by-question performance, a fact that mean scores completely obscure. The strong alignment between Plymouth final-year medical students and FY1 doctors from 20 different medical schools suggests a consistency between institutions that is not shared with PA graduates from the same institution. Based on this analysis, the conclusion that PA and medical graduates possess ‘comparable knowledge’ is not supported by the available evidence. This has important implications for future healthcare workforce policy. Declarations Competing Interest Statement All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years. Both HLE and LD are medically qualified; in addition, HLE is a member of the Hospital Consultants and Specialists Association (HCSA) and LD is a member of the British Medical Association (BMA). These professional affiliations could be perceived as a competing interest as both organisations have taken public positions on the topic of this research. Contributors HLE conceived the study, conducted all statistical analyses, and contributed to writing the manuscript. LD provided medical education expertise, contributed to study design, and contributed to writing the manuscript. Both authors revised the manuscript critically for important intellectual content and approved the final version. HLE is the guarantor. Transparency Declaration The lead author (the manuscript's guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained. Ethical Approval Ethical approval was not required as this study used only publicly available, anonymised data. Funding This research received no specific grant from any funding agency. HLE. is supported by a Dalhousie Department of Medicine Research Fellowship and KCL Centre for Doctoral Studies PhD stipend for his doctoral studies, but no specific funding was provided for this analysis. Role of Study Sponsors Not applicable - no specific sponsors for this study. Independence of Researchers from Funders Not applicable - no specific funders for this study. The authors' general academic funding sources had no role in study design, analysis, or manuscript preparation. Patient and Public Involvement Statement No patients or members of the public were involved in the design, conduct, reporting, or dissemination plans of this research as this was a secondary analysis of existing anonymised data. Data Sharing Statement All data used in this analysis are publicly available at https://osf.io/w2q5z/. Analysis code is available at https://github.com/jhfl2/pa-knowledge-comparison-analysis Acknowledgments We thank the authors of Drew-Hill et al. for making their data publicly available, enabling this secondary analysis. References The King’s Fund [Internet] [cited 2025 Jun 13]. The NHS Long-term Plan Explained. Available from: https://www.kingsfund.org.uk/insight-and-analysis/long-reads/nhs-long-term-plan-explained Lynn É (2023) What you need to know about physician associates. BMJ. ;p2840 England NHS NHS England » NHS Long Term Workforce Plan [Internet]. [cited 2025 Jun 13]. Available from: https://www.england.nhs.uk/long-read/nhs-long-term-workforce-plan-2/ Greenhalgh T, McKee M (2025) Physician associates and anaesthetic associates in UK: rapid systematic review of recent UK based research. BMJ 388:e084613 McCartney M (2024) Why the fuss about physician associates? BMJ 385:q862 GOV.UK [Internet] [cited 2025 Jun 13]. New review of physician and anaesthesia associates launched. Available from: https://www.gov.uk/government/news/new-review-of-physician-and-anaesthesia-associates-launched GOV.UK [Internet] [cited 2025 Jun 13]. Government to tackle NHS workforce crisis with refreshed plan. Available from: https://www.gov.uk/government/news/government-to-tackle-nhs-workforce-crisis-with-refreshed-plan Drew-Hill A, Kisielewska J, Edwards J, Evans S, Burr S, Zahra D et al (2025) Physician Associate graduates have comparable knowledge to medical graduates. MedEdPublish 15:20 R: The R Project for Statistical Computing [Internet]. [cited 2025 May 29]. Available from: https://www.r-project.org/ Wickham H, Averick M, Bryan J, Chang W, McGowan LD, François R et al (2019) Welcome to the Tidyverse. J Open Source Softw 4(43):1686 von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370(9596):1453–1457 Sullivan GM, Feinn R (2012) Using Effect Size—or Why the P Value Is Not Enough. J Grad Med Educ 4(3):279–282 Physician and Anaesthesia Associate roles in the NHS – fact sheet – Department of Health and Social Care Media Centre [Internet]. [cited 2025 Jun 13]. Available from: https://healthmedia.blog.gov.uk/2023/11/03/physician-and-anaesthesia-associate-roles-in-the-nhs-fact-sheet/ McKee M, Brayne C (2024) Physician associates in the UK: some fundamental questions that need answers now. BMJ 384:q699 The British Medical Association is the trade union and professional body for doctors in the UK. [Internet]. 2025 [cited 2025 Jun 13]. Physician associates (PAs) and anaesthesia associates (AAs). Available from: https://www.bma.org.uk/advice-and-support/nhs-delivery-and-workforce/workforce/physician-associates-and-anaesthesia-associates Messick S, American Council on Education (1989) Validity. In: Educational measurement, 3rd ed. ; pp. 13–103. (The American Council on Education/Macmillan series on higher education) Kirkpatrick DJ, Kirkpatrick WK An Introduction to The New World Kirkpatrick® Model. Available from: https://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-The-New-World-Kirkpatrick%C2%AE-Model.pdf Miller GE (1990) The assessment of clinical skills/competence/performance. Acad Med J Assoc Am Med Coll 65(9 Suppl):S63–67 Additional Declarations The authors declare potential competing interests as follows: All authors declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years. Both HLE and LD are medically qualified; in addition, HLE is a member of the Hospital Consultants and Specialists Association (HCSA) and LD is a member of the British Medical Association (BMA). These professional affiliations could be perceived as a competing interest as both organisations have taken public positions on the topic of this research. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7114807","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":484723271,"identity":"0e4314e0-3781-4a05-9fef-e35e2344c8ae","order_by":0,"name":"Hugh Logan Ellis","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABGElEQVRIie2RP0vEMBiHf+Wgt1S75pbzK7xy4B84mq+SUmiXG27zBpFONxVvLfglnJwDB3XpcZtkcIiLk0MmcRITOXFpT9wE8wwhb5KH/N4E8Hj+IASEwHxX6TlDHCGQn1W0T6FdJYhhVAG/Uuy8/UE5ZcWDBk2T1U0ZaEFnfNKutcRlYl3ZqZzXswsC5Vn9KAdkg6V3myVJNBloU3YHU7OcgdYZlAiZVcTJNrLBQgna9vTypRwpMXyzCp+snPK+TykapyRkb3EvFtweVJDBUvYHa19cnlwcKzFwwdK6bUim11k06mmf7otnZhZTPlYiMGZxxeMqe9LmNRkftqI7GSKC3UpLxOZ7UfT+imOo3cj7D3g8Hs+/5wPPs14IR/BwPwAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-6428-0158","institution":"","correspondingAuthor":true,"prefix":"","firstName":"Hugh","middleName":"Logan","lastName":"Ellis","suffix":""},{"id":484723272,"identity":"0c66873c-6cd8-4f48-b5c6-995cce540b95","order_by":1,"name":"Liam Dunnell","email":"","orcid":"https://orcid.org/0000-0002-4642-9334","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Liam","middleName":"","lastName":"Dunnell","suffix":""}],"badges":[],"createdAt":"2025-07-13 17:41:04","currentVersionCode":1,"declarations":{"humanSubjects":true,"vertebrateSubjects":false,"conflictsOfInterestStatement":true,"humanSubjectEthicalGuidelines":true,"humanSubjectConsent":true,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7114807/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7114807/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":86773594,"identity":"db913b7c-700a-4b93-9c6a-1786f0bdce0b","added_by":"auto","created_at":"2025-07-15 12:17:13","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":57030,"visible":true,"origin":"","legend":"\u003cp\u003eC\u003cem\u003eorrelations between physician associates in their second year (PA2s), Foundation Year 1 doctors (F1s), and final year medical students (Medical Stage 5) on each question. While the correlation between F1s and final year medical students is very strong (r = 0.927), showing similar question specific exam performance, there was no correlation between PA2s and either group (r = 0.008 with F1s; r = 0.045 with Medical Stage 5), suggesting entirely different patterns question specific exam performance.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"fig1.png","url":"https://assets-eu.researchsquare.com/files/rs-7114807/v1/9fd17d9a6df20d4a2f01fba4.png"},{"id":86774780,"identity":"31fb2ac3-75c7-43a4-88c8-168614584648","added_by":"auto","created_at":"2025-07-15 12:25:13","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":34727,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eAverage performance on questions showing consistent progression by training stage\u003c/em\u003e\u003c/p\u003e","description":"","filename":"fig2.png","url":"https://assets-eu.researchsquare.com/files/rs-7114807/v1/26fedbb7d79101ba9340cb96.png"},{"id":86773598,"identity":"969036e7-c2aa-4d5e-a036-de0142f4933c","added_by":"auto","created_at":"2025-07-15 12:17:13","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":190406,"visible":true,"origin":"","legend":"\u003cp\u003eP\u003cem\u003eerformance on 24 of the 30 questions showing consistent progression. Physician associate students (PA) achieve high scores on certain questions but perform at or below early-stage medical student levels on others.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"fig3.png","url":"https://assets-eu.researchsquare.com/files/rs-7114807/v1/f2a0a6b0eb1af826dc7e2215.png"},{"id":86773595,"identity":"7e144c28-04b0-47c3-a2f0-65554a88d81d","added_by":"auto","created_at":"2025-07-15 12:17:13","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":70120,"visible":true,"origin":"","legend":"\u003cp\u003eA\u003cem\u003eselection of 6 questions where Physician Associates in their second year (PA2s) performed substantially better than Foundation Year 1 doctors (FY1s) and medical students. Medical students showed limited improvement on these questions across different year groups, and FY1s also performed poorly compared to PAs.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"fig4.png","url":"https://assets-eu.researchsquare.com/files/rs-7114807/v1/b4daf329b769b907b6cc4c1a.png"},{"id":86775436,"identity":"e85d363f-bd68-40fb-9e2f-a78c7a9757d7","added_by":"auto","created_at":"2025-07-15 12:33:18","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":885364,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7114807/v1/5213f963-0989-437f-a3c4-483637edbc54.pdf"}],"financialInterests":"The authors declare potential competing interests as follows: All authors declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years. Both HLE and LD are medically qualified; in addition, HLE is a member of the Hospital Consultants and Specialists Association (HCSA) and LD is a member of the British Medical Association (BMA). These professional affiliations could be perceived as a competing interest as both organisations have taken public positions on the topic of this research.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eDo physician associates and medical graduates have comparable knowledge? A re-analysis of progress test data\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eAging populations, rising demand for care, and increasing costs have placed substantial pressure on health systems in the UK and internationally(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). Doctors are expensive to train and take many years to qualify. In response, the NHS introduced new healthcare roles, including physician associates (PAs), during the early 2000s, with university training programmes starting in 2008(\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Recent workforce plans have proposed expanding the PA role, with the NHS Long-Term Workforce Plan aiming to train 10,000 PAs by 2036/37(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). However, these proposals alongside high-profile concerns voiced in the medical community—have sparked growing debate and public scrutiny about PA training and role expansion (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). In response, the government has commissioned an independent review to examine the safety and appropriate scope of PA practice(\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e) and, the UK government has confirmed that the Workforce Plan will undergo a refresh in summer 2025, aligning with the upcoming 10-Year Health Plan to ensure the NHS workforce model remains current (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn this context, a preprint entitled \"Physician Associate graduates have comparable knowledge to medical graduates\" was published during the review period by Drew-Hill et al.(\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e) This analysis compared the progress test performance of 96 PA students, 1,195 medical students, and 65 FY1 doctors at Plymouth University, concluding that graduating PA students performed at a level \"consistent with\" Stage 4 medical students, and that many met or exceeded the performance of new FY1 doctors. The authors interpreted this as evidence that PA graduates possess \"comparable knowledge\" to medical graduates at the point of graduation.\u003c/p\u003e\u003cp\u003eMotivated by curiosity about these findings, we sought to better understand the Drew-Hill et al. data, to look beyond mean scores and at the patterns of question-specific performance across groups.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cb\u003eData Source\u003c/b\u003e\u003c/p\u003e\u003cp\u003eDrew-Hill et al. made their progress test data from Plymouth University publicly available. Each test originally included 125 multiple-choice questions (MCQ), but due to changes between years, only 88 questions were identical across all groups and included in the analysis. These 88 questions were answered by medical students (years 1–5), PA students (years 1–2), and Foundation Year 1 (FY1) doctors. The tests used negative marking, and the questions were selected to match the official PA curriculum. Medical students took the assessment at the start of each year, while PA students took it at the end.\u003c/p\u003e\u003ch2\u003eStatistical Analysis\u003c/h2\u003e\u003cp\u003eWe used R (version 4.5.0) with the tidyverse package to analyze the data(\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eWe replicated the original cross-sectional analysis by calculating effect sizes (Cohen's d) to compare mean scores between PA Stage 2, Medical Stage 5, and FY1 doctors.\u003c/p\u003e\u003cp\u003eOur main analysis examined how similarly different groups answered individual questions. We calculated the average score for each question within each group, then computed Pearson correlations between groups to measure whether they showed similar patterns of correct and incorrect answers. To test whether correlations differed significantly between group comparisons, we used Fisher's z-transformation .\u003c/p\u003e\u003cp\u003eWhere we found marked differences in performance, we conducted an exploratory analysis examining questions with distinct performance patterns. We identified two contrasting patterns: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) questions showing progressive improvement through medical training, and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) questions where PAs demonstrated strong performance despite limited progression in medical students. While these analyses were post-hoc and exploratory, they provide insight into how training pathways may develop different knowledge profiles. For pattern 1, we identified questions meeting all of the following criteria: Medical Stage 1 performance \u0026lt; 50%, Medical Stage 5 performance \u0026gt; 60%, improvement \u0026gt; 30 percentage points, and consistent year-on-year progression. For pattern 2, we identified questions where PA Stage 2 scored \u0026gt; 60%, medical students showed \u0026lt; 30% improvement from Stage 1 to 5, and both Medical Stage 5 and FY1 doctors scored \u0026lt; 40%.\u003c/p\u003e\u003cp\u003e\u003cb\u003eEthical Considerations\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThis secondary analysis used only publicly available, anonymised data. No ethical approval was required. We followed the STROBE guidelines for reporting observational studies(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e).\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cb\u003eOverall Performance Patterns\u003c/b\u003e\u003c/p\u003e\u003cp\u003eAnalysis of all 88 questions revealed substantial differences in question-specific performance between PAs and medical students/doctors. PA Stage 2 (n\u0026thinsp;=\u0026thinsp;42) students achieved lower mean scores than both Medical Stage 5 students (n\u0026thinsp;=\u0026thinsp;166) (49.92% vs 60.27%, Cohen's d\u0026thinsp;=\u0026thinsp;1.15) and FY1 doctors (n\u0026thinsp;=\u0026thinsp;65) (49.92% vs 57.45%, Cohen's d\u0026thinsp;=\u0026thinsp;0.73).\u003c/p\u003e\u003cp\u003eDrew-Hill et al. described these differences as \"small-to-medium at most\" and emphasised the non-significant difference between PA Stage 2 and Medical Stage 4 (p\u0026thinsp;=\u0026thinsp;0.370). However, standard interpretation of effect sizes(\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e) suggests these differences are substantial. For example, a Cohen's d of 1.15 indicates that the average medical graduate performed better than approximately 87% of PA graduates on the assessment. The authors also reported that PA scores increased by 15.11% per stage, compared to 11.87% for medical students, and presented this as evidence of the \"intensive nature\" of PA programmes. However, this more rapid growth rate still leaves PA graduates well behind medical graduates in absolute terms.\u003c/p\u003e\u003cp\u003e\u003cb\u003eDifferent Patterns of Performance\u003c/b\u003e\u003c/p\u003e\u003cp\u003eDespite similar overall mean scores between PA Stage 2 (49.92%) and Medical Stage 4 (n\u0026thinsp;=\u0026thinsp;180) (51.63%), our correlation analysis showed almost no relationship between how the two groups performed on individual questions (r\u0026thinsp;=\u0026thinsp;0.016, 95% CI: -0.194 to 0.225). Only 23.9% of questions showed a difference of less than 10 percentage points between PAs and medical students. There was also essentially no correlation between PA Stage 2 and either Medical Stage 5 (r\u0026thinsp;=\u0026thinsp;0.045, 95% CI: -0.166 to 0.252) or FY1 doctors (r\u0026thinsp;=\u0026thinsp;0.008, 95% CI: -0.202 to 0.217). In contrast, Medical Stage 5 students and FY1 doctors had a very strong correlation in their question-by-question performance (r\u0026thinsp;=\u0026thinsp;0.927, 95% CI: 0.890 to 0.952). Fisher's z-tests confirmed these correlation differences were statistically significant (both p\u0026thinsp;\u0026lt;\u0026thinsp;0.0001).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eProgressive Improvement Questions\u003c/b\u003e\u003c/p\u003e\u003cp\u003eOf the 88 questions analysed, 30 (34.1%) met our criteria for assessing progressive exam performance. On these questions, Medical Stage 1 students scored 13%, while Medical Stage 5 students achieved 82% and FY1 doctors scored 77%. In contrast, PA Stage 2 students scored 47%. The difference between PA Stage 2 and Medical Stage 5 students was highly significant (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001, t-test) with a very large effect size (Cohen's d\u0026thinsp;=\u0026thinsp;1.73). Even compared to FY1 doctors, PAs showed substantial differences (Cohen's d\u0026thinsp;=\u0026thinsp;1.36).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eCorrelations within these 30 questions showed the same pattern: PA Stage 2 performance showed almost no relationship with either Medical Stage 5 (r = -0.037, 95% CI: -0.392 to 0.328) or FY1 doctors (r = -0.067, 95% CI: -0.417 to 0.301), while Medical Stage 5 and FY1 doctors remained correlated (r\u0026thinsp;=\u0026thinsp;0.75, 95% CI: 0.534 to 0.874).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eCharacteristics of Questions with Better PA Performance\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe identified six questions where PA Stage 2 students substantially outperformed both Medical Stage 5 students and FY1 doctors by \u0026gt;\u0026thinsp;20 percentage points. On these questions, medical students showed minimal progression across training stages. For example, on question M0087 Medical Stage 1 students scored 0%, Medical Stage 5 scored 19%, while PA Stage 2 students achieved 89%. Similar patterns appeared across all six questions, with medical student performance typically improving by less than 20 percentage points from Stage 1 to Stage 5, while PA students consistently scored above 60%. FY1 doctors also performed poorly on these questions, averaging 15% compared to PA Stage 2's 76%.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003e\u003cb\u003ePrincipal Findings\u003c/b\u003e\u003c/p\u003e\u003cp\u003eOur re-analysis of the Drew-Hill et al. data reveals a critical insight that is missed when relying solely on mean exam scores: PA and medical students exhibit fundamentally different patterns of exam performance. While the average scores between second-year PAs and fourth-year medical students were similar, the near-zero correlation in their question-by-question performance (r\u0026thinsp;=\u0026thinsp;0.016) suggests there is no meaningful relationship in what they get right or wrong. In simple terms, knowing that a PA student answered a question correctly gives no indication as to whether a medical student would have done the same.\u003c/p\u003e\u003cp\u003eWhen comparing question-by-question performance of final-year medical students and FY1 doctors, there is an extremely strong correlation (r\u0026thinsp;=\u0026thinsp;0.927), demonstrating a consistent pattern in answering exam questions. Of note, the FY1 doctors represented graduates from 20 different UK medical schools, suggesting patterns in answering remain similar across medical schools. The absence of any such relationship between PAs suggests a potential difference in preparation for these examinations. The reason for this is beyond the scope of the data but we theorise this could be due to differences in their educational training, clinical experiences, exam techniques or exam design. Understanding the reasons for these differences would be a valuable area of more work.\u003c/p\u003e\u003cp\u003e\u003cb\u003eImplications for Policy and Practice\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThese findings have significant implications for UK health policy, particularly the ongoing expansion of the PA role(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). This expansion is primarily justified as a solution to medical workforce shortages and a way to enhance team-based care(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). The expansion of practice for any professional should be appropriately scrutinised and based on evidence. Concerns have been raised by others over the paucity of evidence examining the clinical competence of PAs(\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). This has led to a wider debate about the comparability of their training and skills (\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e). Our analysis, using the only publicly available comparative dataset, challenges Drew-Hill et al\u0026rsquo;s claims of \u0026ldquo;comparable knowledge\u0026rdquo;. It suggests that PA MCQ exam performance is not the same as those who have been through medical school. These data suggest their training may develop a different profile of performance. This is not a criticism of the PA profession itself, but a critique of the evidence provided to inform workforce policy. If these data are generalisable, it challenges the assumption they might be interchangeable with medical graduates, even in supervised roles.\u003c/p\u003e\u003cp\u003e\u003cb\u003eLimitations\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThis study has several important limitations. First, the data come from a single university, and its generalisability to all UK PA and medical programmes is uncertain. Second, the PA student sample size was relatively small, which can affect the stability of the findings. Third, as we did not have access to the exam questions, we cannot comment on their clinical importance, question quality or determine the reasons for differing answer patterns. Fourth, our analysis of \"progressive knowledge\" questions used post-hoc criteria to identify performance patterns. The thresholds chosen were not pre-specified and different criteria might yield different results. Fifth, this is a cross-sectional analysis of performance on an MCQ exam; while exam performance may serve as one indicator of knowledge, it represents a limited measure of what is inherently a complex and multifaceted construct (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). The knowledge that these exams do assess, likely represents an assessment of student learning, rather than in vivo behaviours and ultimately the results of these on patient care (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). Therefore, exam performance may only weakly correlate with clinical competence, professionalism, and patient outcomes, and in isolation does not measure all the capabilities required for effective practice(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). Finally, both authors are medically qualified, which may introduce potential bias in interpreting differences between medical and PA training outcomes.\u003c/p\u003e\u003cp\u003eThis analysis was only possible because the original authors shared their data. Similar transparency from other institutions would help build a more complete picture of PA and medical training outcomes across the UK, aiding policymakers.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eComparing mean exam scores is an inadequate method for assessing knowledge between different healthcare professionals, and even trying to show equivalence of something as multifaceted as knowledge using only MCQs is likely to be flawed. That said, our correlation analysis demonstrates that PAs and medical graduates actually have fundamentally \u003cem\u003edifferent\u003c/em\u003e profiles of exam question-by-question performance, a fact that mean scores completely obscure. The strong alignment between Plymouth final-year medical students and FY1 doctors from 20 different medical schools suggests a consistency between institutions that is not shared with PA graduates from the same institution.\u003c/p\u003e\u003cp\u003eBased on this analysis, the conclusion that PA and medical graduates possess \u0026lsquo;comparable knowledge\u0026rsquo; is not supported by the available evidence. This has important implications for future healthcare workforce policy.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eCompeting Interest Statement\u003c/h2\u003e\n\u003cp\u003eAll authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years. Both HLE and LD are medically qualified; in addition, HLE is a member of the Hospital Consultants and Specialists Association (HCSA) and LD is a member of the British Medical Association (BMA). These professional affiliations could be perceived as a competing interest as both organisations have taken public positions on the topic of this research.\u003c/p\u003e\n\u003ch2\u003eContributors\u003c/h2\u003e\n\u003cp\u003eHLE conceived the study, conducted all statistical analyses, and contributed to writing the manuscript. LD provided medical education expertise, contributed to study design, and contributed to writing the manuscript. Both authors revised the manuscript critically for important intellectual content and approved the final version. HLE is the guarantor.\u003c/p\u003e\n\u003ch2\u003eTransparency Declaration\u003c/h2\u003e\n\u003cp\u003eThe lead author (the manuscript\u0026apos;s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.\u003c/p\u003e\n\u003ch2\u003eEthical Approval\u003c/h2\u003e\n\u003cp\u003eEthical approval was not required as this study used only publicly available, anonymised data.\u003c/p\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eThis research received no specific grant from any funding agency. HLE. is supported by a Dalhousie Department of Medicine Research Fellowship and KCL Centre for Doctoral Studies PhD stipend for his doctoral studies, but no specific funding was provided for this analysis.\u003c/p\u003e\n\u003ch2\u003eRole of Study Sponsors\u003c/h2\u003e\n\u003cp\u003eNot applicable - no specific sponsors for this study.\u003c/p\u003e\n\u003ch2\u003eIndependence of Researchers from Funders\u003c/h2\u003e\n\u003cp\u003eNot applicable - no specific funders for this study. The authors\u0026apos; general academic funding sources had no role in study design, analysis, or manuscript preparation.\u003c/p\u003e\n\u003ch2\u003ePatient and Public Involvement Statement\u003c/h2\u003e\n\u003cp\u003eNo patients or members of the public were involved in the design, conduct, reporting, or dissemination plans of this research as this was a secondary analysis of existing anonymised data.\u003c/p\u003e\n\u003ch2\u003eData Sharing Statement\u003c/h2\u003e\n\u003cp\u003eAll data used in this analysis are publicly available at\u003ca href=\"https://osf.io/w2q5z/\"\u003e\u0026nbsp;\u003c/a\u003ehttps://osf.io/w2q5z/. Analysis code is available at\u003ca href=\"https://github.com/jhfl2/pa-knowledge-comparison-analysis\"\u003e\u0026nbsp;\u003c/a\u003ehttps://github.com/jhfl2/pa-knowledge-comparison-analysis\u003c/p\u003e\n\u003ch2\u003eAcknowledgments\u003c/h2\u003e\n\u003cp\u003eWe thank the authors of Drew-Hill et al. for making their data publicly available, enabling this secondary analysis.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eThe King\u0026rsquo;s Fund [Internet] [cited 2025 Jun 13]. The NHS Long-term Plan Explained. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.kingsfund.org.uk/insight-and-analysis/long-reads/nhs-long-term-plan-explained\u003c/span\u003e\u003cspan address=\"https://www.kingsfund.org.uk/insight-and-analysis/long-reads/nhs-long-term-plan-explained\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLynn \u0026Eacute; (2023) What you need to know about physician associates. BMJ. ;p2840\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eEngland NHS NHS England \u0026raquo; NHS Long Term Workforce Plan [Internet]. [cited 2025 Jun 13]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.england.nhs.uk/long-read/nhs-long-term-workforce-plan-2/\u003c/span\u003e\u003cspan address=\"https://www.england.nhs.uk/long-read/nhs-long-term-workforce-plan-2/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGreenhalgh T, McKee M (2025) Physician associates and anaesthetic associates in UK: rapid systematic review of recent UK based research. BMJ 388:e084613\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcCartney M (2024) Why the fuss about physician associates? BMJ 385:q862\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGOV.UK [Internet] [cited 2025 Jun 13]. New review of physician and anaesthesia associates launched. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.gov.uk/government/news/new-review-of-physician-and-anaesthesia-associates-launched\u003c/span\u003e\u003cspan address=\"https://www.gov.uk/government/news/new-review-of-physician-and-anaesthesia-associates-launched\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGOV.UK [Internet] [cited 2025 Jun 13]. Government to tackle NHS workforce crisis with refreshed plan. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.gov.uk/government/news/government-to-tackle-nhs-workforce-crisis-with-refreshed-plan\u003c/span\u003e\u003cspan address=\"https://www.gov.uk/government/news/government-to-tackle-nhs-workforce-crisis-with-refreshed-plan\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDrew-Hill A, Kisielewska J, Edwards J, Evans S, Burr S, Zahra D et al (2025) Physician Associate graduates have comparable knowledge to medical graduates. MedEdPublish 15:20\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eR: The R Project for Statistical Computing [Internet]. [cited 2025 May 29]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.r-project.org/\u003c/span\u003e\u003cspan address=\"https://www.r-project.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWickham H, Averick M, Bryan J, Chang W, McGowan LD, Fran\u0026ccedil;ois R et al (2019) Welcome to the Tidyverse. J Open Source Softw 4(43):1686\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003evon Elm E, Altman DG, Egger M, Pocock SJ, G\u0026oslash;tzsche PC, Vandenbroucke JP (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370(9596):1453\u0026ndash;1457\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSullivan GM, Feinn R (2012) Using Effect Size\u0026mdash;or Why the P Value Is Not Enough. J Grad Med Educ 4(3):279\u0026ndash;282\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePhysician and Anaesthesia Associate roles in the NHS \u0026ndash; fact sheet \u0026ndash; Department of Health and Social Care Media Centre [Internet]. [cited 2025 Jun 13]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://healthmedia.blog.gov.uk/2023/11/03/physician-and-anaesthesia-associate-roles-in-the-nhs-fact-sheet/\u003c/span\u003e\u003cspan address=\"https://healthmedia.blog.gov.uk/2023/11/03/physician-and-anaesthesia-associate-roles-in-the-nhs-fact-sheet/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcKee M, Brayne C (2024) Physician associates in the UK: some fundamental questions that need answers now. BMJ 384:q699\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eThe British Medical Association is the trade union and professional body for doctors in the UK. [Internet]. 2025 [cited 2025 Jun 13]. Physician associates (PAs) and anaesthesia associates (AAs). Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.bma.org.uk/advice-and-support/nhs-delivery-and-workforce/workforce/physician-associates-and-anaesthesia-associates\u003c/span\u003e\u003cspan address=\"https://www.bma.org.uk/advice-and-support/nhs-delivery-and-workforce/workforce/physician-associates-and-anaesthesia-associates\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMessick S, American Council on Education (1989) Validity. In: Educational measurement, 3rd ed. ; pp. 13\u0026ndash;103. (The American Council on Education/Macmillan series on higher education)\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKirkpatrick DJ, Kirkpatrick WK An Introduction to The New World Kirkpatrick\u0026reg; Model. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-The-New-World-Kirkpatrick%C2%AE-Model.pdf\u003c/span\u003e\u003cspan address=\"https://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-The-New-World-Kirkpatrick%C2%AE-Model.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMiller GE (1990) The assessment of clinical skills/competence/performance. Acad Med J Assoc Am Med Coll 65(9 Suppl):S63\u0026ndash;67\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Dalhousie University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7114807/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7114807/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eObjective\u003c/h2\u003e\u003cp\u003eTo re-evaluate the claim that physician associate (PA) and medical graduates have \"comparable knowledge\" by analysing the correlation patterns of their performance on a standardised multiple choice question test, rather than relying solely on mean scores.\u003c/p\u003e\u003ch2\u003eDesign\u003c/h2\u003e\u003cp\u003eSecondary cross-sectional analysis of publicly available data from a single UK institution.\u003c/p\u003e\u003ch2\u003eParticipants\u003c/h2\u003e\u003cp\u003eProgress test results from 96 PA students (across two stages), 1,195 medical students (stages 1\u0026ndash;5), and 65 Foundation Year 1 (FY1) doctors.\u003c/p\u003e\u003ch2\u003eMain Outcome Measures\u003c/h2\u003e\u003cp\u003ePearson correlation coefficients of question-by-question performance between groups, supplemented with mean scores on an 88-item multiple-choice test.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e\u003cp\u003eDespite similar mean scores between second-year PAs and fourth-year medical students, their question-by-question performance showed no correlation. PA performance was also uncorrelated with that of final-year medical students (r\u0026thinsp;=\u0026thinsp;0.045) and FY1 doctors (r\u0026thinsp;=\u0026thinsp;0.008), in stark contrast to the strong correlation observed between medical students and FY1 doctors (r\u0026thinsp;=\u0026thinsp;0.927). These divergent patterns led to extreme performance differences on specific questions. For example, on one question (M3433) PAs scored 89% while final-year medical students scored 5%. Conversely, on another (M3497), PAs scored 2% while medical students scored 95%.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e\u003cp\u003eRelying on mean scores alone to assess knowledge equivalence is misleading. Correlation analysis reveals that the PAs and medical graduates in this cohort possess fundamentally different knowledge structures, demonstrating distinct patterns of strengths and weaknesses. These data do not support the claim that the two professions have comparable knowledge bases.\u003c/p\u003e","manuscriptTitle":"Do physician associates and medical graduates have comparable knowledge? A re-analysis of progress test data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-15 12:17:08","doi":"10.21203/rs.3.rs-7114807/v1","editorialEvents":[{"type":"communityComments","content":2}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"705c55b5-8402-4cf5-ad0e-2e779113a641","owner":[],"postedDate":"July 15th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":51456723,"name":"Other Public Policy"}],"tags":[],"updatedAt":"2025-07-15T12:17:08+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-15 12:17:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7114807","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7114807","identity":"rs-7114807","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}