{"paper_id":"24eb746d-d234-4e21-a63b-7e18e7f0e5c0","body_text":"Keywords\nHospital-acquired pneumonia, antimicrobial stewardship, diagnosis.\nALL Metrics\n-\nViews\nDownloads\nHow to cite this article\nQuarton S, Baragilly M and Sapey E. Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.13853.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.\nExport Citation\nSciwheel\nEndNote\nRef. Manager\nBibtex\nProCite\nSente\nSelect a format first\n▬\n✚\nStudy Protocol\n[version 1; peer review: 2 approved with reservations]\nSamuel Quarton\nhttps://orcid.org/0009-0005-7235-3424\n1, Mohammed Baragilly https://orcid.org/0000-0003-4775-4742\n2, Elizabeth Sapey1,2Samuel Quarton\nhttps://orcid.org/0009-0005-7235-3424\n1, Mohammed Baragilly https://orcid.org/0000-0003-4775-4742\n2, Elizabeth Sapey1,2 PUBLISHED 23 Apr 2025\nAuthor details Author details\n1 NIHR Birmingham Biomedical Research Centre, Birmingham, England, UK\n2 Department of Health Informatics, University Hospitals Birmingham NHS Foundation Trust, PIONEER Health Data Research Hub in Acute Care, Birmingham, B15 2WB, UK\n2 Department of Health Informatics, University Hospitals Birmingham NHS Foundation Trust, PIONEER Health Data Research Hub in Acute Care, Birmingham, B15 2WB, UK\nSamuel Quarton\nRoles: Conceptualization, Methodology, Writing – Original Draft Preparation\nRoles: Conceptualization, Methodology, Writing – Original Draft Preparation\nMohammed Baragilly\nRoles: Methodology, Writing – Review & Editing\nRoles: Methodology, Writing – Review & Editing\nElizabeth Sapey\nRoles: Methodology, Supervision, Writing – Review & Editing\nRoles: Methodology, Supervision, Writing – Review & Editing\nOPEN PEER REVIEW\nREVIEWER STATUS\nHospital-acquired pneumonia (HAP) is an important complication of hospital admission, with both high incidence and consequences for patients. However, our understanding of causative organisms and prognostic factors is limited. Although ventilator-associated pneumonia (VAP,) an important subset of HAP,has been - extensively investigated, less is known about non-ventilated cases, leading to calls for focused research in this group. This retrospective observational cohort study aims to define a population of patients treated as HAP by comparing ventilated and non-ventilated cases. It aims to clarify how often a microbiological diagnosis is reached, what organisms are frequently identified, and whether this has a relevant impact on the outcomes. The relative impact of positive radiographic changes among patients treated for HAP will also be assessed.\nData will be obtained from the Health Data Research UK acute care hub, ‘. ’PIONEER’ Cases meeting the coding criteria or a clinical surveillance definition of HAP over a 5-year period will be extracted. Demographic, clinical, and microbiological variables will be analysed initially descriptively, and subsequently, with multiple logistic regression analysis to investigate factors affecting microbiological diagnosis. Key outcome variables are in-hospital, 30-day and 1 year mortality, as well as all-cause readmissions within 1 year. Secondary outcomes include nosocomial infections, such as C. difficile. Kaplan-Meier curves and a Cox proportional hazards regression model will be used to investigate outcomes and compare subgroups. A key comparison is between those in whom a putative pathogen is identified and those treated entirely empirically. For this purpose, we will also compare outcomes using an inverse probability of treatment weighting analysis. Additionally, we will explore identifying consolidation in chest imaging reports using natural language processing to allow consideration of the relative impact this may have on mortality and readmission rates.\nPneumonia – a chest infection where areas of the lung get filled with pus and inflammation – can develop among people admitted to hospital for other reasons. When this occurs, we call it ‘hospital-acquired pneumonia’ or HAP. This includes people who get pneumonia whilst on a ventilator (or artificial breathing machine), as well as those who are not, despite potentially important differences between these two groups. There is limited data related to pneumonia in non-ventilated patients, meaning there is uncertainty about which viruses and bacteria commonly cause it, and what factors might affect recovery.\nCurrently, we often don’t find the bacteria or virus causing HAP. This means we can’t give targeted antibiotics, instead having to rely on our ‘best-guess’ treatment, which may not work, or may cause side-effects. We also know that many people are treated as ‘HAP’ despite not having the changes on chest x-ray they would technically need to meet a definition of pneumonia. However it is not clear whether this matters in terms of outcomes, as chest x-rays can be unreliable.\nThis study therefore hopes to:\nLook at any differences between ventilated and non-ventilated patients with pneumonia\nWork out how often the bacteria or virus causing infection is found, and whether this affects how often patients recover.\nCompare patients who have changes on the chest xray with those that don’t, to see if there are any important differences between these groups.\nTo investigate these questions, we will access a database of all patients treated at four hospitals over a 5-year period. We will look at any patients labelled as developing a HAP during their hospital stay. This is known to have limited accuracy, so we will also get patients where their data suggests they had a HAP, even if they were not given a formal label.\nHospital-acquired pneumonia, antimicrobial stewardship, diagnosis.\nCorresponding Author(s)\nSamuel Quarton (s.quarton@bham.ac.uk)\nGrant information: This project is funded by the National Institute for Health and Care Research (NIHR) under its [‘Research for Patient Benefit (RfPB) Programme’ (Grant Reference Number PB-PG-NIHR203326)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.\nThe funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nCopyright: © 2025 Quarton S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Quarton S, Baragilly M and Sapey E. Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.13853.1) First published: 23 Apr 2025, 5:36 (https://doi.org/10.3310/nihropenres.13853.1) Latest published: 15 Sep 2025, 5:36 (https://doi.org/10.3310/nihropenres.13853.2) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nThere is a newer version\nof this article available.\nof this article available.\nAs the most common nosocomial infection1, hospital-acquired pneumonia (HAP) causes significant harm to individuals and the wider healthcare system. Compared to community-acquired pneumonia, the causative organisms vary more with a greater incidence of antimicrobial resistance2,3, and so treatment usually involves the use of broad-spectrum antibiotics, with implications for antimicrobial stewardship. Although the significance of HAP is widely zrecognised, it is poorly defined. Diagnosis is clinical, with varying criteria applied, and no gold-standard test available4. Causative organisms are only identified in a minority of cases, and while case definitions typically require radiological changes in keeping with a pneumonic process5, ythese have been found to be absent in up to half of patients treated with HAP6, possibly because of the high variability in the interpretation of chest radiographs or the insensitivity of the test7,8. Furthermore, HAP encompasses both ventilator-associated pneumonia (VAP) and pneumonia occurring among non-ventilated zhospitalised patients (NV-HAP). Despite these groups having different demographics and risk factors, research has generally not distinguished between these two subgroups or focused solely on VAP. The NV-HAP group, which accounts for the majority of HAP cases, has been relatively understudied, leading to a lack of clarity regarding clinical, microbiological, and prognostic factors9.\nOne difference between NV-HAP and VAP is the relative difficulty in obtaining a microbiological diagnosis among nonventilated patients. Unlike VAP, where bronchoalveolar lavage can be performed more readily, obtaining microbiological specimens in NV-HAP often relies on spontaneous expectoration of sputum. This may not be possiblebecause many of the factors predisposing patients to the development of HAP can also make expectoration more difficult. However, the importance of this remains uncertain. In the absence of an organism to target, treatment consists of empirical antibiotics, which intuitively have several drawbacks compared with the use of organism-guided antibiotics. These include uncertainty that patients’ infections are covered by the antibiotic administered, increased harm from side effects associated with broad-spectrum antibiotics, and increased risk of driving antimicrobial resistance. However, evidence for the benefit of obtaining a microbiological diagnosis in this cohort of patients is currently lacking.\nFurthermore, as already highlighted, cases treated as HAP are often found on retrospective review, not to have the radiological changes needed to meet the current case definitions. However, the importance of this finding remains unclear. It may be that radiological changes have little to add to clinical decision making if patients are otherwise showing signs of a developing chest infection, such as fever, new productive cough, and raised inflammatory markers. This is more plausible considering the reduced negative predictive value of chest radiography when performed in bed-bound patients10, as is frequently the case for HAP. Conversely, the presence or absence of new infiltrates on radiographs may indeed usefully separate the two distinct cohorts of patients with different prognoses. Even if that is the case, one could argue that if patients are being treated pragmatically for HAP regardless of X-ray changes, we should study them on that basis – data related to a case definition that is not routinely applied in clinical practice would have limited relevance.\nTherefore, this retrospective cohort study proposes to consider the following broad questions:\n1. How do clinical and microbiological characteristics differ between VAP and NV-HAP?\n2. How frequently are microbiological causes identified, and is this associated with patient outcomes?\n3. Is there a meaningful difference in clinical features or outcomes between those treated as HAP with or without corresponding changes on radiographs?\nTo address the questions identified above, we will interrogate a large retrospective database of patients treated for HAP (both NV-HAP and VAP), investigating the following specific aims:\n- Characterise the demographic and clinical characteristics of this patient group.\n- Characterise the microbiology of HAP, to include the rates of microbiological sampling, the organisms identified, and whether these were covered by antimicrobialsgiven.\n- Compare changes in the above between patients with NV-HAP and VAP and over time.\n- Factors associated with the likelihood of sending samples for microbiological analysis and with organisms being positively identified.\n- Identify factors influencing prognosis in HAP, including comparing outcomes in patients treated with organism-guided antibiotics with those in whom empirical antibiotics alone were used.\n- Where it can be established through natural language processing of radiological reports, the difference in outcomes among patients with and without consolidation should be assessed.\nThe methods and analysis plan used to address these aims are outlined below, followed by a discussion of the potential limitations of this methodology.\nData is provided from PIONEER, the Health Data Research UK Acute Care Data Hub. PIONEER holds acute healthcare data for patients accessing acute care services from four separate hospitals within the University Hospital Birmingham NHS Trust.\nPatients aged ≥ 18 or over who were treated at any of the UHB sites in a 5-year period between January 2018 and December 2022 were included. This timeframe allows for outcome data to be reviewed for 1-year post-discharge. The following definitions were used to select the patients:\n- Inpatient care episodes were associated with the ICD10 code of J12-18 followed by Y95 or where there was a prior admission in the previous 10 days.\n- SNOMED code of 425464007\nIt has previously been demonstrated that coding data for hospital-acquired pneumonia are often inaccurate. Furthermore, because hospital-acquired pneumonia can occur at any point in the hospital stay after the first 48 hours, identifying the date and time of diagnosis from coding data alone is not possible, limiting the extent of the analysis that can be performed.\nTherefore, a further cohort of patients was also included, approximating a clinical definition of hospital-acquired pneumonia using readily available and routinely collected electronic data. This has been adapted from a previously published surveillance definition11 as follows:\n1. New antibiotic prescription lasting for at least three days (prescribed >48 h after admission)\nAND\n2. Worsening oxygenation within 24hours of this prescription date (defined as a new oxygen requirement, increase in oxygen flow rate, or increase in FIO2 recorded in observations)\nAND\n3. Chest imaging was requested within 24 hours of the antibiotic prescription date. (Plain chest radiography, CT thorax, high-resolution CT thorax, or CT pulmonary angiography).\nPatients under the age of 18 will be excluded; otherwise, there were no specific exclusion criteria. To avoid issues related to the non-independence of the results, where patients meet the inclusion criteria on multiple occasions, the first spell alone will be used.\nAs the period in question covers the Covid-19 pandemic, patients testing positive for covid-19 will be flagged to allow for a sensitivity analysis to be performed, but will not be excluded.\nAs part of the planning for the Birmingham BRC acute care theme, public workshops were held to discuss which infections patients wanted us to focus on, and hospital-acquired infections, including HAP, were identified as a major priority for patients. Patients also identified that they wanted us to perform research on the management of acute infections in hospitals, including creating better tools to identify those at risk of deterioration during infections.\nOnce the specific project was developed in detail, it was presented to members of a support group for patients living with long-term lung conditions who discussed the data that might be needed and provided feedback. Several patients in this meeting had experienced HAP, and those that had not recognized that they were at a higher risk of HAP if they were admitted to the hospital, so their perspective on what to zprioritise within this project was extremely relevant. They emphasised the importance of looking for factors that could be changed or lead to improved outcomes. The resulting data request form was then reviewed by the PIONEER data trust committee, a lay group made up of volunteer members of the public who supported the use of their data for this study.\nWhere the proportion of missing data for a given variable is <10%, cases with missing data will simply be excluded from the analysis. Where the degree of missing data is higher, the randomness of the missing data will be assessed, and, if appropriate, multiple imputation is performed. If the degree of missing data for any variable was greater than 20%, the variable will not be included in the analysis.\nData analysis will be performed using the ‘R’ software package (version 4.3)12. The code used will be saved and published along with the resulting publications.\nAim 1 – Characterising the demographic and clinical characteristics of patients treated for HAP\nPatient demographics and baseline data were graphically visualized and characterized using descriptive statistics. For continuous data, mean and standard deviations will be calculated where data are normally distributed and median and interquartile range where not. Categorical data are presented as proportions/frequencies. The included variables were age, sex, ethnicity, place of residence before admission, comorbidities (including the Charleson comorbidity index), medications on admission, and reasons for admission. In addition to reviewing the proportion of cases associated with specific reasons for admission, these cases will be split into elective and emergency admissions.\nWhere possible, among patients meeting the clinical definition, the date and time of diagnosis will be approximated based on the time of antibiotic prescription. For these patients, the associated vital signs (heart rate, blood pressure, respiratory rate, oxygen saturation, temperature, and mental status) and blood test results (hemoglobin, hematocrit, platelet count, white cell count and differential, CRP, sodium, urea, albumin, and liver enzymes) within 24 h will also be characterized.\nThe outcome measures were all-cause readmission rates within 1 year of discharge, inpatient mortality, and 30-day and 1-year mortality rates. As secondary outcome measures, we will also assess the length of hospital stay and rate of nosocomial infections potentially related to antibiotic use, such as Clostridium difficile. Kaplan-Meier survival curves will also drawn, and log-rank tests performed to compare differences between groups (i.e., between non-ventilated HAP and ventilator-associated pneumonia, and between empirical vs. organism-guided therapy).\nAim 2 – Characterise the microbiology of HAP, to include the rates of microbiological sampling, organisms identified, and whether these were covered by antimicrobials given\nDescriptive statistics will again be used to characterize microbiological sampling rates, including the type of sample used and tests performed (for example, culture or PCR). The antimicrobials used are also described.\nThe proportion of patients in whom a putative pathogen is identified will also be calculated, and the identified pathogens will be compared to the antibiotics that patients received.\nThe proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated. This will be used to simulate the number of patients who could potentially have antibiotics rationalized in different scenarios based on increased organism identification rates.\nAim 3 - Compare changes in the above between patients with NV-HAP and VAP, and over time\nTrends in the clinical and microbiological variables over time were compared based on the year of admission. This is of particular relevance, given the inclusion of the covid-19 pandemic within the selected timeframe. A comparison between VAP and NV-HAP. The chi-squared test will be used for categorical data, and the t-test or Mann-Whitney test will be used for continuous data. Log-rank tests will be performed to compare differences in survival between VAP and NV-HAP based on Kaplan-Meier curves.\nWhere the time of diagnosis can be approximated, VAP will be defined as any case where the onset of pneumonia occurs >48hours after ventilation. Where this is not possible (for example, where patients are included based on coding data alone), they will be labelled as VAP if they were ventilated at any point during admission.\nAim 4 - Determine factors associated with the likelihood of sending samples for microbiological analysis, and with organisms being positively identified\nMultiple logistic regression analysis will be performed to identify factors associated with both sending microbiological samples for analysis and obtaining positive microbiology results among those sent.\nA stepwise regression approach will be used to identify the most important predictors, with co-variates considered, including those identified in aim 1, together with the setting in which pneumonia occurred (surgical ward, medical ward, or intensive care unit) and time of diagnosis (split between ‘in hours’ (between the hours of 8 and 5)’ and ‘out of hours.’ It is hypothesized that the factors affecting the sending of samples include both patient and healthcare setting factors. Patient factors may include the ability of patients to produce samples and the severity of illness, while system factors may include prioritization, staffing levels, and experience within a unit. These co-variates, while unlikely to be comprehensive, have been selected as likely related to the ability to produce samples, the severity of infection, or system-related variability.\nAim 5 - Identify factors influencing prognosis in HAP, including comparing outcomes in patients treated with organism-guided antibiotics vs those in whom empirical antibiotics alone were used\nThe outcome measures used to assess prognostic factors were mortality within 3 and 12 months and readmission rates within 12 months. All cause readmissions have been chosen rather than those related to infection only, as the potential sequelae of HAP include a generalized increase in frailty that may lead to admission for many different reasons.\nIn each case, a Cox proportional hazard regression model will be used to investigate the prognostic factors. The factors used as covariates are those identified in aim A, as well as the presence or absence of a microbiological diagnosis as a further categorical variable.\nThe predictive performance of the existing severity scores will be assessed using the area under the receiver operating characteristic curves. The scores to be assessed included pneumonia-specific scores (CURB-65 and PSI) and general severity scores (NEWS2 and SOFA). Waterlow score is a clinical tool used to predict the development of pressure ulcers and is routinely recorded for all inpatients. However, many of its components (such as age, mobility status, and neurological deficit) are likely to impact the prognosis of patients with HAP; therefore, the Waterlow score will also be investigated for its prognostic value.\nIn particular, we focused on the relative impact of obtaining a microbiological diagnosis on the outcome. However, this was influenced by several confounding variables. As a further analysis of the potential impact of this on prognosis, propensity scores will be calculated, and inverse probability of treatment weighting is used to create a pseudo-population in which confounding variables are distributed equally between the control and treatment arms. Covariate balance between the groups will be assessed after adjustment using standardized mean differences. The Co-variates used for propensity scoring will be those used in Aim 4. Differences in outcome measures will then be compared between the groups.\nAim 6 - Assess - where it can be established through natural language processing of radiological reports - the difference in outcomes among patients with consolidation and those without\nIf natural language processing of radiology reports can identify patients in whom consolidation was present, a sensitivity analysis of the impact of this on mortality and readmission rates should be performed.\nThis protocol provides the basis for insight into an important and understudied disease. Access to the Health Data Research UK PIONEER database provides a large volume of data, with preliminary searches indicating a population of several thousand patients treated for HAP within the 5 year-frame specified. This is combined with the high granularity of data, allowing detailed analyses to be conducted. However, there are several limitations inherent in the study design that may lead to bias in the results.\nFirst, and common to much research on HAP, there is uncertainty that all included patients do indeed have HAP. This arises because of the difficulty in making a clinical diagnosis of HAP as well as limitations within coding data. It is well known that coding data for HAP vehas high rates of both false positives and false negatives6,13,14. The included clinical definition aims to mitigate this by identifying additional cases, and a similar surveillance definition was previously found to have incidence and mortality comparable to estimates of HAP from manual case reviews11. However, this clinical definition still does not help to exclude false positives, and it is acknowledged that there will be significant heterogeneity within the identified cohort. This is not an insignificant limitation. However, this also reflects data available in practice.\nSecond, coding data did not provide the date or time of diagnosis. An episode of HAP can occur at any point during hospital admission, and patients’ riskfactors, medications, and clinical status willvary significantly during that time. Without the date and time of diagnosis, it is not possible to incorporate many variables that might have an impact on outcomes. Therefore, these it will be necessary to include only patients meeting the clinical surveillance criteria, where the time of diagnosis can be estimated based on the recorded times of prescriptions. This reduces the power of the analysis and may reflect a biased subset of HAP patients if those meeting the surveillance criteria are not representative of the entire HAP population.\nThird, while the PIONEER data hub provides access to a large volume of data from a diverse group of patients, all patients are treated within one large NHS trust encompassing four urban hospitals in the UK. This may limit the applicability of findings to sites in other settings or countries, where clinical practice, antimicrobial guidelines, zorganisational culture, and access to investigations may vary.\nThis study will use unconsented, anonymous health data, with the need for individual participant consent waived by the ethical approval committee. All study activity is approved by the East Midlands–Derby REC (reference: 20/EM/0158). Specific approvals have been provided by East Midlands–Derby REC (reference: 20/EM/0158) to use unconsented, anonymized health data. The project was also reviewed and approved by the PIONEER Data Trust Committee. Ethical approval was gained on 28/06/2024.\nThe results will be submitted for publication in peer-reviewed journals and for presentations at national and international conferences.\nNo data are associated with this article.\nFollowing study completion, data access will be provided by PIONEER and can be accessed through a data access request by contacting pioneer@uhb.nhs.uk.\nThe authors would like to thank the members of “Breathe-Easy Redditch”, and the PIONEER data trust committee for their support in developing this project.\nFaculty Opinions recommendedReferences\n- 1. Mitchell BG, Russo PL, Cheng AC, et al.: Strategies to reduce Non-Ventilator-associated Hospital-Acquired Pneumonia: a systematic review. Infect Dis Health. 2019; 24(4): 229–239. PubMed Abstract | Publisher Full Text\n- 2. Wattoo MA, Tabassum M, Bhutta KR, et al.: Clinical and microbiological analysis of Hospital-Acquired Pneumonia among patients with ischemic stroke: a retrospective outlook. Cureus. 2021; 13(5): e15214. PubMed Abstract | Publisher Full Text | Free Full Text\n- 3. Feng DY, Zhou YQ, Zou XL, et al.: Differences in microbial etiology between Hospital-Acquired Pneumonia and Ventilator-Associated Pneumonia: a single-center retrospective study in Guang Zhou. Infect Drug Resist. 2019; 12: 993–1000. PubMed Abstract | Publisher Full Text | Free Full Text\n- 4. Quarton S, Livesey A, Pittaway H, et al.: Clinical challenge of diagnosing Non-Ventilator Hospital-Acquired Pneumonia and identifying causative pathogens: a narrative review. J Hosp Infect. 2024; 149: 189–200. PubMed Abstract | Publisher Full Text\n- 5. Kalil AC, Metersky ML, Klompas M, et al.: Management of adults with Hospital-Acquired and Ventilator-Associated Pneumonia: 2016 clinical practice guidelines by the Infectious Diseases Society of America and the American Thoracic Society. Clin Infect Dis. 2016; 63(5): e61–e111. PubMed Abstract | Publisher Full Text | Free Full Text\n- 6. Burton LA, Price R, Barr KE, et al.: Hospital-Acquired Pneumonia incidence and diagnosis in older patients. Age Ageing. 2016; 45(1): 171–4. PubMed Abstract | Publisher Full Text\n- 7. Albaum MN, Hill LC, Murphy M, et al.: Interobserver reliability of the chest radiograph in Community-Acquired Pneumonia. PORT Investigators. Chest. 1996; 110(2): 343–50. PubMed Abstract | Publisher Full Text\n- 8. Singh B, Curtis J, Gordon SB, et al.: P251 Junior doctors' interpretation of CXRs is more consistent than consultants in the context of possible pneumonia. Thorax. 2011; 66(Suppl 4): A169–A70. Publisher Full Text\n- 9. Munro SC, Baker D, Giuliano KK, et al.: Nonventilator Hospital-Acquired Pneumonia: a call to action: recommendations from the National Organization to Prevent Hospital-Acquired Pneumonia (NOHAP) among nonventilated patients. Infect Control Hosp Epidemiol. 2021; 42(8): 991–996. PubMed Abstract | Publisher Full Text | Free Full Text\n- 10. Esayag Y, Nikitin I, Bar-Ziv J, et al.: Diagnostic value of chest radiographs in bedridden patients suspected of having pneumonia. Am J Med. 2010; 123(1): 88.e1–5. PubMed Abstract | Publisher Full Text\n- 11. Ji W, McKenna C, Ochoa A, et al.: Development and assessment of objective surveillance definitions for Nonventilator Hospital-Acquired Pneumonia. JAMA Netw Open. 2019; 2(10): e1913674. PubMed Abstract | Publisher Full Text | Free Full Text\n- 12. R Core Team: R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2021. Reference Source\n- 13. Ewan VC, Witham MD, Kiernan M, et al.: Hospital-Acquired Pneumonia surveillance-an unmet need. Lancet Respir Med. 2017; 5(10): 771–772. PubMed Abstract | Publisher Full Text\n- 14. Wolfensberger A, Meier AH, Kuster SP, et al.: Should International Classification of Diseases codes be used to survey Hospital-Acquired Pneumonia? J Hosp Infect. 2018; 99(1): 81–84. PubMed Abstract | Publisher Full Text\nAuthor details Author details\n1 NIHR Birmingham Biomedical Research Centre, Birmingham, England, UK\n2 Department of Health Informatics, University Hospitals Birmingham NHS Foundation Trust, PIONEER Health Data Research Hub in Acute Care, Birmingham, B15 2WB, UK\n2 Department of Health Informatics, University Hospitals Birmingham NHS Foundation Trust, PIONEER Health Data Research Hub in Acute Care, Birmingham, B15 2WB, UK\nSamuel Quarton\nRoles: Conceptualization, Methodology, Writing – Original Draft Preparation\nRoles: Conceptualization, Methodology, Writing – Original Draft Preparation\nMohammed Baragilly\nRoles: Methodology, Writing – Review & Editing\nRoles: Methodology, Writing – Review & Editing\nElizabeth Sapey\nRoles: Methodology, Supervision, Writing – Review & Editing\nRoles: Methodology, Supervision, Writing – Review & Editing\nCompeting interests\nNo competing interests were disclosed.\nGrant information\nThis project is funded by the National Institute for Health and Care Research (NIHR) under its [‘Research for Patient Benefit (RfPB) Programme’ (Grant Reference Number PB-PG-NIHR203326)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.\nThe funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nThe funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nArticle Versions (2)\nCopyright\n© 2025 Quarton S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.\nmetrics\nVIEWS\n$counts.viewCount\ndownloads\nCitations\nCITE\nhow to cite this article\nQuarton S, Baragilly M and Sapey E. Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.13853.1)\nNOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.\ntrack\nreceive updates on this article\nTrack an article to receive email alerts on any updates to this article.\nCurrent Reviewer Status: ?\nKey to Reviewer Statuses VIEW HIDE\nApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested\nApproved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.\nNot approvedFundamental flaws in the paper seriously undermine the findings and conclusions\nVersion 1\nVERSION 1\nPUBLISHED 23 Apr 2025 Views\n0\nHow to cite this report:\nKolar M. Reviewer Report For: Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.15054.r36716) The direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-36716\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-36716\nNOTE: it is important to ensure the information in square brackets after the title is included in this citation.\nReviewer Report 04 Sep 2025\nApproved with Reservations\nVIEWS 0\nThis retrospective study analyzes patients treated for hospital-acquired pneumonia (HAP), distinguishing between ventilator-associated (VAP) and non-ventilator-associated (NV-HAP) forms. The topic is highly relevant and clinically significant, particularly in light of the increasing prevalence of antimicrobial resistance (AMR) and the need ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close\nThis retrospective study analyzes patients treated for hospital-acquired pneumonia (HAP), distinguishing between ventilator-associated (VAP) and non-ventilator-associated (NV-HAP) forms. The topic is highly relevant and clinically significant, particularly in light of the increasing prevalence of antimicrobial resistance (AMR) and the need to optimize diagnostic and therapeutic strategies. Importantly, inadequately treated HAP may progress to sepsis and septic shock. I believe the study has the potential to provide valuable insights for clinical practice and further research.\nStrengths of the study:\nComments and Recommendations:\n1. Microbiological protocol\n2. Antibiotic treatment and AMR\n3. Inflammatory markers\n4. NV-HAP vs. VAP\nConclusion:\nThe study represents a significant contribution to the understanding of HAP, particularly NV-HAP. To enhance its clinical relevance, I recommend incorporating the above points into the study protocol.\nStrengths of the study:\n- The focus on NV-HAP, which remains underexplored, is highly beneficial.\n- The use of the extensive PIONEER database enables analysis of a large patient cohort with HAP.\n- The study is methodologically well-designed, including appropriate statistical methods and chest-imaging based outcome analysis.\nComments and Recommendations:\n1. Microbiological protocol\n- The types of clinical samples to be evaluated are not clearly specified. For VAP, lower respiratory tract specimens (e.g., endotracheal aspirate, BAL) are assumed, but this should be explicitly stated. What types of samples will be analyzed in NV-HAP? How will clinically significant findings be distinguished from colonization? I recommend clarifying the criteria for evaluating microbiological results.\n- PCR test results are mentioned in the protocol. The authors should specify which results and from which clinical materials.\n- I recommend to consider including positive blood cultures as a potential diagnostic criterion.\n- The definition of a “putative pathogen” should be clarified, including criteria for distinguishing contaminants from true bacterial pathogens.\n2. Antibiotic treatment and AMR\n- The study should include not only the types of antibiotics used but also their dosages, as this can significantly influence treatment outcomes and resistance development.\n- The term “antibiotic rationalization” should be clearly defined—does it refer to de-escalation, targeted therapy, or treatment discontinuation?\n- I recommend analyzing bacterial resistance and its impact on HAP treatment outcomes. Also, assess the effect of inadequate initial antibiotic therapy (i.e., pathogen resistant to the initial treatment) compared to adequate initial therapy (i.e., pathogen susceptible to the administered antibiotic).\n3. Inflammatory markers\n- I recommend expanding the protocol to include monitoring of inflammatory markers, particularly procalcitonin (PCT), which may aid in decisions regarding initiation and discontinuation of antibiotic therapy.\n- Evaluating the dynamics of CRP, leukocytosis, and other laboratory parameters could contribute to better patient stratification.\n4. NV-HAP vs. VAP\n- The study appropriately distinguishes between these two groups but should further reflect differences in diagnostic method availability and sampling feasibility.\n- Consider evaluating access to specialized procedures (e.g., bronchoscopy) as a systemic factor influencing diagnostic accuracy.\nConclusion:\nThe study represents a significant contribution to the understanding of HAP, particularly NV-HAP. To enhance its clinical relevance, I recommend incorporating the above points into the study protocol.\n-\nIs the rationale for, and objectives of, the study clearly described?\nYes\n-\nIs the study design appropriate for the research question?\nYes\n-\nAre sufficient details of the methods provided to allow replication by others?\nPartly\n-\nAre the datasets clearly presented in a useable and accessible format?\nPartly\nCompeting Interests: No competing interests were disclosed.\nReviewer Expertise: Clinical microbiology, AMR, bacterial infections, antibiotic therapy\nCITE\nHOW TO CITE THIS REPORT Kolar M. Reviewer Report For: Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.15054.r36716)\nThe direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-36716\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-36716\nNOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.\n- Author Response 15 Sep 2025Samuel Quarton, NIHR Birmingham Biomedical Research Centre, Birmingham, UK15 Sep 2025Author ResponseMany thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study will be beneficial, and we value that you have ... Continue reading Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study will be beneficial, and we value that you have highlighted some of the strengths of the proposed study.\nPlease see below for our responses to your specific comments, together with the changes made in light of these. The updated version of the text has been submitted and will be visible for review shortly.\n1. Microbiological protocol- The types of clinical samples to be evaluated are not clearly specified. For VAP, lower respiratory tract specimens (e.g., endotracheal aspirate, BAL) are assumed, but this should be explicitly stated. What types of samples will be analyzed in NV-HAP? How will clinically significant findings be distinguished from colonization? I recommend clarifying the criteria for evaluating microbiological results.\nDistinguishing findings from colonisation is a challenge across pneumonia diagnosis generally and would not be possible to accurately perform without a case-by-case analysis of other clinical and radiological results for each patient, which would not be feasible across the dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. (Where there is mixed growth or upper-respiratory tract organisms identified only, these are reported as such in our hospital, rather than to a species level. The text has been updated to reinforce these are potential pathogens only, and may represent colonisation.\n- PCR test results are mentioned in the protocol. The authors should specify which results and from which clinical materials.\n- I recommend to consider including positive blood cultures as a potential diagnostic criterion.\n- The definition of a “putative pathogen” should be clarified, including criteria for distinguishing contaminants from true bacterial pathogens.\n- The study should include not only the types of antibiotics used but also their dosages, as this can significantly influence treatment outcomes and resistance development.\n- The term “antibiotic rationalization” should be clearly defined—does it refer to de-escalation, targeted therapy, or treatment discontinuation?\n- I recommend analyzing bacterial resistance and its impact on HAP treatment outcomes. Also, assess the effect of inadequate initial antibiotic therapy (i.e., pathogen resistant to the initial treatment) compared to adequate initial therapy (i.e., pathogen susceptible to the administered antibiotic).\n- I recommend expanding the protocol to include monitoring of inflammatory markers, particularly procalcitonin (PCT), which may aid in decisions regarding initiation and discontinuation of antibiotic therapy.\n- Evaluating the dynamics of CRP, leukocytosis, and other laboratory parameters could contribute to better patient stratification.\n4. NV-HAP vs. VAP- The study appropriately distinguishes between these two groups but should further reflect differences in diagnostic method availability and sampling feasibility.\n- Consider evaluating access to specialized procedures (e.g., bronchoscopy) as a systemic factor influencing diagnostic accuracy.\nHave added statements around availability of bronchoscopy, and the variability in access to diagnostic approaches among NVHAP and VAP.\nMany thanks again for your time and consideration in reviewing our manuscript.Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study will be beneficial, and we value that you have highlighted some of the strengths of the proposed study.Competing Interests: No competing interests were disclosed. Close\nPlease see below for our responses to your specific comments, together with the changes made in light of these. The updated version of the text has been submitted and will be visible for review shortly.\n1. Microbiological protocol- The types of clinical samples to be evaluated are not clearly specified. For VAP, lower respiratory tract specimens (e.g., endotracheal aspirate, BAL) are assumed, but this should be explicitly stated. What types of samples will be analyzed in NV-HAP? How will clinically significant findings be distinguished from colonization? I recommend clarifying the criteria for evaluating microbiological results.\nDistinguishing findings from colonisation is a challenge across pneumonia diagnosis generally and would not be possible to accurately perform without a case-by-case analysis of other clinical and radiological results for each patient, which would not be feasible across the dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. (Where there is mixed growth or upper-respiratory tract organisms identified only, these are reported as such in our hospital, rather than to a species level. The text has been updated to reinforce these are potential pathogens only, and may represent colonisation.\n- PCR test results are mentioned in the protocol. The authors should specify which results and from which clinical materials.\n- I recommend to consider including positive blood cultures as a potential diagnostic criterion.\n- The definition of a “putative pathogen” should be clarified, including criteria for distinguishing contaminants from true bacterial pathogens.\n- The study should include not only the types of antibiotics used but also their dosages, as this can significantly influence treatment outcomes and resistance development.\n- The term “antibiotic rationalization” should be clearly defined—does it refer to de-escalation, targeted therapy, or treatment discontinuation?\n- I recommend analyzing bacterial resistance and its impact on HAP treatment outcomes. Also, assess the effect of inadequate initial antibiotic therapy (i.e., pathogen resistant to the initial treatment) compared to adequate initial therapy (i.e., pathogen susceptible to the administered antibiotic).\n- I recommend expanding the protocol to include monitoring of inflammatory markers, particularly procalcitonin (PCT), which may aid in decisions regarding initiation and discontinuation of antibiotic therapy.\n- Evaluating the dynamics of CRP, leukocytosis, and other laboratory parameters could contribute to better patient stratification.\n4. NV-HAP vs. VAP- The study appropriately distinguishes between these two groups but should further reflect differences in diagnostic method availability and sampling feasibility.\n- Consider evaluating access to specialized procedures (e.g., bronchoscopy) as a systemic factor influencing diagnostic accuracy.\nHave added statements around availability of bronchoscopy, and the variability in access to diagnostic approaches among NVHAP and VAP.\nMany thanks again for your time and consideration in reviewing our manuscript.\nCOMMENTS ON THIS REPORT\n- Author Response 15 Sep 2025Samuel Quarton, NIHR Birmingham Biomedical Research Centre, Birmingham, UK15 Sep 2025Author ResponseMany thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study will be beneficial, and we value that you have ... Continue reading Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study will be beneficial, and we value that you have highlighted some of the strengths of the proposed study.\nPlease see below for our responses to your specific comments, together with the changes made in light of these. The updated version of the text has been submitted and will be visible for review shortly.\n1. Microbiological protocol- The types of clinical samples to be evaluated are not clearly specified. For VAP, lower respiratory tract specimens (e.g., endotracheal aspirate, BAL) are assumed, but this should be explicitly stated. What types of samples will be analyzed in NV-HAP? How will clinically significant findings be distinguished from colonization? I recommend clarifying the criteria for evaluating microbiological results.\nDistinguishing findings from colonisation is a challenge across pneumonia diagnosis generally and would not be possible to accurately perform without a case-by-case analysis of other clinical and radiological results for each patient, which would not be feasible across the dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. (Where there is mixed growth or upper-respiratory tract organisms identified only, these are reported as such in our hospital, rather than to a species level. The text has been updated to reinforce these are potential pathogens only, and may represent colonisation.\n- PCR test results are mentioned in the protocol. The authors should specify which results and from which clinical materials.\n- I recommend to consider including positive blood cultures as a potential diagnostic criterion.\n- The definition of a “putative pathogen” should be clarified, including criteria for distinguishing contaminants from true bacterial pathogens.\n- The study should include not only the types of antibiotics used but also their dosages, as this can significantly influence treatment outcomes and resistance development.\n- The term “antibiotic rationalization” should be clearly defined—does it refer to de-escalation, targeted therapy, or treatment discontinuation?\n- I recommend analyzing bacterial resistance and its impact on HAP treatment outcomes. Also, assess the effect of inadequate initial antibiotic therapy (i.e., pathogen resistant to the initial treatment) compared to adequate initial therapy (i.e., pathogen susceptible to the administered antibiotic).\n- I recommend expanding the protocol to include monitoring of inflammatory markers, particularly procalcitonin (PCT), which may aid in decisions regarding initiation and discontinuation of antibiotic therapy.\n- Evaluating the dynamics of CRP, leukocytosis, and other laboratory parameters could contribute to better patient stratification.\n4. NV-HAP vs. VAP- The study appropriately distinguishes between these two groups but should further reflect differences in diagnostic method availability and sampling feasibility.\n- Consider evaluating access to specialized procedures (e.g., bronchoscopy) as a systemic factor influencing diagnostic accuracy.\nHave added statements around availability of bronchoscopy, and the variability in access to diagnostic approaches among NVHAP and VAP.\nMany thanks again for your time and consideration in reviewing our manuscript.Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study will be beneficial, and we value that you have highlighted some of the strengths of the proposed study.Competing Interests: No competing interests were disclosed. Close\nPlease see below for our responses to your specific comments, together with the changes made in light of these. The updated version of the text has been submitted and will be visible for review shortly.\n1. Microbiological protocol- The types of clinical samples to be evaluated are not clearly specified. For VAP, lower respiratory tract specimens (e.g., endotracheal aspirate, BAL) are assumed, but this should be explicitly stated. What types of samples will be analyzed in NV-HAP? How will clinically significant findings be distinguished from colonization? I recommend clarifying the criteria for evaluating microbiological results.\nDistinguishing findings from colonisation is a challenge across pneumonia diagnosis generally and would not be possible to accurately perform without a case-by-case analysis of other clinical and radiological results for each patient, which would not be feasible across the dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. (Where there is mixed growth or upper-respiratory tract organisms identified only, these are reported as such in our hospital, rather than to a species level. The text has been updated to reinforce these are potential pathogens only, and may represent colonisation.\n- PCR test results are mentioned in the protocol. The authors should specify which results and from which clinical materials.\n- I recommend to consider including positive blood cultures as a potential diagnostic criterion.\n- The definition of a “putative pathogen” should be clarified, including criteria for distinguishing contaminants from true bacterial pathogens.\n- The study should include not only the types of antibiotics used but also their dosages, as this can significantly influence treatment outcomes and resistance development.\n- The term “antibiotic rationalization” should be clearly defined—does it refer to de-escalation, targeted therapy, or treatment discontinuation?\n- I recommend analyzing bacterial resistance and its impact on HAP treatment outcomes. Also, assess the effect of inadequate initial antibiotic therapy (i.e., pathogen resistant to the initial treatment) compared to adequate initial therapy (i.e., pathogen susceptible to the administered antibiotic).\n- I recommend expanding the protocol to include monitoring of inflammatory markers, particularly procalcitonin (PCT), which may aid in decisions regarding initiation and discontinuation of antibiotic therapy.\n- Evaluating the dynamics of CRP, leukocytosis, and other laboratory parameters could contribute to better patient stratification.\n4. NV-HAP vs. VAP- The study appropriately distinguishes between these two groups but should further reflect differences in diagnostic method availability and sampling feasibility.\n- Consider evaluating access to specialized procedures (e.g., bronchoscopy) as a systemic factor influencing diagnostic accuracy.\nHave added statements around availability of bronchoscopy, and the variability in access to diagnostic approaches among NVHAP and VAP.\nMany thanks again for your time and consideration in reviewing our manuscript.\nViews\n0\nHow to cite this report:\nBai AD. Reviewer Report For: Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.15054.r35522) The direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-35522\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-35522\nNOTE: it is important to ensure the information in square brackets after the title is included in this citation.\nReviewer Report 27 May 2025\nApproved with Reservations\nVIEWS 0\nThis is a study protocol for a retrospective cohort study on hospital acquired pneumonia using hospital electronic database. The objectives are to compare ventilated and non-ventilated cases in terms of microbiologic diagnosis and how microbiologic diagnosis impact mortality. The study ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close\nThis is a study protocol for a retrospective cohort study on hospital acquired pneumonia using hospital electronic database. The objectives are to compare ventilated and non-ventilated cases in terms of microbiologic diagnosis and how microbiologic diagnosis impact mortality. The study uses the data from 4 hospitals within the University Hospital Birmingham NHS Trust.\nOverall, I think it is a worthwhile study given that non-ventilated HAP is not well characterized or studied. It would also be worthwhile to understand the practices such as microbiologic work-up for these patients and see if there are any association between practice patterns and patient outcomes with the hope that it may lead to improvements in quality of care and patient outcome.\nMy main criticism is how the HAP patients are defined. Based on the described methodology, there were 2 ways of how HAP was diagnosed based on diagnosis codes and based on surveillance definition. There would be two different cohorts based on these two different definitions. It is unclear if the subsequent analysis is done on either or both of these cohorts. Either way will have a significant number of false positives and negatives. It should be noted that the criteria based on reference 11 has only a kappa of 0.33 in terms of agreement with clinician diagnosis. I think it may be worthwhile to try to do manual chart review on a proportion of cases and have clinicians determine which patients had HAP. Then see how this compares to the above 2 ways to define VAP. This would provide some important internal validation and can help decide which way to define the cohort between doing the rest of the analysis.\nMajor comments:\n- Protocol, Data sources and variables: For the criteria based on reference 11, criteria on new antibiotic prescription, the antibiotic should be specified. The original criteria used a list of antibiotics (in their eTable 1) that are used empirically for HAP. This should be specified in this protocol and this antibiotic list may change depending on the England guidelines of what is appropriate antibiotic therapy for HAP\n- Statistical analysis plan Aim 1: Can the authors justify the rationale for all-cause readmission within 1 year of discharge? This seems like a very long follow-up for readmission. I am not sure how a readmission at end of 1 year post discharge would be related to the original HAP. Note that within 1 year of discharge would extend follow-up to beyond 1 year given discharge would occur after HAP diagnosis.\n- Statistical analysis plan Aim 1: For 30-day and 1-year mortality, would the database be able to capture out-of-hospital deaths? If so, it should be specified how this can be captured and how accurate this would be.\n- Statistical analysis plan Aim 1: Nosocomial infections potentially related to antibiotic such as Clostridium difficile - this should be defined better.\n- Statistical analysis plan Aim 2: The microbiological sampling considered in the study should be specified. Sputum and bronchoscopy samples are a given, but what about nasopharyngeal swab or blood culture?\n- Statistical analysis plan Aim 2: How will putative pathogen be defined? It will need interpretation of the culture results, as culture may yield multiple organisms and many organisms can be considered contaminants (e.g. candida, oral flora bacteria).\n- Statistical analysis plan Aim 2: How will the study describe antimicrobial used? Would the antimicrobial agents be listed individually or in groups? I would suggest to describe group of antibiotics based on coverage such as anti-MRSA coverage, anti-pseudomonal coverage etc.\n- Statistical analysis plan Aim 2: I don't understand what rationalized mean in \"The proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated.\" Please explain.\n- Statistical analysis plan Aim 2: Sometimes a culture may be helpful even if it does not grow anything. For example, in a patient whose sputum gram stain showed no gram-positive cocci and culture did not grow MRSA, then some clinicians may discontinue MRSA coverage. The authors should consider accounting for this and look at if culture taken (whether positive or negative) can lead to narrowing of antimicrobials.\n- Statistical analysis plan Aim 4: One important system factor would be if the hospitals have respirology / pulmonology service and bronchoscopy availability\n- Statistical analysis plan Aim 5: I understand the rationale for this objective. However, one criticism to anticipate is that there is clearly confounding by indication with respect of obtaining cultures on a patient. There is a reason that clinicians ordered sputum cultures or performed bronchoscopy on one patient but not in the other. In the end, these two groups will be fundamentally different that the propensity score weighting will not be able to balance them and there will always be residual confounding.\n- Statistical analysis plan Aim 5: There should be description of how the outcomes will be compared between the two groups after propensity score weighting.\n- Statistical analysis plan Aim 6: There should be description of how natural language processing will be done and how the gold standard of consolidation is done.\nMinor comments\n- The study period from 2018 to 2022 is a bit outdated even considering the 1-year follow-up, can the study period be extended up to 2023 or 2024?\n- Note that C difficile should be Clostridioides difficile\nMinor edits\n- Introduction: \"Although the significance of HAP is widely zrecognised\" - typo\n- Introduction: \"ythese have been found to be absent in up to half of patients treated with HAP\" - typo\n- Introduction: \"pneumonia occurring among non-ventilated zhospitalised patients (NV-HAP).\" - typo\n- Introduction: \"This may not be possiblebecause many of the factors predisposing patients\" - needs space between possible and because\n- Aims: \"Characterise the microbiology of HAP, to include the rates of microbiological sampling, the organisms identified, and whether these were covered by antimicrobialsgiven\" - needs space between antimicrobials and given\n- Patient and Public Involvement: \"their perspective on what to zprioritise\" - typo\n- Discussion: \"clinical status willvary significantly during that time.\" - needs space between will and vary\n- Discussion: \"Therefore, these it will be necessary\" - delete these\nOverall, I think it is a worthwhile study given that non-ventilated HAP is not well characterized or studied. It would also be worthwhile to understand the practices such as microbiologic work-up for these patients and see if there are any association between practice patterns and patient outcomes with the hope that it may lead to improvements in quality of care and patient outcome.\nMy main criticism is how the HAP patients are defined. Based on the described methodology, there were 2 ways of how HAP was diagnosed based on diagnosis codes and based on surveillance definition. There would be two different cohorts based on these two different definitions. It is unclear if the subsequent analysis is done on either or both of these cohorts. Either way will have a significant number of false positives and negatives. It should be noted that the criteria based on reference 11 has only a kappa of 0.33 in terms of agreement with clinician diagnosis. I think it may be worthwhile to try to do manual chart review on a proportion of cases and have clinicians determine which patients had HAP. Then see how this compares to the above 2 ways to define VAP. This would provide some important internal validation and can help decide which way to define the cohort between doing the rest of the analysis.\nMajor comments:\n- Protocol, Data sources and variables: For the criteria based on reference 11, criteria on new antibiotic prescription, the antibiotic should be specified. The original criteria used a list of antibiotics (in their eTable 1) that are used empirically for HAP. This should be specified in this protocol and this antibiotic list may change depending on the England guidelines of what is appropriate antibiotic therapy for HAP\n- Statistical analysis plan Aim 1: Can the authors justify the rationale for all-cause readmission within 1 year of discharge? This seems like a very long follow-up for readmission. I am not sure how a readmission at end of 1 year post discharge would be related to the original HAP. Note that within 1 year of discharge would extend follow-up to beyond 1 year given discharge would occur after HAP diagnosis.\n- Statistical analysis plan Aim 1: For 30-day and 1-year mortality, would the database be able to capture out-of-hospital deaths? If so, it should be specified how this can be captured and how accurate this would be.\n- Statistical analysis plan Aim 1: Nosocomial infections potentially related to antibiotic such as Clostridium difficile - this should be defined better.\n- Statistical analysis plan Aim 2: The microbiological sampling considered in the study should be specified. Sputum and bronchoscopy samples are a given, but what about nasopharyngeal swab or blood culture?\n- Statistical analysis plan Aim 2: How will putative pathogen be defined? It will need interpretation of the culture results, as culture may yield multiple organisms and many organisms can be considered contaminants (e.g. candida, oral flora bacteria).\n- Statistical analysis plan Aim 2: How will the study describe antimicrobial used? Would the antimicrobial agents be listed individually or in groups? I would suggest to describe group of antibiotics based on coverage such as anti-MRSA coverage, anti-pseudomonal coverage etc.\n- Statistical analysis plan Aim 2: I don't understand what rationalized mean in \"The proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated.\" Please explain.\n- Statistical analysis plan Aim 2: Sometimes a culture may be helpful even if it does not grow anything. For example, in a patient whose sputum gram stain showed no gram-positive cocci and culture did not grow MRSA, then some clinicians may discontinue MRSA coverage. The authors should consider accounting for this and look at if culture taken (whether positive or negative) can lead to narrowing of antimicrobials.\n- Statistical analysis plan Aim 4: One important system factor would be if the hospitals have respirology / pulmonology service and bronchoscopy availability\n- Statistical analysis plan Aim 5: I understand the rationale for this objective. However, one criticism to anticipate is that there is clearly confounding by indication with respect of obtaining cultures on a patient. There is a reason that clinicians ordered sputum cultures or performed bronchoscopy on one patient but not in the other. In the end, these two groups will be fundamentally different that the propensity score weighting will not be able to balance them and there will always be residual confounding.\n- Statistical analysis plan Aim 5: There should be description of how the outcomes will be compared between the two groups after propensity score weighting.\n- Statistical analysis plan Aim 6: There should be description of how natural language processing will be done and how the gold standard of consolidation is done.\nMinor comments\n- The study period from 2018 to 2022 is a bit outdated even considering the 1-year follow-up, can the study period be extended up to 2023 or 2024?\n- Note that C difficile should be Clostridioides difficile\nMinor edits\n- Introduction: \"Although the significance of HAP is widely zrecognised\" - typo\n- Introduction: \"ythese have been found to be absent in up to half of patients treated with HAP\" - typo\n- Introduction: \"pneumonia occurring among non-ventilated zhospitalised patients (NV-HAP).\" - typo\n- Introduction: \"This may not be possiblebecause many of the factors predisposing patients\" - needs space between possible and because\n- Aims: \"Characterise the microbiology of HAP, to include the rates of microbiological sampling, the organisms identified, and whether these were covered by antimicrobialsgiven\" - needs space between antimicrobials and given\n- Patient and Public Involvement: \"their perspective on what to zprioritise\" - typo\n- Discussion: \"clinical status willvary significantly during that time.\" - needs space between will and vary\n- Discussion: \"Therefore, these it will be necessary\" - delete these\n-\nIs the rationale for, and objectives of, the study clearly described?\nYes\n-\nIs the study design appropriate for the research question?\nYes\n-\nAre sufficient details of the methods provided to allow replication by others?\nPartly\n-\nAre the datasets clearly presented in a useable and accessible format?\nNot applicable\nCompeting Interests: No competing interests were disclosed.\nReviewer Expertise: Infectious diseases, clinical research, analysis of hospital databases\nCITE\nHOW TO CITE THIS REPORT Bai AD. Reviewer Report For: Study Protocol: A retrospective observational analysis of patients treated for hospital-acquired pneumonia. [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:36 (https://doi.org/10.3310/nihropenres.15054.r35522)\nThe direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-35522\nhttps://openresearch.nihr.ac.uk/articles/5-36/v1#referee-response-35522\nNOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.\n- Author Response 15 Sep 2025Samuel Quarton, NIHR Birmingham Biomedical Research Centre, Birmingham, UK15 Sep 2025Author ResponseMany thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study is of value, and your comments and input are ... Continue reading Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study is of value, and your comments and input are greatly appreciated.\nPlease see below for our response to your specific comments, together with changes based on your insights. The updated version has been submitted and will be visible shortly.\nMy main criticism is how the HAP patients are defined. Based on the described methodology, there were 2 ways of how HAP was diagnosed based on diagnosis codes and based on surveillance definition. There would be two different cohorts based on these two different definitions. It is unclear if the subsequent analysis is done on either or both of these cohorts. Either way will have a significant number of false positives and negatives. It should be noted that the criteria based on reference 11 has only a kappa of 0.33 in terms of agreement with clinician diagnosis. I think it may be worthwhile to try to do manual chart review on a proportion of cases and have clinicians determine which patients had HAP. Then see how this compares to the above 2 ways to define VAP. This would provide some important internal validation and can help decide which way to define the cohort between doing the rest of the analysis.\nResponse: We agree that this is an important limitation. At present we are keen to include both cohorts – diagnostic coding as this is the primary way in which cases are retrospectively identified, and the surveillance definition due to known limitations in the coding data, and due to an ability to associate this with a date and time for analysis of time-dependent variables.\nSensitivity analysis will be performed - comparing patients who meet the coding definition, those meeting the surveillance definition, and those meeting both. We have added a statement regarding this in the manuscript.\nInternal validation would be helpful, however given limitations in the anonymised available data, accessing patient records to compare with a manual review is not possible.\nHowever, we will separately use both definitions on a smaller dataset from one of the four hospital sites where case review is possible and perform manual case review among this group. While this will not be as helpful as random sampling from within the cohort used for analysis, it will give some indication of the utility of these definitions among our patients.\n- Protocol, Data sources and variables: For the criteria based on reference 11, criteria on new antibiotic prescription, the antibiotic should be specified. The original criteria used a list of antibiotics (in their eTable 1) that are used empirically for HAP. This should be specified in this protocol and this antibiotic list may change depending on the England guidelines of what is appropriate antibiotic therapy for HAP\nResponse: We agree and apologies for this omission. The list of included antibiotics is now included. We have based our list on antibiotics used empirically for HAP, with reference to the included citation, NICE guideline 139 (Pneumonia (hospital-acquired): antimicrobial prescribing), and local trust guidelines.\nAs a further screen, where patients met the criteria based on antibiotics that did not meet local guidance, but with potential use as empirical treatment for HAP, and where these affected >100 possible cases, review of case notes will be performed to identify in what proportion HAP is mentioned. Where fewer than 50% of cases reference HAP, these antibiotics will not be included in the final list.\n- Statistical analysis plan Aim 1: Can the authors justify the rationale for all-cause readmission within 1 year of discharge? This seems like a very long follow-up for readmission. I am not sure how a readmission at end of 1 year post discharge would be related to the original HAP. Note that within 1 year of discharge would extend follow-up to beyond 1 year given discharge would occur after HAP diagnosis.\nResponse: While 1 year follow up is a relatively long period of time, we believe it is justified. Pneumonia has previously been shown to have lasting impacts on outcome (Long-term prognosis in community-acquired pneumonia - PMC). The cohort at greatest risk of HAP is older and more co-morbid, both associated with frailty. HAP may plausibly lead to a step-change in increased frailty, and this may be captured in following up all-cause readmissions for up to 1 year.\n- Statistical analysis plan Aim 1: For 30-day and 1-year mortality, would the database be able to capture out-of-hospital deaths? If so, it should be specified how this can be captured and how accurate this would be.\nResponse: Added wording to clarify this – yes we capture out-of-hospital deaths using primary care data linked to official registers of deaths.\n- Statistical analysis plan Aim 1: Nosocomial infections potentially related to antibiotic such as Clostridium difficile - this should be defined better.\nResponse: Thank you for the feedback. This has been reworded.\n- Statistical analysis plan Aim 2: The microbiological sampling considered in the study should be specified. Sputum and bronchoscopy samples are a given, but what about nasopharyngeal swab or blood culture?\nResponse: Updated to specify this. Sputum, bronchoscopy samples, pleural fluid, blood culture and nasopharyngeal swabs will all be captured and analysed, together and separately.\n- Statistical analysis plan Aim 2: How will putative pathogen be defined? It will need interpretation of the culture results, as culture may yield multiple organisms and many organisms can be considered contaminants (e.g. candida, oral flora bacteria).\nResponse: Within our hospital sites oral flora bacteria are reported in those terms – ie “mixed oral flora” or “mixed upper respiratory tract flora only”, rather than as specific organisms identified. There remains an inability to differentiate between colonisation/contamination and infection from microbiology report alone, however more detailed analysis on a patient-by-patient basis would not be feasible across the whole dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. Rates of candida or other environmental fungi identified will be reported but not used as part of this analysis, which is focused primarily on the spectrum of antibiotic usage. Text changed to clarify this, and reinforce these are potential pathogens only.\n- Statistical analysis plan Aim 2: How will the study describe antimicrobial used? Would the antimicrobial agents be listed individually or in groups? I would suggest to describe group of antibiotics based on coverage such as anti-MRSA coverage, anti-pseudomonal coverage etc.\nResponse: We will group antibiotics by spectrum of activity, and compare proportions received of each group in patients with and without an organism identified. Text updated to clarify this.\n- Statistical analysis plan Aim 2: I don't understand what rationalized mean in \"The proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated.\" Please explain\nResponse: We agree this could be worded more clearly. We aim to capture what proportion of patients had their antimicrobial regime changed in light of microbiological results, for example to a narrower-spectrum antibiotic, or changed to add additional cover. Text updated to hopefully make this more explicit and specific.\n- Statistical analysis plan Aim 2: Sometimes a culture may be helpful even if it does not grow anything. For example, in a patient whose sputum gram stain showed no gram-positive cocci and culture did not grow MRSA, then some clinicians may discontinue MRSA coverage. The authors should consider accounting for this and look at if culture taken (whether positive or negative) can lead to narrowing of antimicrobials.\nResponse: Thank you for this suggestion. In addition to the impact of positive microbiology, we will indeed examine the impact on antimicrobial prescribing of taking cultures. Text updated to reflect this.\n- Statistical analysis plan Aim 4: One important system factor would be if the hospitals have respirology / pulmonology service and bronchoscopy availability\nResponse: This is a key system factor, however all hospitals included in the analysis have these services available on site.\n- Statistical analysis plan Aim 5: I understand the rationale for this objective. However, one criticism to anticipate is that there is clearly confounding by indication with respect of obtaining cultures on a patient. There is a reason that clinicians ordered sputum cultures or performed bronchoscopy on one patient but not in the other. In the end, these two groups will be fundamentally different that the propensity score weighting will not be able to balance them and there will always be residual confounding.\nResponse: We understand – and largely agree with – this criticism. However, clinician request is likely not the only factor impacting whether cultures are sent, something which will hopefully be illuminated by aim 4.\nFurther, as the ATS 2016 guidelines recommend all patients with suspected HAP undergo culture of lower respiratory tract secretions, we believe the indication for sputum culture should be present for all included patients. Sputum and bronchoscopy samples will be assessed separately, as we agree that clearly patients who are selected to undergo bronchoscopy will represent a very different cohort from those who are not. Among the NV-HAP cohort, very few are anticipated to have undergone bronchoscopy. Despite the clear challenges with confounding, we believe this is an important area to examine.\n- Statistical analysis plan Aim 5: There should be description of how the outcomes will be compared between the two groups after propensity score weighting.\nResponse: Mortality will be assessed with a Cox proportional hazards analysis. Added to the text to clarify.\n- Statistical analysis plan Aim 6: There should be description of how natural language processing will be done and how the gold standard of consolidation is done.\nResponse: Methodology for this is under development. We have added that the gold standard we are using is radiologist reports, and that we will provide information on methods to develop and validate results with any subsequent results paper.\nMinor edits\nResponse: Thank you for flagging these. Several of these typos, plus additional inconsistencies in tense, seem to have been introduced during the copyedit process, for which we apologies.Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study is of value, and your comments and input are greatly appreciated.Competing Interests: No competing interests were disclosed. Close\nPlease see below for our response to your specific comments, together with changes based on your insights. The updated version has been submitted and will be visible shortly.\nMy main criticism is how the HAP patients are defined. Based on the described methodology, there were 2 ways of how HAP was diagnosed based on diagnosis codes and based on surveillance definition. There would be two different cohorts based on these two different definitions. It is unclear if the subsequent analysis is done on either or both of these cohorts. Either way will have a significant number of false positives and negatives. It should be noted that the criteria based on reference 11 has only a kappa of 0.33 in terms of agreement with clinician diagnosis. I think it may be worthwhile to try to do manual chart review on a proportion of cases and have clinicians determine which patients had HAP. Then see how this compares to the above 2 ways to define VAP. This would provide some important internal validation and can help decide which way to define the cohort between doing the rest of the analysis.\nResponse: We agree that this is an important limitation. At present we are keen to include both cohorts – diagnostic coding as this is the primary way in which cases are retrospectively identified, and the surveillance definition due to known limitations in the coding data, and due to an ability to associate this with a date and time for analysis of time-dependent variables.\nSensitivity analysis will be performed - comparing patients who meet the coding definition, those meeting the surveillance definition, and those meeting both. We have added a statement regarding this in the manuscript.\nInternal validation would be helpful, however given limitations in the anonymised available data, accessing patient records to compare with a manual review is not possible.\nHowever, we will separately use both definitions on a smaller dataset from one of the four hospital sites where case review is possible and perform manual case review among this group. While this will not be as helpful as random sampling from within the cohort used for analysis, it will give some indication of the utility of these definitions among our patients.\n- Protocol, Data sources and variables: For the criteria based on reference 11, criteria on new antibiotic prescription, the antibiotic should be specified. The original criteria used a list of antibiotics (in their eTable 1) that are used empirically for HAP. This should be specified in this protocol and this antibiotic list may change depending on the England guidelines of what is appropriate antibiotic therapy for HAP\nResponse: We agree and apologies for this omission. The list of included antibiotics is now included. We have based our list on antibiotics used empirically for HAP, with reference to the included citation, NICE guideline 139 (Pneumonia (hospital-acquired): antimicrobial prescribing), and local trust guidelines.\nAs a further screen, where patients met the criteria based on antibiotics that did not meet local guidance, but with potential use as empirical treatment for HAP, and where these affected >100 possible cases, review of case notes will be performed to identify in what proportion HAP is mentioned. Where fewer than 50% of cases reference HAP, these antibiotics will not be included in the final list.\n- Statistical analysis plan Aim 1: Can the authors justify the rationale for all-cause readmission within 1 year of discharge? This seems like a very long follow-up for readmission. I am not sure how a readmission at end of 1 year post discharge would be related to the original HAP. Note that within 1 year of discharge would extend follow-up to beyond 1 year given discharge would occur after HAP diagnosis.\nResponse: While 1 year follow up is a relatively long period of time, we believe it is justified. Pneumonia has previously been shown to have lasting impacts on outcome (Long-term prognosis in community-acquired pneumonia - PMC). The cohort at greatest risk of HAP is older and more co-morbid, both associated with frailty. HAP may plausibly lead to a step-change in increased frailty, and this may be captured in following up all-cause readmissions for up to 1 year.\n- Statistical analysis plan Aim 1: For 30-day and 1-year mortality, would the database be able to capture out-of-hospital deaths? If so, it should be specified how this can be captured and how accurate this would be.\nResponse: Added wording to clarify this – yes we capture out-of-hospital deaths using primary care data linked to official registers of deaths.\n- Statistical analysis plan Aim 1: Nosocomial infections potentially related to antibiotic such as Clostridium difficile - this should be defined better.\nResponse: Thank you for the feedback. This has been reworded.\n- Statistical analysis plan Aim 2: The microbiological sampling considered in the study should be specified. Sputum and bronchoscopy samples are a given, but what about nasopharyngeal swab or blood culture?\nResponse: Updated to specify this. Sputum, bronchoscopy samples, pleural fluid, blood culture and nasopharyngeal swabs will all be captured and analysed, together and separately.\n- Statistical analysis plan Aim 2: How will putative pathogen be defined? It will need interpretation of the culture results, as culture may yield multiple organisms and many organisms can be considered contaminants (e.g. candida, oral flora bacteria).\nResponse: Within our hospital sites oral flora bacteria are reported in those terms – ie “mixed oral flora” or “mixed upper respiratory tract flora only”, rather than as specific organisms identified. There remains an inability to differentiate between colonisation/contamination and infection from microbiology report alone, however more detailed analysis on a patient-by-patient basis would not be feasible across the whole dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. Rates of candida or other environmental fungi identified will be reported but not used as part of this analysis, which is focused primarily on the spectrum of antibiotic usage. Text changed to clarify this, and reinforce these are potential pathogens only.\n- Statistical analysis plan Aim 2: How will the study describe antimicrobial used? Would the antimicrobial agents be listed individually or in groups? I would suggest to describe group of antibiotics based on coverage such as anti-MRSA coverage, anti-pseudomonal coverage etc.\nResponse: We will group antibiotics by spectrum of activity, and compare proportions received of each group in patients with and without an organism identified. Text updated to clarify this.\n- Statistical analysis plan Aim 2: I don't understand what rationalized mean in \"The proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated.\" Please explain\nResponse: We agree this could be worded more clearly. We aim to capture what proportion of patients had their antimicrobial regime changed in light of microbiological results, for example to a narrower-spectrum antibiotic, or changed to add additional cover. Text updated to hopefully make this more explicit and specific.\n- Statistical analysis plan Aim 2: Sometimes a culture may be helpful even if it does not grow anything. For example, in a patient whose sputum gram stain showed no gram-positive cocci and culture did not grow MRSA, then some clinicians may discontinue MRSA coverage. The authors should consider accounting for this and look at if culture taken (whether positive or negative) can lead to narrowing of antimicrobials.\nResponse: Thank you for this suggestion. In addition to the impact of positive microbiology, we will indeed examine the impact on antimicrobial prescribing of taking cultures. Text updated to reflect this.\n- Statistical analysis plan Aim 4: One important system factor would be if the hospitals have respirology / pulmonology service and bronchoscopy availability\nResponse: This is a key system factor, however all hospitals included in the analysis have these services available on site.\n- Statistical analysis plan Aim 5: I understand the rationale for this objective. However, one criticism to anticipate is that there is clearly confounding by indication with respect of obtaining cultures on a patient. There is a reason that clinicians ordered sputum cultures or performed bronchoscopy on one patient but not in the other. In the end, these two groups will be fundamentally different that the propensity score weighting will not be able to balance them and there will always be residual confounding.\nResponse: We understand – and largely agree with – this criticism. However, clinician request is likely not the only factor impacting whether cultures are sent, something which will hopefully be illuminated by aim 4.\nFurther, as the ATS 2016 guidelines recommend all patients with suspected HAP undergo culture of lower respiratory tract secretions, we believe the indication for sputum culture should be present for all included patients. Sputum and bronchoscopy samples will be assessed separately, as we agree that clearly patients who are selected to undergo bronchoscopy will represent a very different cohort from those who are not. Among the NV-HAP cohort, very few are anticipated to have undergone bronchoscopy. Despite the clear challenges with confounding, we believe this is an important area to examine.\n- Statistical analysis plan Aim 5: There should be description of how the outcomes will be compared between the two groups after propensity score weighting.\nResponse: Mortality will be assessed with a Cox proportional hazards analysis. Added to the text to clarify.\n- Statistical analysis plan Aim 6: There should be description of how natural language processing will be done and how the gold standard of consolidation is done.\nResponse: Methodology for this is under development. We have added that the gold standard we are using is radiologist reports, and that we will provide information on methods to develop and validate results with any subsequent results paper.\nMinor edits\nResponse: Thank you for flagging these. Several of these typos, plus additional inconsistencies in tense, seem to have been introduced during the copyedit process, for which we apologies.\nCOMMENTS ON THIS REPORT\n- Author Response 15 Sep 2025Samuel Quarton, NIHR Birmingham Biomedical Research Centre, Birmingham, UK15 Sep 2025Author ResponseMany thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study is of value, and your comments and input are ... Continue reading Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study is of value, and your comments and input are greatly appreciated.\nPlease see below for our response to your specific comments, together with changes based on your insights. The updated version has been submitted and will be visible shortly.\nMy main criticism is how the HAP patients are defined. Based on the described methodology, there were 2 ways of how HAP was diagnosed based on diagnosis codes and based on surveillance definition. There would be two different cohorts based on these two different definitions. It is unclear if the subsequent analysis is done on either or both of these cohorts. Either way will have a significant number of false positives and negatives. It should be noted that the criteria based on reference 11 has only a kappa of 0.33 in terms of agreement with clinician diagnosis. I think it may be worthwhile to try to do manual chart review on a proportion of cases and have clinicians determine which patients had HAP. Then see how this compares to the above 2 ways to define VAP. This would provide some important internal validation and can help decide which way to define the cohort between doing the rest of the analysis.\nResponse: We agree that this is an important limitation. At present we are keen to include both cohorts – diagnostic coding as this is the primary way in which cases are retrospectively identified, and the surveillance definition due to known limitations in the coding data, and due to an ability to associate this with a date and time for analysis of time-dependent variables.\nSensitivity analysis will be performed - comparing patients who meet the coding definition, those meeting the surveillance definition, and those meeting both. We have added a statement regarding this in the manuscript.\nInternal validation would be helpful, however given limitations in the anonymised available data, accessing patient records to compare with a manual review is not possible.\nHowever, we will separately use both definitions on a smaller dataset from one of the four hospital sites where case review is possible and perform manual case review among this group. While this will not be as helpful as random sampling from within the cohort used for analysis, it will give some indication of the utility of these definitions among our patients.\n- Protocol, Data sources and variables: For the criteria based on reference 11, criteria on new antibiotic prescription, the antibiotic should be specified. The original criteria used a list of antibiotics (in their eTable 1) that are used empirically for HAP. This should be specified in this protocol and this antibiotic list may change depending on the England guidelines of what is appropriate antibiotic therapy for HAP\nResponse: We agree and apologies for this omission. The list of included antibiotics is now included. We have based our list on antibiotics used empirically for HAP, with reference to the included citation, NICE guideline 139 (Pneumonia (hospital-acquired): antimicrobial prescribing), and local trust guidelines.\nAs a further screen, where patients met the criteria based on antibiotics that did not meet local guidance, but with potential use as empirical treatment for HAP, and where these affected >100 possible cases, review of case notes will be performed to identify in what proportion HAP is mentioned. Where fewer than 50% of cases reference HAP, these antibiotics will not be included in the final list.\n- Statistical analysis plan Aim 1: Can the authors justify the rationale for all-cause readmission within 1 year of discharge? This seems like a very long follow-up for readmission. I am not sure how a readmission at end of 1 year post discharge would be related to the original HAP. Note that within 1 year of discharge would extend follow-up to beyond 1 year given discharge would occur after HAP diagnosis.\nResponse: While 1 year follow up is a relatively long period of time, we believe it is justified. Pneumonia has previously been shown to have lasting impacts on outcome (Long-term prognosis in community-acquired pneumonia - PMC). The cohort at greatest risk of HAP is older and more co-morbid, both associated with frailty. HAP may plausibly lead to a step-change in increased frailty, and this may be captured in following up all-cause readmissions for up to 1 year.\n- Statistical analysis plan Aim 1: For 30-day and 1-year mortality, would the database be able to capture out-of-hospital deaths? If so, it should be specified how this can be captured and how accurate this would be.\nResponse: Added wording to clarify this – yes we capture out-of-hospital deaths using primary care data linked to official registers of deaths.\n- Statistical analysis plan Aim 1: Nosocomial infections potentially related to antibiotic such as Clostridium difficile - this should be defined better.\nResponse: Thank you for the feedback. This has been reworded.\n- Statistical analysis plan Aim 2: The microbiological sampling considered in the study should be specified. Sputum and bronchoscopy samples are a given, but what about nasopharyngeal swab or blood culture?\nResponse: Updated to specify this. Sputum, bronchoscopy samples, pleural fluid, blood culture and nasopharyngeal swabs will all be captured and analysed, together and separately.\n- Statistical analysis plan Aim 2: How will putative pathogen be defined? It will need interpretation of the culture results, as culture may yield multiple organisms and many organisms can be considered contaminants (e.g. candida, oral flora bacteria).\nResponse: Within our hospital sites oral flora bacteria are reported in those terms – ie “mixed oral flora” or “mixed upper respiratory tract flora only”, rather than as specific organisms identified. There remains an inability to differentiate between colonisation/contamination and infection from microbiology report alone, however more detailed analysis on a patient-by-patient basis would not be feasible across the whole dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. Rates of candida or other environmental fungi identified will be reported but not used as part of this analysis, which is focused primarily on the spectrum of antibiotic usage. Text changed to clarify this, and reinforce these are potential pathogens only.\n- Statistical analysis plan Aim 2: How will the study describe antimicrobial used? Would the antimicrobial agents be listed individually or in groups? I would suggest to describe group of antibiotics based on coverage such as anti-MRSA coverage, anti-pseudomonal coverage etc.\nResponse: We will group antibiotics by spectrum of activity, and compare proportions received of each group in patients with and without an organism identified. Text updated to clarify this.\n- Statistical analysis plan Aim 2: I don't understand what rationalized mean in \"The proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated.\" Please explain\nResponse: We agree this could be worded more clearly. We aim to capture what proportion of patients had their antimicrobial regime changed in light of microbiological results, for example to a narrower-spectrum antibiotic, or changed to add additional cover. Text updated to hopefully make this more explicit and specific.\n- Statistical analysis plan Aim 2: Sometimes a culture may be helpful even if it does not grow anything. For example, in a patient whose sputum gram stain showed no gram-positive cocci and culture did not grow MRSA, then some clinicians may discontinue MRSA coverage. The authors should consider accounting for this and look at if culture taken (whether positive or negative) can lead to narrowing of antimicrobials.\nResponse: Thank you for this suggestion. In addition to the impact of positive microbiology, we will indeed examine the impact on antimicrobial prescribing of taking cultures. Text updated to reflect this.\n- Statistical analysis plan Aim 4: One important system factor would be if the hospitals have respirology / pulmonology service and bronchoscopy availability\nResponse: This is a key system factor, however all hospitals included in the analysis have these services available on site.\n- Statistical analysis plan Aim 5: I understand the rationale for this objective. However, one criticism to anticipate is that there is clearly confounding by indication with respect of obtaining cultures on a patient. There is a reason that clinicians ordered sputum cultures or performed bronchoscopy on one patient but not in the other. In the end, these two groups will be fundamentally different that the propensity score weighting will not be able to balance them and there will always be residual confounding.\nResponse: We understand – and largely agree with – this criticism. However, clinician request is likely not the only factor impacting whether cultures are sent, something which will hopefully be illuminated by aim 4.\nFurther, as the ATS 2016 guidelines recommend all patients with suspected HAP undergo culture of lower respiratory tract secretions, we believe the indication for sputum culture should be present for all included patients. Sputum and bronchoscopy samples will be assessed separately, as we agree that clearly patients who are selected to undergo bronchoscopy will represent a very different cohort from those who are not. Among the NV-HAP cohort, very few are anticipated to have undergone bronchoscopy. Despite the clear challenges with confounding, we believe this is an important area to examine.\n- Statistical analysis plan Aim 5: There should be description of how the outcomes will be compared between the two groups after propensity score weighting.\nResponse: Mortality will be assessed with a Cox proportional hazards analysis. Added to the text to clarify.\n- Statistical analysis plan Aim 6: There should be description of how natural language processing will be done and how the gold standard of consolidation is done.\nResponse: Methodology for this is under development. We have added that the gold standard we are using is radiologist reports, and that we will provide information on methods to develop and validate results with any subsequent results paper.\nMinor edits\nResponse: Thank you for flagging these. Several of these typos, plus additional inconsistencies in tense, seem to have been introduced during the copyedit process, for which we apologies.Many thanks for your thorough and helpful appraisal of our proposed study. It is encouraging to hear you feel the study is of value, and your comments and input are greatly appreciated.Competing Interests: No competing interests were disclosed. Close\nPlease see below for our response to your specific comments, together with changes based on your insights. The updated version has been submitted and will be visible shortly.\nMy main criticism is how the HAP patients are defined. Based on the described methodology, there were 2 ways of how HAP was diagnosed based on diagnosis codes and based on surveillance definition. There would be two different cohorts based on these two different definitions. It is unclear if the subsequent analysis is done on either or both of these cohorts. Either way will have a significant number of false positives and negatives. It should be noted that the criteria based on reference 11 has only a kappa of 0.33 in terms of agreement with clinician diagnosis. I think it may be worthwhile to try to do manual chart review on a proportion of cases and have clinicians determine which patients had HAP. Then see how this compares to the above 2 ways to define VAP. This would provide some important internal validation and can help decide which way to define the cohort between doing the rest of the analysis.\nResponse: We agree that this is an important limitation. At present we are keen to include both cohorts – diagnostic coding as this is the primary way in which cases are retrospectively identified, and the surveillance definition due to known limitations in the coding data, and due to an ability to associate this with a date and time for analysis of time-dependent variables.\nSensitivity analysis will be performed - comparing patients who meet the coding definition, those meeting the surveillance definition, and those meeting both. We have added a statement regarding this in the manuscript.\nInternal validation would be helpful, however given limitations in the anonymised available data, accessing patient records to compare with a manual review is not possible.\nHowever, we will separately use both definitions on a smaller dataset from one of the four hospital sites where case review is possible and perform manual case review among this group. While this will not be as helpful as random sampling from within the cohort used for analysis, it will give some indication of the utility of these definitions among our patients.\n- Protocol, Data sources and variables: For the criteria based on reference 11, criteria on new antibiotic prescription, the antibiotic should be specified. The original criteria used a list of antibiotics (in their eTable 1) that are used empirically for HAP. This should be specified in this protocol and this antibiotic list may change depending on the England guidelines of what is appropriate antibiotic therapy for HAP\nResponse: We agree and apologies for this omission. The list of included antibiotics is now included. We have based our list on antibiotics used empirically for HAP, with reference to the included citation, NICE guideline 139 (Pneumonia (hospital-acquired): antimicrobial prescribing), and local trust guidelines.\nAs a further screen, where patients met the criteria based on antibiotics that did not meet local guidance, but with potential use as empirical treatment for HAP, and where these affected >100 possible cases, review of case notes will be performed to identify in what proportion HAP is mentioned. Where fewer than 50% of cases reference HAP, these antibiotics will not be included in the final list.\n- Statistical analysis plan Aim 1: Can the authors justify the rationale for all-cause readmission within 1 year of discharge? This seems like a very long follow-up for readmission. I am not sure how a readmission at end of 1 year post discharge would be related to the original HAP. Note that within 1 year of discharge would extend follow-up to beyond 1 year given discharge would occur after HAP diagnosis.\nResponse: While 1 year follow up is a relatively long period of time, we believe it is justified. Pneumonia has previously been shown to have lasting impacts on outcome (Long-term prognosis in community-acquired pneumonia - PMC). The cohort at greatest risk of HAP is older and more co-morbid, both associated with frailty. HAP may plausibly lead to a step-change in increased frailty, and this may be captured in following up all-cause readmissions for up to 1 year.\n- Statistical analysis plan Aim 1: For 30-day and 1-year mortality, would the database be able to capture out-of-hospital deaths? If so, it should be specified how this can be captured and how accurate this would be.\nResponse: Added wording to clarify this – yes we capture out-of-hospital deaths using primary care data linked to official registers of deaths.\n- Statistical analysis plan Aim 1: Nosocomial infections potentially related to antibiotic such as Clostridium difficile - this should be defined better.\nResponse: Thank you for the feedback. This has been reworded.\n- Statistical analysis plan Aim 2: The microbiological sampling considered in the study should be specified. Sputum and bronchoscopy samples are a given, but what about nasopharyngeal swab or blood culture?\nResponse: Updated to specify this. Sputum, bronchoscopy samples, pleural fluid, blood culture and nasopharyngeal swabs will all be captured and analysed, together and separately.\n- Statistical analysis plan Aim 2: How will putative pathogen be defined? It will need interpretation of the culture results, as culture may yield multiple organisms and many organisms can be considered contaminants (e.g. candida, oral flora bacteria).\nResponse: Within our hospital sites oral flora bacteria are reported in those terms – ie “mixed oral flora” or “mixed upper respiratory tract flora only”, rather than as specific organisms identified. There remains an inability to differentiate between colonisation/contamination and infection from microbiology report alone, however more detailed analysis on a patient-by-patient basis would not be feasible across the whole dataset. For the purposes of this study we will count any bacteria identified, reviewed by microbiology and reported to clinicians to a genus or species level as a potential pathogen. Rates of candida or other environmental fungi identified will be reported but not used as part of this analysis, which is focused primarily on the spectrum of antibiotic usage. Text changed to clarify this, and reinforce these are potential pathogens only.\n- Statistical analysis plan Aim 2: How will the study describe antimicrobial used? Would the antimicrobial agents be listed individually or in groups? I would suggest to describe group of antibiotics based on coverage such as anti-MRSA coverage, anti-pseudomonal coverage etc.\nResponse: We will group antibiotics by spectrum of activity, and compare proportions received of each group in patients with and without an organism identified. Text updated to clarify this.\n- Statistical analysis plan Aim 2: I don't understand what rationalized mean in \"The proportion of patients for whom antibiotics were rationalized following microbiological sampling will be calculated.\" Please explain\nResponse: We agree this could be worded more clearly. We aim to capture what proportion of patients had their antimicrobial regime changed in light of microbiological results, for example to a narrower-spectrum antibiotic, or changed to add additional cover. Text updated to hopefully make this more explicit and specific.\n- Statistical analysis plan Aim 2: Sometimes a culture may be helpful even if it does not grow anything. For example, in a patient whose sputum gram stain showed no gram-positive cocci and culture did not grow MRSA, then some clinicians may discontinue MRSA coverage. The authors should consider accounting for this and look at if culture taken (whether positive or negative) can lead to narrowing of antimicrobials.\nResponse: Thank you for this suggestion. In addition to the impact of positive microbiology, we will indeed examine the impact on antimicrobial prescribing of taking cultures. Text updated to reflect this.\n- Statistical analysis plan Aim 4: One important system factor would be if the hospitals have respirology / pulmonology service and bronchoscopy availability\nResponse: This is a key system factor, however all hospitals included in the analysis have these services available on site.\n- Statistical analysis plan Aim 5: I understand the rationale for this objective. However, one criticism to anticipate is that there is clearly confounding by indication with respect of obtaining cultures on a patient. There is a reason that clinicians ordered sputum cultures or performed bronchoscopy on one patient but not in the other. In the end, these two groups will be fundamentally different that the propensity score weighting will not be able to balance them and there will always be residual confounding.\nResponse: We understand – and largely agree with – this criticism. However, clinician request is likely not the only factor impacting whether cultures are sent, something which will hopefully be illuminated by aim 4.\nFurther, as the ATS 2016 guidelines recommend all patients with suspected HAP undergo culture of lower respiratory tract secretions, we believe the indication for sputum culture should be present for all included patients. Sputum and bronchoscopy samples will be assessed separately, as we agree that clearly patients who are selected to undergo bronchoscopy will represent a very different cohort from those who are not. Among the NV-HAP cohort, very few are anticipated to have undergone bronchoscopy. Despite the clear challenges with confounding, we believe this is an important area to examine.\n- Statistical analysis plan Aim 5: There should be description of how the outcomes will be compared between the two groups after propensity score weighting.\nResponse: Mortality will be assessed with a Cox proportional hazards analysis. Added to the text to clarify.\n- Statistical analysis plan Aim 6: There should be description of how natural language processing will be done and how the gold standard of consolidation is done.\nResponse: Methodology for this is under development. We have added that the gold standard we are using is radiologist reports, and that we will provide information on methods to develop and validate results with any subsequent results paper.\nMinor edits\nResponse: Thank you for flagging these. Several of these typos, plus additional inconsistencies in tense, seem to have been introduced during the copyedit process, for which we apologies.\nAlongside their report, reviewers assign a status to the article:\n- Approved\n- Approved with reservations\n- Not approved\n| Invited Reviewers | ||\n|---|---|---|\n| 1 | 2 | |\n| Version 2 (revision) 15 Sep 25 | read | read |\n| Version 1 23 Apr 25 | read | read |\nSign up for content alerts\nYou are now signed up to receive this alert\nAlongside their report, reviewers assign a status to the article:\nApproved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested\nApproved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.\nNot approved - fundamental flaws in the paper seriously undermine the findings and conclusions\nProvide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:\nExamples of 'Non-Financial Competing Interests'\n- Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper.\n- You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors.\n- You are a close professional associate of any of the authors (e.g. scientific mentor, recent student).\n- You work at the same institute as any of the authors.\n- You hope/expect to benefit (e.g. favour or employment) as a result of your submission.\n- You are an Editor for the journal in which the article is published.\nExamples of 'Financial Competing Interests'\n- You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements.\n- You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors.\n- You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on.\nSign up for content alerts and receive a weekly or monthly email with all newly published articles\nRegister with NIHR Open Research\nAlready registered? Sign in\nclose\nError\nIf you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research.\nWe'll keep you updated on any major new updates to NIHR Open Research\nSign In\nIf you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.\nEmail us for further assistance.\nThe email address should be the one you originally registered with F1000.\nEmail address not valid, please try again\nYou registered with F1000 via Google, so we cannot reset your password.\nTo sign in, please click here.\nIf you still need help with your Google account password, please click here.\nYou registered with F1000 via Facebook, so we cannot reset your password.\nTo sign in, please click here.\nIf you still need help with your Facebook account password, please click here.\nCode not correct, please try again\nServer error, please try again.\nIf your email address is registered with us, we will email you instructions to reset your password.\nIf you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.\nPlease wait...","source_license":"CC-BY-4.0","license_restricted":false}