Gaining insights into a funding portfolio through publication tracking

funding, portfolio, publications, citations, case studies, Research Excellence Framework ALL Metrics - Views Downloads How to cite this article Suri-Ogilvie R, Hicks S, Capostagno D et al. Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.14022.3) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Research Article Revised Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved] Reetika Suri-Ogilvie https://orcid.org/0000-0001-5509-2502 1, Sandra Hicks1, Dominique Capostagno1, Ashley Banks1, Elena Ahmed1, Kelly Makarona1Reetika Suri-Ogilvie https://orcid.org/0000-0001-5509-2502 1, Sandra Hicks1, [...] Dominique Capostagno1, Ashley Banks1, Elena Ahmed1, Kelly Makarona1 PUBLISHED 01 Dec 2025 Author details Author details 1 NIHR Infrastructure, National Institute of Health and Care Research, London, UK Reetika Suri-Ogilvie Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing Sandra Hicks Roles: Data Curation, Formal Analysis Roles: Data Curation, Formal Analysis Dominique Capostagno Roles: Data Curation, Methodology, Writing – Review & Editing Roles: Data Curation, Methodology, Writing – Review & Editing Ashley Banks Roles: Data Curation, Methodology, Writing – Review & Editing Roles: Data Curation, Methodology, Writing – Review & Editing Elena Ahmed Roles: Data Curation, Formal Analysis Roles: Data Curation, Formal Analysis Kelly Makarona Roles: Supervision, Writing – Review & Editing Roles: Supervision, Writing – Review & Editing OPEN PEER REVIEW REVIEWER STATUS The National Institute for Health and Care Research (NIHR) is the UK’s biggest funder for health and social care research, funded by the Department of Health and Social Care (DHSC). The NIHR infrastructure provides research expertise, specialist facilities, a research delivery workforce and support services, all of which help to support and deliver the research we fund, and research funded by others. The NIHR is committed to maximising the impact of the research we support and fund 1 and therefore, it is crucial for the organisation to understand the mechanisms for the movement of research between these different pieces of research infrastructure and pathways to impact on the health and wealth of the nation. The aim of this article is to share our approach to developing an understanding of pathways to impact, enablers and barriers and lessons learnt. We used publications reported to us by our infrastructure as receiving infrastructure support and forward and backward citation analysis to trace infrastructure support for REF 2021 impact case studies and research that has had an impact on policy. We used these data to develop impact case studies for NIHR infrastructure. Of the 6,361 REF impact case studies that are publicly available, the NIHR infrastructure has supported 327 of which 59 are supported by more than one scheme. Through our forward and backward citation analysis we have also developed impact case studies in the following NIHR priority areas: - Reducing health inequalities - Digital health - Artificial intelligence - Workforce resilience The use of forward and backward citation analysis can also help research funders to understand how research is moving between different parts of their funding portfolios, pathways to impact and any gaps and opportunities. However, this comes with some challenges which need mitigation. The National Institute of Health and Care research (NIHR) is funded by the Department of Health and Social Care. It funds a £600M/year portoflio of infrastructure which provided support and a delivery workforce for experimental and clinical research in England. NIHR infrastructure report to the NIHR on an annual basis and one of the reporting requirements is a list of research publications that have been supported by the infrastructure. Using these it is possible to trace which research publications have either been cited by NIHR infrastructure supported research publications and which research publications cite NIHR infrastructure supported publications. This helps the NIHR to understand what types of research and within its portfolio and how the research moves within the health and care ecosystem to have an impact on the real world. This is important for the NIHR to be able to demonstrate the value of publicly funded research and fulfil its mission of improving the health and wealth of the nation. In this article we describe our methods for doing this to enable other funders to be able to learn from our experience. funding, portfolio, publications, citations, case studies, Research Excellence Framework Corresponding Author(s) Reetika Suri-Ogilvie ([email protected]) Grant information: The author(s) declared that no grants were involved in supporting this work. Copyright: © 2025 Suri-Ogilvie R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Suri-Ogilvie R, Hicks S, Capostagno D et al. Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.14022.3) First published: 22 Jul 2025, 5:62 (https://doi.org/10.3310/nihropenres.14022.1) Latest published: 01 Dec 2025, 5:62 (https://doi.org/10.3310/nihropenres.14022.3) We have corrected three minor typos as kindly pointed out by our reviewers. These do not materially change the content of the manuscript. No new figures, changes to authors or affiliations. We have only fixed minor typos which do not materially change the content of the manuscript. We have corrected three minor typos as kindly pointed out by our reviewers. These do not materially change the content of the manuscript. No new figures, changes to authors or affiliations. We have only fixed minor typos which do not materially change the content of the manuscript. See the authors' detailed response to the review by Erik Canton See the authors' detailed response to the review by Josie Coburn See the authors' detailed response to the review by Shaoxiong Brian Xu The National Institute for Health and Care Research (NIHR) is the UK’s biggest funder for health and social care research, funded by the Department of Health and Social Care (DHSC). The NIHR invests approximately £600M/year in research infrastructure2, which provides a platform to enable early stage and applied health research. The NIHR infrastructure provides research expertise, specialist facilities, a research delivery workforce and support services, all of which help to support and deliver the research we fund, and research funded by others. The NIHR’s research delivery infrastructure is focused on supporting clinical trial delivery, whereas, its research infrastructure portfolio is made up of four schemes which span the innovation pathway from early phase experimental research through to clinical evaluation, implementation and adoption and spread. Our Biomedical Research Centres (BRCs)3 translate discovery research into clinical settings. They take innovations or interventions that have evidence of proof of concept and study translatability into NHS settings. Our Patient Safety Research Collaborations (PSRCs, and their precursor)4 evaluate patient safety for innovations and interventions for NHS use. The HealthTech Research Centres (HRCs, and their precursors)5 evaluate health technology based interventions for NHS use and our Applied Health and Social Care Collaborations (ARCs, and their precursor)6 focus on implementation science and evaluations in real-world settings to help build the evidence-base for scale up and roll-out within NHS settings. The NIHR is committed to maximising the impact of the research we support and fund1 and therefore, it is crucial for the organisation to understand the mechanisms for the movement of research between these different pieces of research infrastructure and pathways to impact on the health and wealth of the nation. NIHR infrastructure funding can be used towards pump priming funding. However, it is largely aimed at supporting research and research delivery workforce roles which enable research to take place and career development. The research that is supported by NIHR infrastructure is typically either exclusively or partly funded by external funding. This makes attributing the impact of research projects to NIHR infrastructure difficult due to a lack of a direct, linear relationship between the infrastructure funding and specific projects in the majority of cases. The Research Excellence Framework (REF)7 is a crucial data source for impact case studies for the NIHR because it is the UK’s exercise for evaluating the quality of research produced by Higher Education Institutes (HEIs), approximately every 7 years. HEIs produce independently evidenced impact case studies which have been underpinned by excellent research from the submitting HEI. Impact case studies that do not contain commercially or otherwise sensitive information are made publicly available at the end of the assessment exercise. The last REF exercise that had been conducted at the time of writing was in 2021. The aim of this article is to share our approach to developing an understanding of pathways to impact, enablers and barriers and lessons learnt. The REF 2021 impact case study dataset. The REF 2021 impact case study dataset was downloaded from the REF 2021 website8. At the time of download, there were 6,361 publicly available impact case studies. Each impact case study contains information on the summary of the impacts, the underpinning research, the details of the impacts and list of publications and sources of evidence of the impacts. NIHR infrastructure publication data. Each year, NIHR infrastructure award holders submit annual reports to the NIHR. These contain information about publications that have been supported by the infrastructure. The publication data provided include a publication reference in the form of a publication citation as well as the associated Digital Object Identifiers (DOI). In some cases, Pubmed or Pubmed Central identifiers are provided instead of the DOI. Where an identifier is missing, the publication reference is used to query Pubmed to systematically source the relevant information. Publication identifiers are validated using Dimensions, Europe Pubmed Central and Pubmed. Linking up REF data with NIHR infrastructure publication data. The list of publications section of the REF 2021 impact case studies could also be optionally used by HEIs to include information about funding for the underpinning research but we found that this option was inconsistently used across the HEIs. When used, we found that this optional element typically covered project funding for which a specific amount of funding was directly attributable to the project, which is not a feature of NIHR infrastructure funding. Therefore, we used the list of publications from the REF 2021 impact case studies to link the case studies to NIHR infrastructure support. Additional citation analysis. Using forward and backward citation analysis9, as shown in Figure 1, we also traced the movement of research as follows: Backward citations provide an indication of where research has been ‘pulled’ into the NIHR infrastructure from and forward citations provide an indication of where research from the NIHR infrastructure has been ‘pushed’ along the innovation pathway into further research or real-world settings. We then ran ARC publications that had been underpinned by BRCs, PSRCs or HRCs through Overton10 to search for policy citations. The policy papers were manually coded to the following NIHR priority areas of interest: NIHR case study development and validation. The REF 2021 impact case studies were coded to NIHR priority areas based on key word searches, for example ‘health inequalities’, ‘digital’, ‘digital health’, ‘artificial intelligence’, ‘workforce’ within the titles, impact summaries and details of the impact section. These were manually cross-checked. Case studies in our priority areas that had been supported by collaborative working across more than one infrastructure scheme were then selected for adaptation to create NIHR impact case studies to help us to understand how different pieces of infrastructure work together. We also developed impact cases studies in the four areas of policy interest based on ‘bodies of work’ involving multi-disciplinary researchers and research teams from different parts of NIHR research infrastructure. Semi-structured interviews were conducted with researchers or representatives of research teams and thematically analysed using inductive theming in order to understand how different pieces of NIHR infrastructure work together, enablers, barriers and challenges and evidence of impact. Any impact claims were validated using data sources that were independent of the researchers involved. Patient and Public Involvement. Patients and the public were not involved in this research because it is not health research. The authors of this article work for the NIHR and have conducted this research on research to understand what is funded and supported by the NIHR, to provide transparency and accountability for the public money that the NIHR spends on research. Of the 6,361 REF impact case studies that are publicly available, the NIHR infrastructure has supported 327 of which 59 are supported by more than one scheme. Figure 2 shows the numbers for how many REF impact case studies were supported by each type of NIHR infrastructure and where impact case studies were supported by just one infrastructure scheme. Figure 3 breaks the total number of REF impact case studies supported by NIHR infrastructure i.e 327 into the different types of impact, as classified by the REF dataset. The interviews that formed part of the case study development process explored how different pieces of NIHR infrastructure work together, enablers and barriers for impact. This revealed five areas of strength in terms of how NIHR infrastructure delivers impact and areas for future work to better understand some of these mechanisms. The first is strategic partnerships and collaborations between different pieces of NIHR infrastructure and how these help to move projects down the innovation pathway11. An example of this is a strategic partnership between the MindTech HRC12 and the SLaM BRC13. The BRC brings expertise in the early stage investigation of mechanisms that underpin new interventions or technologies. The HRC brings expertise in the evaluation of these interventions or technologies in NHS settings. By working together, these two pieces of infrastructure can ensure that minimal time and money is lost in developing products that won’t work in the NHS. This way of working is also called ‘fast failure’14 and enables the movement of interventions from bench to bedside quicker than the typical 10–15 year time lag between research and impact15. The ability to embed research in the NHS is a key strength of NIHR infrastructure. An example of this is research16 by the Yorkshire and Humber PSRC which looked at the effects of wellbeing and burnout of healthcare professionals on patient safety. Prior to this research, the interest in NHS workforce resilience was primarily on retention and skills rather than wellbeing. The direct involvement of clinical and academic staff in research projects also means that clinical data that are not routinely collected can be monitored to help understand unmet needs but also build the evidence base for novel interventions. The case study development process revealed close working relationships between the NIHR infrastructure, particularly the ARCs, and commissioners through the Health Innovation Networks (HINs)17. The HINs support the evaluation and implementation of innovations in the NHS to address specific challenges and foster economic growth. This allows NIHR infrastructure to build the evidence base for and push new interventions and technologies into the NHS but also pull ideas and interventions from the HINs to evaluate in various ways. An example of this is ARC North Thames which evaluated the benefits of co-locating welfare advice services in GP settings, demonstrating significant improvements in patient mental health and well-being, reaching those most in need and supporting healthcare staff. This directly led to services being retained in primary care settings in specific parts of England and roll-out nationally in Scotland18. Understanding the push/pull mechanisms between the NIHR infrastructure and the HINs and wider health and care system is an area for future work. The next area of strength is core funding. NIHR infrastructure contracts tend to be at least five years in length which means that infrastructure centres have well developed patient and public involvement and engagement (PPIE)19, business development, and intellectual property management functions which enable early and seamless input but also the ability to quickly pivot towards addressing new or emerging health and care system needs. In the absence of these, external funding would need to be sought for specific projects which could cause administrative delays or lead to the ‘death’ of projects considered high risk or not in line with specific funder priorities. The infrastructure also has established mechanisms for sharing and linking with various databases and sources which enrich the research. Another area for future work is understanding why this ‘valley of death’20 exists for projects and what additional support can be provided to help. The NIHR infrastructure supports PhD students, post-docs and other researchers from a range of disciplines and at different career stages, recruited to support specific projects. Core funding enables a greatly reduced reliance on external, short-term, precarious funding21 to attract and retain research talent. The buy-out of their time allows them to develop and explore new ideas, sometimes using seed funding from the infrastructure, to get them to a stage where external funding can be sought. Early career researchers also have opportunities to co-lead various initiatives in the PPIE or knowledge mobilisation space, for example, which helps with career development for succession planning within the infrastructure. The NIHR Academy22 provides training and support to health and care researchers at all stages of their careers to build national health and care research capacity and capability. Evaluating our capacity building programmes helps us to understand their value and further develop them to meet the dynamic needs of our workforce. The interviews also revealed a specific need for more join up across the social care system, which is fragmented due to the fact that it is made up of several private organisations, commissioned to deliver NHS or community care services. This poses a challenge for adoption, implementation and downstream impacts on patients, due to the amount of time and costs of working with multiple stakeholders across the care system. Targeted funding to support researchers with policy- or implementation-ready research findings to work with multiple companies within the care system could help with this. Publications are typically used to track what’s in a funding portfolio in terms of research areas but also reach of research through various altmetrics. The use of forward and backward citation analysis can also help research funders to understand how research is moving between different parts of their funding portfolios, pathways to impact and any gaps and opportunities. However, this does come with some challenges around data quality and timeliness, for example, which need mitigation. The data are publicly available: REF 2021 impact case studies8. NIHR impact case studies23. Research publications can be accessed using databases such as Pubmed or Europe PMC. We have used Dimensions and Overton for altmetric analyses. We would like to thank the NIHR teams that have contributed to the development of the thinking presented in this paper and our infrastructure awardees for completing our annual reports and for their time for interviews. Faculty Opinions recommendedReferences - 1. National Institute for Health and Care Research: Our impact vision. [Accessed December 2024]. Reference Source - 2. National Institute of Health and Care Research: Investing in infrastructure. [Accessed December 2024]. Reference Source - 3. National Institute of Health and Care Research: Biomedical research centres. [Accessed December 2024]. Reference Source - 4. National Institute for Health and Care Research: Patient safety research collaborations. [Accessed December 2024]. Reference Source - 5. National Institute of Health and Care Research: HealthTech research centres. [Accessed December 2024]. Reference Source - 6. National Institute for Health and Care Research: Applied research collaborations. [Accessed December 2024]. Reference Source - 7. Research Excellence Framework: Securing a world-class, dynamic and responsive research base across the full academic spectrum within UK higher education. January 2025. Reference Source - 8. Research Excellence Framework: Impact case study database. January 2025. https://results2021.ref.ac.uk/impact - 9. Haddaway NR, Grainger MJ, Gray CT: Citationchaser: a tool for transparent and efficient forward and backward citation chasing in systematic searching. Res Synth Methods Software Focus. 2022; 13(4): 533–545. PubMed Abstract | Publisher Full Text - 10. Overton: We connect research with policy. [Accessed 2024]. Reference Source - 11. Office for Life Sciences: A guide to navigating the innovation pathway in England. 2016. Reference Source - 12. NIHR HealthTech Research Centre: Technology innovation for better mental health. MindTech. [Accessed January 2025]. Reference Source - 13. NIHR Maudsley Biomedical Research Centre: Increased air pollution exposure during midlife may harm brain health as we age. [Accessed January 2025]. Reference Source - 14. National Institute for Health and Care Research: FAST innovation to improve the health and wealth of the nation. [Accessed January 2025]. Reference Source - 15. Research Impact Canada: 17 years from research to impact – is this true? [Accessed January 2025]. Reference Source - 16. Prudenzi A, Graham CD, Flaxman PE, et al.: Wellbeing, burnout, and safe practice among healthcare professionals: predictive influences of mindfulness, values, and self-compassion. Psychol Health Med. 2022; 27(5): 1130–1143. PubMed Abstract | Publisher Full Text - 17. Health Innovation Network: We are the health innovation adoption experts. [Accessed January 2025]. Reference Source - 18. NIHR Applied Research Collaboration North Thames: The mental health, economic and societal benefits of co-locating welfare advice in primary care. [Accessed January 2025]. Reference Source - 19. National Institute of Health and Care Research: NIHR announces new standards for public involvement in research. [Accessed January 2025]. Reference Source - 20. Editorial: Traversing the valley of death. Nat Rev Bioeng. 2023; 1: 875. Publisher Full Text - 21. University and College Union: New report exposes 'gig-economy' reality of prestigious university research departments. [Accessed January 2025]. Reference Source - 22. National Institute of Health and Care Research: NIHR academy. Reference Source - 23. National Institute for Health and Care Research: What we do. [Accessed December 2024]. https://www.nihr.ac.uk/our-research/stories?funding_programme%5BInfrastructure%5D=Infrastructure Author details Author details 1 NIHR Infrastructure, National Institute of Health and Care Research, London, UK Reetika Suri-Ogilvie Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing Sandra Hicks Roles: Data Curation, Formal Analysis Roles: Data Curation, Formal Analysis Dominique Capostagno Roles: Data Curation, Methodology, Writing – Review & Editing Roles: Data Curation, Methodology, Writing – Review & Editing Ashley Banks Roles: Data Curation, Methodology, Writing – Review & Editing Roles: Data Curation, Methodology, Writing – Review & Editing Elena Ahmed Roles: Data Curation, Formal Analysis Roles: Data Curation, Formal Analysis Kelly Makarona Roles: Supervision, Writing – Review & Editing Roles: Supervision, Writing – Review & Editing Competing interests The authors of this article work for the NIHR. Grant information The author(s) declared that no grants were involved in supporting this work. Article Versions (3) Copyright © 2025 Suri-Ogilvie R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article Suri-Ogilvie R, Hicks S, Capostagno D et al. Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.14022.3) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 3 VERSION 3 PUBLISHED 01 Dec 2025 Revised Views 0 How to cite this report: Heyard R. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38755) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38755 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38755 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 07 Jan 2026 Not Approved VIEWS 0 The paper by Suri-Ogilvie presents an interesting analysis on the actual impact of the research infrastructure funded by NIHR. By linking data from the REF 2021 and data/information on publications of research supported by NIHR infrastucture (even if externally funded) ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above. Close The paper by Suri-Ogilvie presents an interesting analysis on the actual impact of the research infrastructure funded by NIHR. By linking data from the REF 2021 and data/information on publications of research supported by NIHR infrastucture (even if externally funded) the authors bring an innovative / novel perspective to the impact of infrastructure funding. The linked dataset was then augmented via a foreward-backward citation search and a search for citations in policy documents (through Overton). On top of this already very interesting data and quantititative analysis, the authors also conducted interviews to better understand links between infrastructure funding and impact beyond publications. While the data collected and linked by the authors is extremely rich and has great potential, the presented manuscript lacks transparency in crucial aspects of the methodology and the presentation of the results. First of all, the semi-structured interviews, which seem to be crucial for the development of the case studies and take a center stage in the results are first mentioned on page 5, at the very end of the methods. There is no mention of interviews in the abstract or title or the data sources. Interview transcripts etc should in my opinion be seen as (qualitative) data. Specifically with respect to the interviews that were conducted the manuscript is lacking transparency. I'd recommend the authors have a look at a reporting guideline for research based on interviews (e.g., CREDES). How were researchers or representatives selected or found? Are these authors of the publications included in the bibliographic analysis, or other? How many interviews with how many researchers were conducted? Were they online or onsite? Who conducted the interviews? Was there an interview guide? Were interviews piloted? ETC. Further, a table or so with some descriptive statistics on both interviewers and interviewees is needed (e.g., ECRs or established researchers? disciplines represented? ...) Also for the bibliographic data analysis I am missing transparency - I personally would not be able to reproduce the analysis. For example, the authors need to provide dates of access of the various datasets, since they seem to be "living", i.e., updated regularly. For the backward and forward citation, it should say in the methods that the search was done within Dimensions; and I would have like to find the list of initial DOIs somewhere, and again importantly the date of the search - which would today already give different results for the forward part. The same goes for Overton, possibly also the search string that was used. For the keyword search conducted on titles etc of the REF impact case studies, I would like to see the whole list of keywords somewhere (if it is long, in an appendix) and not just a selection. As for the presentation of the results, the discussion of the quantitative analysis is very limited. I would have expected some sort of plot / diagram showing the linkage of the different data sources (maybe a sankey plot or similar?). Figure 2, for me, is non sensical - what data is represented here? It is not the 327 case studies, right? since then, the numbers do not add up. I could not find any mention of the policy paper analysis in the results - or at least I could not understand how they contributed to the results. The discussion then continues with a long presentation of case studies - but it is unclear how they came together from the presented methodology. It therefore reads a bit like cherry picked stories and henceforth not very trustworth. All in all, I find that the data and the idea presented by the authors have a lot of potential - I however am less convinced with the execution, mainly because the quality of the reporting and the transparency leave lots of open questions. While the data collected and linked by the authors is extremely rich and has great potential, the presented manuscript lacks transparency in crucial aspects of the methodology and the presentation of the results. First of all, the semi-structured interviews, which seem to be crucial for the development of the case studies and take a center stage in the results are first mentioned on page 5, at the very end of the methods. There is no mention of interviews in the abstract or title or the data sources. Interview transcripts etc should in my opinion be seen as (qualitative) data. Specifically with respect to the interviews that were conducted the manuscript is lacking transparency. I'd recommend the authors have a look at a reporting guideline for research based on interviews (e.g., CREDES). How were researchers or representatives selected or found? Are these authors of the publications included in the bibliographic analysis, or other? How many interviews with how many researchers were conducted? Were they online or onsite? Who conducted the interviews? Was there an interview guide? Were interviews piloted? ETC. Further, a table or so with some descriptive statistics on both interviewers and interviewees is needed (e.g., ECRs or established researchers? disciplines represented? ...) Also for the bibliographic data analysis I am missing transparency - I personally would not be able to reproduce the analysis. For example, the authors need to provide dates of access of the various datasets, since they seem to be "living", i.e., updated regularly. For the backward and forward citation, it should say in the methods that the search was done within Dimensions; and I would have like to find the list of initial DOIs somewhere, and again importantly the date of the search - which would today already give different results for the forward part. The same goes for Overton, possibly also the search string that was used. For the keyword search conducted on titles etc of the REF impact case studies, I would like to see the whole list of keywords somewhere (if it is long, in an appendix) and not just a selection. As for the presentation of the results, the discussion of the quantitative analysis is very limited. I would have expected some sort of plot / diagram showing the linkage of the different data sources (maybe a sankey plot or similar?). Figure 2, for me, is non sensical - what data is represented here? It is not the 327 case studies, right? since then, the numbers do not add up. I could not find any mention of the policy paper analysis in the results - or at least I could not understand how they contributed to the results. The discussion then continues with a long presentation of case studies - but it is unclear how they came together from the presented methodology. It therefore reads a bit like cherry picked stories and henceforth not very trustworth. All in all, I find that the data and the idea presented by the authors have a lot of potential - I however am less convinced with the execution, mainly because the quality of the reporting and the transparency leave lots of open questions. - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? No - Are sufficient details of methods and analysis provided to allow replication by others? No - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? No - Are the conclusions drawn adequately supported by the results? Partly

1. Tong A, Sainsbury P, Craig J: Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care. 2007; 19 (6): 349-357 Publisher Full TextCompeting Interests: No competing interests were disclosed. Reviewer Expertise: Statistics; funding allocation systems; meta-research; reproducibility; open science CITE HOW TO CITE THIS REPORT Heyard R. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38755) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38755 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38755 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Canton E. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38606) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38606 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38606 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 07 Jan 2026 Erik Canton, Directorate-General for Research and Innovation, European Commission, Brussels, Belgium Approved VIEWS 0 The authors have improved the manuscript based on the feedback provided. The authors did not provide a more elaborate discussion on the forward and backward citation analysis in the revised version. If this information is available through other means I ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close The authors have improved the manuscript based on the feedback provided. The authors did not provide a more elaborate discussion on the forward and backward citation analysis in the revised version. If this information is available through other means I would encourage the authors to include a reference so that interested readers can understand the results of the analysis more in-depth. Competing Interests: No competing interests were disclosed. CITE HOW TO CITE THIS REPORT Canton E. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38606) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38606 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38606 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Bloch C. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38756) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38756 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38756 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 07 Jan 2026 Approved VIEWS 0 This paper is clear and provides and interesting description of the NIHR's approach to study impact through the linking of different grants. I can also see that the authors have addressed comments to previous versions of this paper. The use ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close This paper is clear and provides and interesting description of the NIHR's approach to study impact through the linking of different grants. I can also see that the authors have addressed comments to previous versions of this paper. The use of examples of cases help to illustrate what can be gained from this method. One point that might be worth further consideration and clarification is the "forward and backward citation analysis". There still seems to be some uncertainty of what this entails. For example is it mainly just used to link grants (as mentioned in the manuscript), or are these forward and backward citations used in greater detail to aid in understanding how subsequent grant work (as described in the citing paper) was based on the previous research? One point that might be worth further consideration and clarification is the "forward and backward citation analysis". There still seems to be some uncertainty of what this entails. For example is it mainly just used to link grants (as mentioned in the manuscript), or are these forward and backward citations used in greater detail to aid in understanding how subsequent grant work (as described in the citing paper) was based on the previous research? - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? Yes - Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Research funding evaluation, analysis of societal impact of research CITE HOW TO CITE THIS REPORT Bloch C. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38756) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38756 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38756 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Xu SB. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38607) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38607 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38607 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 23 Dec 2025 Approved with Reservations VIEWS 0 The revised manuscript contains no mention of retracted publications. If the study did not involve retracted publications, it should be explicitly stated in the methods section, and a one-sentence mention would do. If the study did involve retracted publications, how ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close The revised manuscript contains no mention of retracted publications. If the study did not involve retracted publications, it should be explicitly stated in the methods section, and a one-sentence mention would do. If the study did involve retracted publications, how they were handled should be disclosed. Competing Interests: No competing interests were disclosed. Reviewer Expertise: Retraction; Research integrity CITE HOW TO CITE THIS REPORT Xu SB. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15436.r38607) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38607 https://openresearch.nihr.ac.uk/articles/5-62/v3#referee-response-38607 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Version 2 VERSION 2 PUBLISHED 06 Oct 2025 Revised Views 0 How to cite this report: Xu SB. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15363.r37657) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37657 https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37657 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 18 Nov 2025 Approved with Reservations VIEWS 0 The authors have substantially improved the manuscript by incorporating feedback from previous reviewers, resulting in a fairly solid manuscript. However, one important methodological and analytical considerations require attention. As a retraction researcher, I have a fundamental concern ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close As a retraction researcher, I have a fundamental concern ... Continue reading The authors have substantially improved the manuscript by incorporating feedback from previous reviewers, resulting in a fairly solid manuscript. However, one important methodological and analytical considerations require attention. As a retraction researcher, I have a fundamental concern regarding the handling of retracted publications in the citation analysis. The manuscript does not specify whether retracted publications were included in the forward and backward citation analyses or how they were treated if encountered. Given the documented high-stakes nature of retracted research in medical and life sciences and evidence of continued post-retraction citations, any retracted publications should be explicitly excluded from the citation analysis to ensure accurate assessment of genuine research impact. I recommend conducting a focused analysis of excluded retracted publications, if any, that received NIHR funding or infrastructure support. This analysis would identify high-risk research areas and researchers requiring targeted research integrity interventions, providing valuable insights for the NIHR's quality assurance and oversight functions. Three specific issues need correction. First, the final sentence of the first paragraph in "Research design and methods" mentions "sources of evidence of the impacts" without adequate explanation - a brief introduction to these sources would enhance clarity. Second, the heading "our publication data" in the third paragraph of the same section lacks specificity and could be improved by changing it to "Linking up REF data with NIHR data" for better signposting. Third, the first sentence of the final paragraph in "Results and discussion" contains a grammatical error where "that is it made up of" should be corrected to "that it is made up of". As a retraction researcher, I have a fundamental concern regarding the handling of retracted publications in the citation analysis. The manuscript does not specify whether retracted publications were included in the forward and backward citation analyses or how they were treated if encountered. Given the documented high-stakes nature of retracted research in medical and life sciences and evidence of continued post-retraction citations, any retracted publications should be explicitly excluded from the citation analysis to ensure accurate assessment of genuine research impact. I recommend conducting a focused analysis of excluded retracted publications, if any, that received NIHR funding or infrastructure support. This analysis would identify high-risk research areas and researchers requiring targeted research integrity interventions, providing valuable insights for the NIHR's quality assurance and oversight functions. Three specific issues need correction. First, the final sentence of the first paragraph in "Research design and methods" mentions "sources of evidence of the impacts" without adequate explanation - a brief introduction to these sources would enhance clarity. Second, the heading "our publication data" in the third paragraph of the same section lacks specificity and could be improved by changing it to "Linking up REF data with NIHR data" for better signposting. Third, the first sentence of the final paragraph in "Results and discussion" contains a grammatical error where "that is it made up of" should be corrected to "that it is made up of". - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Partly - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Yes - Are all the source data underlying the results available to ensure full reproducibility? Yes - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: Retraction; Research integrity CITE HOW TO CITE THIS REPORT Xu SB. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15363.r37657) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37657 https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37657 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. - Author Response 04 Dec 2025Reetika Suri-Ogilvie, NIHR Infrastructure, National Institute of Health and Care Research, London, UK04 Dec 2025Author ResponseWe thank our reviewer for the time taken to read and comment on our manuscript and their kind and constructive feedback. We have fixed the minor typos that have been ... Continue reading We thank our reviewer for the time taken to read and comment on our manuscript and their kind and constructive feedback. We have fixed the minor typos that have been pointed out. While retracted publications are a very important consideration for publication analysis, taking a specific approach to analysing them was out of scope for this work. However, they will certainly be a consideration for future work. For this work, we took an annual data cut of papers that held the status of ‘published’ at the time of the data cut and took those papers forward into our downstream analysis. We hope that this response provides clarity on our approach.We thank our reviewer for the time taken to read and comment on our manuscript and their kind and constructive feedback. We have fixed the minor typos that have been pointed out.Competing Interests: No competing interests were disclosed. Close While retracted publications are a very important consideration for publication analysis, taking a specific approach to analysing them was out of scope for this work. However, they will certainly be a consideration for future work. For this work, we took an annual data cut of papers that held the status of ‘published’ at the time of the data cut and took those papers forward into our downstream analysis. We hope that this response provides clarity on our approach. COMMENTS ON THIS REPORT - Author Response 04 Dec 2025Reetika Suri-Ogilvie, NIHR Infrastructure, National Institute of Health and Care Research, London, UK04 Dec 2025Author ResponseWe thank our reviewer for the time taken to read and comment on our manuscript and their kind and constructive feedback. We have fixed the minor typos that have been ... Continue reading We thank our reviewer for the time taken to read and comment on our manuscript and their kind and constructive feedback. We have fixed the minor typos that have been pointed out. While retracted publications are a very important consideration for publication analysis, taking a specific approach to analysing them was out of scope for this work. However, they will certainly be a consideration for future work. For this work, we took an annual data cut of papers that held the status of ‘published’ at the time of the data cut and took those papers forward into our downstream analysis. We hope that this response provides clarity on our approach.We thank our reviewer for the time taken to read and comment on our manuscript and their kind and constructive feedback. We have fixed the minor typos that have been pointed out.Competing Interests: No competing interests were disclosed. Close While retracted publications are a very important consideration for publication analysis, taking a specific approach to analysing them was out of scope for this work. However, they will certainly be a consideration for future work. For this work, we took an annual data cut of papers that held the status of ‘published’ at the time of the data cut and took those papers forward into our downstream analysis. We hope that this response provides clarity on our approach. Views 0 How to cite this report: Coburn J. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15363.r37563) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37563 https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37563 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 31 Oct 2025 Approved VIEWS 0 Thank you for updating this article. I think the clarity has improved and I'm happy with the quality. I think one more minor typo has been introduced in the editing in the introduction: "from early phase experimental ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close I think one more minor typo has been introduced in the editing in the introduction: "from early phase experimental ... Continue reading Thank you for updating this article. I think the clarity has improved and I'm happy with the quality. I think one more minor typo has been introduced in the editing in the introduction: "from early phase experimental and research through to clinical evaluation" - I would suggest removing the "and" here. I think one more minor typo has been introduced in the editing in the introduction: "from early phase experimental and research through to clinical evaluation" - I would suggest removing the "and" here. Competing Interests: No competing interests were disclosed. Reviewer Expertise: Science, technology and innovation policy; science and technology studies; research on research; metascience; research targeting; research funding; research evaluation; research impact CITE HOW TO CITE THIS REPORT Coburn J. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15363.r37563) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37563 https://openresearch.nihr.ac.uk/articles/5-62/v2#referee-response-37563 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Version 1 VERSION 1 PUBLISHED 22 Jul 2025 Views 0 How to cite this report: Coburn J. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15250.r36488) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36488 https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36488 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 03 Sep 2025 Approved with Reservations VIEWS 0 This paper provides a very interesting investigation of the impacts of UK National Institute for Health and Care Research (NIHR) infrastructure funding by linking publications cited by UK Research Excellence Framework (REF) impact case studies with those supported by NIHR ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close This paper provides a very interesting investigation of the impacts of UK National Institute for Health and Care Research (NIHR) infrastructure funding by linking publications cited by UK Research Excellence Framework (REF) impact case studies with those supported by NIHR infrastructure funding, including which scheme(s) they were funded by; by tracing forwards and backwards citations associated with these publications; by identifying policy papers that cite these papers; by classifying them according to NIHR priority areas; and by developing NIHR impact case studies. I have a few suggestions to improve the clarity of the paper for readers who may not be familiar with some elements of the research, and to provide more details about the methods and results. In the introduction, the NIHR translational research infrastructure and four schemes are introduced. I would suggest adding a little more detail about how the translational research infrastructure relates to the (broader?) NIHR infrastructure, and about the four schemes introduced in paragraph 2 of the introduction. In the methods section, in the 'Additional citation analysis section', I would suggest adding more details about the forwards and backwards citation analysis, in particular with reference to the four schemes and how they relate to each other. This research involves REF impact case studies, NIHR infrastructure publications, forwards and backwards citation analysis, the four NIHR translational research schemes, and policy papers from Overton, which are then coded to NIHR priority areas of interest. I think I diagram providing more details about how these different entities relate to each other might help to improve the clarity of the data and methods section. Some more specific questions that could be clarified are: Where are the NIHR priority areas of interest from (bottom of page 4 and also reference number 23, which leads to a broader 'What we do' page and not to case studies)? Which key words were used for the key word searches and how were they identified (top of page 5)? Why were case studies selected that had been supported by more than one infrastructure scheme and not only one? What were the 'bodies of work' highlighted on page 5 and how were they used to develop impact case studies? Where are the NIHR case studies and how were they validated? What sort of interviews were conducted, for how long, with whom and how was the qualitative data analysed? In the results section, I would love to see more descriptive statistics about the data. For instance, how many publications were linked, how many links forwards and backwards were there between the different types of data, how many policy documents were linked, how many policy documents were assigned to each priority area? Can you say anything about the topics of the NIHR infrastructure grants, publications and policy documents compared to the topics of the case studies? Are they the same or different? The insights from research interviews are organised into 5 areas of strength and areas of future work. In the third area, Health Innovation Networks (HINs) are introduced, but not defined. I would suggest adding a little more detail about what HINs are and what they do. I find areas 3 and 1 to be quite similar. Please consider clarifying the difference between these two areas. There are also just a few minor typos, e.g. page 2: Plain English Summary – there are a few typos in this text and the clarity of it could be improved; page 5: Humbar = Humber; page 6: "In the absence of these, external funding would need to be sought for specific projects which could * administrative delays…", I think there is a word missing where I have inserted the *. Overall, I think this is an interesting and valuable paper. I would just like to see more details. This paper, which links funding proposals and REF impact case studies, may be of interest to you: Yaqub O et al 2023 (Ref 1) I have a few suggestions to improve the clarity of the paper for readers who may not be familiar with some elements of the research, and to provide more details about the methods and results. In the introduction, the NIHR translational research infrastructure and four schemes are introduced. I would suggest adding a little more detail about how the translational research infrastructure relates to the (broader?) NIHR infrastructure, and about the four schemes introduced in paragraph 2 of the introduction. In the methods section, in the 'Additional citation analysis section', I would suggest adding more details about the forwards and backwards citation analysis, in particular with reference to the four schemes and how they relate to each other. This research involves REF impact case studies, NIHR infrastructure publications, forwards and backwards citation analysis, the four NIHR translational research schemes, and policy papers from Overton, which are then coded to NIHR priority areas of interest. I think I diagram providing more details about how these different entities relate to each other might help to improve the clarity of the data and methods section. Some more specific questions that could be clarified are: Where are the NIHR priority areas of interest from (bottom of page 4 and also reference number 23, which leads to a broader 'What we do' page and not to case studies)? Which key words were used for the key word searches and how were they identified (top of page 5)? Why were case studies selected that had been supported by more than one infrastructure scheme and not only one? What were the 'bodies of work' highlighted on page 5 and how were they used to develop impact case studies? Where are the NIHR case studies and how were they validated? What sort of interviews were conducted, for how long, with whom and how was the qualitative data analysed? In the results section, I would love to see more descriptive statistics about the data. For instance, how many publications were linked, how many links forwards and backwards were there between the different types of data, how many policy documents were linked, how many policy documents were assigned to each priority area? Can you say anything about the topics of the NIHR infrastructure grants, publications and policy documents compared to the topics of the case studies? Are they the same or different? The insights from research interviews are organised into 5 areas of strength and areas of future work. In the third area, Health Innovation Networks (HINs) are introduced, but not defined. I would suggest adding a little more detail about what HINs are and what they do. I find areas 3 and 1 to be quite similar. Please consider clarifying the difference between these two areas. There are also just a few minor typos, e.g. page 2: Plain English Summary – there are a few typos in this text and the clarity of it could be improved; page 5: Humbar = Humber; page 6: "In the absence of these, external funding would need to be sought for specific projects which could * administrative delays…", I think there is a word missing where I have inserted the *. Overall, I think this is an interesting and valuable paper. I would just like to see more details. This paper, which links funding proposals and REF impact case studies, may be of interest to you: Yaqub O et al 2023 (Ref 1) - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? Partly - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? Partly - Are the conclusions drawn adequately supported by the results? Yes

1. Yaqub O, Malkov D, Siepel J: How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework. Research Evaluation. 2023; 32 (2): 273-285 Publisher Full TextCompeting Interests: No competing interests were disclosed. Reviewer Expertise: Science, technology and innovation policy; science and technology studies; research on research; metascience; research targeting; research funding; research evaluation; research impact CITE HOW TO CITE THIS REPORT Coburn J. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15250.r36488) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36488 https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36488 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. - Author Response 06 Oct 2025Reetika Suri-Ogilvie, NIHR Infrastructure, National Institute of Health and Care Research, London, UK06 Oct 2025Author ResponseWe would like to thank the reviewer for their time taken to provide feedback on our manuscript and for their positive and constructive comments. We have added a bit ... Continue reading We would like to thank the reviewer for their time taken to provide feedback on our manuscript and for their positive and constructive comments. We have added a bit more detail about each of our infrastructure schemes and how the research infrastructure differs from our research delivery infrastructure. We have also included a flow diagram to help with reader understanding of how we have thought about the movement of research within our research infrastructure portfolio. We have rectified reference 23. We have provided a bit more detail about the key word searches but we found these of limited utility due to the vagueness of the themes we were searching for, so the majority of the filtering was done manually, as has now been explained in the text. We have provided an explanation of the term ‘bodies of work’ in the text. We have also explained why case studies where different pieces of infrastructure had worked together were of interest to us and provided a bit more detail about the interviews we conducted and analysed. We have not included descriptive statistics for the citation analysis in this manuscript as we would like the focus of this manuscript to remain on pathways and mechanisms for impact and any gaps. With the citation analysis, we are simply showcasing a use case for tracing the movement of research. The NIHR has a commitment to the responsible use of metrics, and as such, we would not like to focus on numbers that were not meaningful for this piece of work because the focus of this manuscript is not on what ‘good’ should or might look like in terms of citations but on understanding how we might be able to use citations to follow the movement of research within our portfolio and towards impact. In the discussion, we have provided a bit more information about the HINs and have clarified the difference between strength 1 and 3. We appreciate that this was previously not as clearly articulated as we would have liked. We are undertaking a piece of work to look at clusters of research activity and expertise within our infrastructure portfolio. However, it is very difficult to directly compare project topics with case studies. There are a number of reasons for this. The NIHR infrastructure portfolio is made up of approximately 16,000 projects in any given financial year. These are typically at different stages of maturity in terms of impact. Additionally, we have found that the majority of successful impact examples come from ‘bodies of work’ which involve multi-disciplinary expertise and academics from different parts of infrastructure working with other stakeholders from across the health and care ecosystem, rather than individual projects. Furthermore, research can have unintended impacts, which, in some cases, the researchers are completely unaware of or have had no involvement with or influence over. A list of NIHR infrastructure supported projects can be found here.We would like to thank the reviewer for their time taken to provide feedback on our manuscript and for their positive and constructive comments.Competing Interests: No competing interests were disclosed. Close We have added a bit more detail about each of our infrastructure schemes and how the research infrastructure differs from our research delivery infrastructure. We have also included a flow diagram to help with reader understanding of how we have thought about the movement of research within our research infrastructure portfolio. We have rectified reference 23. We have provided a bit more detail about the key word searches but we found these of limited utility due to the vagueness of the themes we were searching for, so the majority of the filtering was done manually, as has now been explained in the text. We have provided an explanation of the term ‘bodies of work’ in the text. We have also explained why case studies where different pieces of infrastructure had worked together were of interest to us and provided a bit more detail about the interviews we conducted and analysed. We have not included descriptive statistics for the citation analysis in this manuscript as we would like the focus of this manuscript to remain on pathways and mechanisms for impact and any gaps. With the citation analysis, we are simply showcasing a use case for tracing the movement of research. The NIHR has a commitment to the responsible use of metrics, and as such, we would not like to focus on numbers that were not meaningful for this piece of work because the focus of this manuscript is not on what ‘good’ should or might look like in terms of citations but on understanding how we might be able to use citations to follow the movement of research within our portfolio and towards impact. In the discussion, we have provided a bit more information about the HINs and have clarified the difference between strength 1 and 3. We appreciate that this was previously not as clearly articulated as we would have liked. We are undertaking a piece of work to look at clusters of research activity and expertise within our infrastructure portfolio. However, it is very difficult to directly compare project topics with case studies. There are a number of reasons for this. The NIHR infrastructure portfolio is made up of approximately 16,000 projects in any given financial year. These are typically at different stages of maturity in terms of impact. Additionally, we have found that the majority of successful impact examples come from ‘bodies of work’ which involve multi-disciplinary expertise and academics from different parts of infrastructure working with other stakeholders from across the health and care ecosystem, rather than individual projects. Furthermore, research can have unintended impacts, which, in some cases, the researchers are completely unaware of or have had no involvement with or influence over. A list of NIHR infrastructure supported projects can be found here. COMMENTS ON THIS REPORT - Author Response 06 Oct 2025Reetika Suri-Ogilvie, NIHR Infrastructure, National Institute of Health and Care Research, London, UK06 Oct 2025Author ResponseWe would like to thank the reviewer for their time taken to provide feedback on our manuscript and for their positive and constructive comments. We have added a bit ... Continue reading We would like to thank the reviewer for their time taken to provide feedback on our manuscript and for their positive and constructive comments. We have added a bit more detail about each of our infrastructure schemes and how the research infrastructure differs from our research delivery infrastructure. We have also included a flow diagram to help with reader understanding of how we have thought about the movement of research within our research infrastructure portfolio. We have rectified reference 23. We have provided a bit more detail about the key word searches but we found these of limited utility due to the vagueness of the themes we were searching for, so the majority of the filtering was done manually, as has now been explained in the text. We have provided an explanation of the term ‘bodies of work’ in the text. We have also explained why case studies where different pieces of infrastructure had worked together were of interest to us and provided a bit more detail about the interviews we conducted and analysed. We have not included descriptive statistics for the citation analysis in this manuscript as we would like the focus of this manuscript to remain on pathways and mechanisms for impact and any gaps. With the citation analysis, we are simply showcasing a use case for tracing the movement of research. The NIHR has a commitment to the responsible use of metrics, and as such, we would not like to focus on numbers that were not meaningful for this piece of work because the focus of this manuscript is not on what ‘good’ should or might look like in terms of citations but on understanding how we might be able to use citations to follow the movement of research within our portfolio and towards impact. In the discussion, we have provided a bit more information about the HINs and have clarified the difference between strength 1 and 3. We appreciate that this was previously not as clearly articulated as we would have liked. We are undertaking a piece of work to look at clusters of research activity and expertise within our infrastructure portfolio. However, it is very difficult to directly compare project topics with case studies. There are a number of reasons for this. The NIHR infrastructure portfolio is made up of approximately 16,000 projects in any given financial year. These are typically at different stages of maturity in terms of impact. Additionally, we have found that the majority of successful impact examples come from ‘bodies of work’ which involve multi-disciplinary expertise and academics from different parts of infrastructure working with other stakeholders from across the health and care ecosystem, rather than individual projects. Furthermore, research can have unintended impacts, which, in some cases, the researchers are completely unaware of or have had no involvement with or influence over. A list of NIHR infrastructure supported projects can be found here.We would like to thank the reviewer for their time taken to provide feedback on our manuscript and for their positive and constructive comments.Competing Interests: No competing interests were disclosed. Close We have added a bit more detail about each of our infrastructure schemes and how the research infrastructure differs from our research delivery infrastructure. We have also included a flow diagram to help with reader understanding of how we have thought about the movement of research within our research infrastructure portfolio. We have rectified reference 23. We have provided a bit more detail about the key word searches but we found these of limited utility due to the vagueness of the themes we were searching for, so the majority of the filtering was done manually, as has now been explained in the text. We have provided an explanation of the term ‘bodies of work’ in the text. We have also explained why case studies where different pieces of infrastructure had worked together were of interest to us and provided a bit more detail about the interviews we conducted and analysed. We have not included descriptive statistics for the citation analysis in this manuscript as we would like the focus of this manuscript to remain on pathways and mechanisms for impact and any gaps. With the citation analysis, we are simply showcasing a use case for tracing the movement of research. The NIHR has a commitment to the responsible use of metrics, and as such, we would not like to focus on numbers that were not meaningful for this piece of work because the focus of this manuscript is not on what ‘good’ should or might look like in terms of citations but on understanding how we might be able to use citations to follow the movement of research within our portfolio and towards impact. In the discussion, we have provided a bit more information about the HINs and have clarified the difference between strength 1 and 3. We appreciate that this was previously not as clearly articulated as we would have liked. We are undertaking a piece of work to look at clusters of research activity and expertise within our infrastructure portfolio. However, it is very difficult to directly compare project topics with case studies. There are a number of reasons for this. The NIHR infrastructure portfolio is made up of approximately 16,000 projects in any given financial year. These are typically at different stages of maturity in terms of impact. Additionally, we have found that the majority of successful impact examples come from ‘bodies of work’ which involve multi-disciplinary expertise and academics from different parts of infrastructure working with other stakeholders from across the health and care ecosystem, rather than individual projects. Furthermore, research can have unintended impacts, which, in some cases, the researchers are completely unaware of or have had no involvement with or influence over. A list of NIHR infrastructure supported projects can be found here. Views 0 How to cite this report: Canton E. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15250.r36489) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36489 https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36489 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 23 Aug 2025 Erik Canton, Directorate-General for Research and Innovation, European Commission, Brussels, Belgium Approved with Reservations VIEWS 0 This paper by Suri-Ogilvie et al. on "Gaining insights into a funding portfolio through publication tracking" investigates the impact of the research supported and funded by the National Institute for Health and Care (NIHR) in the UK. To maximise the ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close This paper by Suri-Ogilvie et al. on "Gaining insights into a funding portfolio through publication tracking" investigates the impact of the research supported and funded by the National Institute for Health and Care (NIHR) in the UK. To maximise the impact of the research, the authors want to better understand the movement of research projects between the different pieces of translational research infrastructure and pathways. The authors use publication analysis of the impact case studies and conclude that forward and backward citation analysis of publications help funders understand how research is moving between different parts of their funding portfolios. The authors have chosen an important and interesting topic with direct policy relevance. I have a number of comments that could help the authors to further develop the analysis for the article and maybe also in follow-up research. First, as a general comment, the readability of the article can be improved. For readers not very familiar with the NIHR the text is sometimes difficult to follow, also because of the use of many acronyms and the sometimes very short explanatory text related to NIHR's organisation and functioning. Also, there are several typos and grammatical issues in the text that should be fixed (e.g. "mobilistion"). Second, there are some descriptive statistics and figures in the article which are interesting, but they could benefit from some further elaboration. Figure 1 shows how many impact case studies have been supported by the different types of NIHR infrastructure, whereas figure 2 shows the number of impact cases broken down by impact type. It would be interesting to see some more reflection on this figure 2, for example on efficiency/effectiveness of the research vis-à-vis these different impact types and on the robustness of the methodologies employed in these studies to infer causality. From the abstract and introduction I was also expecting some insights from the citation analysis, on backward and forward citations and how these citations help to move between the different pieces of research infrastructure. The publication tracking part mentioned in the manuscript's title should be further developed in the text, and the reader will be curious to learn more about this. Third, the authors do not provide a clear definition of funding portfolio and how portfolio considerations can help improve the impact of the funded research. Do they have in mind some type of ex ante portfolio model as in Canton, 2025 (Ref 1) with implications for project financing decisions (ex ante, so before starting the project) or do the authors mostly refer to an ex post approach, where the value is maximised of the ongoing set of projects within a portfolio? It seems that the authors have the latter approach in mind, but it might be interesting to also include some reflections in the article on financing decisions when portfolio considerations play a role. The authors seem to be interested in achieving societal value by promoting projects to move more quickly from low TRLs to higher ones, as covered by the NIHR, but they also refer to different impact types and it is not clear to me which mechanisms the authors want to emphasise, and how this relates to the citation analysis. More generally, and also referring to figure 2 in the article, the analysis leaves unanswered the question whether additional considerations, such as impact type, should matter for funding decisions, in addition to more traditional criteria such as academic excellence. It would be interesting to include some reflections on that in the article. In particular, can the impact of the funded projects be further increased by including portfolio considerations in the financing decisions of the projects, deviating from the more traditional merit-based approach? The authors have chosen an important and interesting topic with direct policy relevance. I have a number of comments that could help the authors to further develop the analysis for the article and maybe also in follow-up research. First, as a general comment, the readability of the article can be improved. For readers not very familiar with the NIHR the text is sometimes difficult to follow, also because of the use of many acronyms and the sometimes very short explanatory text related to NIHR's organisation and functioning. Also, there are several typos and grammatical issues in the text that should be fixed (e.g. "mobilistion"). Second, there are some descriptive statistics and figures in the article which are interesting, but they could benefit from some further elaboration. Figure 1 shows how many impact case studies have been supported by the different types of NIHR infrastructure, whereas figure 2 shows the number of impact cases broken down by impact type. It would be interesting to see some more reflection on this figure 2, for example on efficiency/effectiveness of the research vis-à-vis these different impact types and on the robustness of the methodologies employed in these studies to infer causality. From the abstract and introduction I was also expecting some insights from the citation analysis, on backward and forward citations and how these citations help to move between the different pieces of research infrastructure. The publication tracking part mentioned in the manuscript's title should be further developed in the text, and the reader will be curious to learn more about this. Third, the authors do not provide a clear definition of funding portfolio and how portfolio considerations can help improve the impact of the funded research. Do they have in mind some type of ex ante portfolio model as in Canton, 2025 (Ref 1) with implications for project financing decisions (ex ante, so before starting the project) or do the authors mostly refer to an ex post approach, where the value is maximised of the ongoing set of projects within a portfolio? It seems that the authors have the latter approach in mind, but it might be interesting to also include some reflections in the article on financing decisions when portfolio considerations play a role. The authors seem to be interested in achieving societal value by promoting projects to move more quickly from low TRLs to higher ones, as covered by the NIHR, but they also refer to different impact types and it is not clear to me which mechanisms the authors want to emphasise, and how this relates to the citation analysis. More generally, and also referring to figure 2 in the article, the analysis leaves unanswered the question whether additional considerations, such as impact type, should matter for funding decisions, in addition to more traditional criteria such as academic excellence. It would be interesting to include some reflections on that in the article. In particular, can the impact of the funded projects be further increased by including portfolio considerations in the financing decisions of the projects, deviating from the more traditional merit-based approach? - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? Partly - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Partly - Are all the source data underlying the results available to ensure full reproducibility? Partly - Are the conclusions drawn adequately supported by the results? Partly

1. Canton E: A portfolio approach to research funding. Research Policy. 2025; 54 (1). Publisher Full TextCompeting Interests: No competing interests were disclosed. Reviewer Expertise: Economics of research and innovation CITE HOW TO CITE THIS REPORT Canton E. Reviewer Report For: Gaining insights into a funding portfolio through publication tracking [version 3; peer review: 3 approved, 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:62 (https://doi.org/10.3310/nihropenres.15250.r36489) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36489 https://openresearch.nihr.ac.uk/articles/5-62/v1#referee-response-36489 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. - Author Response 06 Oct 2025Reetika Suri-Ogilvie, NIHR Infrastructure, National Institute of Health and Care Research, London, UK06 Oct 2025Author ResponseWe thank our reviewers for their time taken to review our work and kind and helpful comments. We have fixed the typos in the manuscript. We have also included a ... Continue reading We thank our reviewers for their time taken to review our work and kind and helpful comments. We have fixed the typos in the manuscript. We have also included a bit more detail about the different types of NIHR infrastructure. However, it is a complex funded portfolio, and we did not want to include too much detail which deviates from the methodology and its use that we would like to focus on in this article. Further details of each piece of NIHR infrastructure can be found in the reference links. The Research Excellence Framework (REF) is an exercise that is conduced by Research England, which is a completely separate entity from the NIHR. Participation in the REF is at a university level whereas the majority of NIHR infrastructure is hosted by NHS Trusts which are partnered with UK universities. REF impact case studies are presented from a university’s perspective and represent only a small proportion of carefully selected and curated research being undertaken in any given university. This, combined with the fact that NIHR infrastructure enables and facilitates, through expertise, collaborations and networks, but may not directly or fully fund, research, makes it very difficult to define a linear relationship between NIHR infrastructure supported research and impact. We used citation analysis as a starting point to determine where NIHR infrastructure has played an enabling role in research, and qualitative methods to further understand the pathways to impact. Pathways to impact can be convoluted and messy and impacts can be unintended which also makes it difficult to comment on the relationship between research methodologies and efficiency of research and resulting impacts. We have a provided a bit more detail on our thinking around forward and backward citation analysis and how it has helped us to understand the movement of research between different parts of our funded portfolio. It would be helpful to understand what further insights might be of interest to our readers. The aim of this article is to present our methodologies and learning from using citation analysis to understand pathways to impact through the movement of research within our infrastructure portfolio. The NIHR spends public money on research, and therefore, showing return on investment and being accountable for this spend is crucial. Lessons learnt from this work have fed into thinking about gaps and opportunities for our infrastructure to enable impact at a systems level, rather than funding decisions on individual projects based on impact types. Additionally, NIHR infrastructure is funded to be responsive to policy needs and the needs of the health and care system, for which impact might look like a variety of different things that deviate from traditional impact types.We thank our reviewers for their time taken to review our work and kind and helpful comments.Competing Interests: No competing interests were disclosed. Close We have fixed the typos in the manuscript. We have also included a bit more detail about the different types of NIHR infrastructure. However, it is a complex funded portfolio, and we did not want to include too much detail which deviates from the methodology and its use that we would like to focus on in this article. Further details of each piece of NIHR infrastructure can be found in the reference links. The Research Excellence Framework (REF) is an exercise that is conduced by Research England, which is a completely separate entity from the NIHR. Participation in the REF is at a university level whereas the majority of NIHR infrastructure is hosted by NHS Trusts which are partnered with UK universities. REF impact case studies are presented from a university’s perspective and represent only a small proportion of carefully selected and curated research being undertaken in any given university. This, combined with the fact that NIHR infrastructure enables and facilitates, through expertise, collaborations and networks, but may not directly or fully fund, research, makes it very difficult to define a linear relationship between NIHR infrastructure supported research and impact. We used citation analysis as a starting point to determine where NIHR infrastructure has played an enabling role in research, and qualitative methods to further understand the pathways to impact. Pathways to impact can be convoluted and messy and impacts can be unintended which also makes it difficult to comment on the relationship between research methodologies and efficiency of research and resulting impacts. We have a provided a bit more detail on our thinking around forward and backward citation analysis and how it has helped us to understand the movement of research between different parts of our funded portfolio. It would be helpful to understand what further insights might be of interest to our readers. The aim of this article is to present our methodologies and learning from using citation analysis to understand pathways to impact through the movement of research within our infrastructure portfolio. The NIHR spends public money on research, and therefore, showing return on investment and being accountable for this spend is crucial. Lessons learnt from this work have fed into thinking about gaps and opportunities for our infrastructure to enable impact at a systems level, rather than funding decisions on individual projects based on impact types. Additionally, NIHR infrastructure is funded to be responsive to policy needs and the needs of the health and care system, for which impact might look like a variety of different things that deviate from traditional impact types. COMMENTS ON THIS REPORT - Author Response 06 Oct 2025Reetika Suri-Ogilvie, NIHR Infrastructure, National Institute of Health and Care Research, London, UK06 Oct 2025Author ResponseWe thank our reviewers for their time taken to review our work and kind and helpful comments. We have fixed the typos in the manuscript. We have also included a ... Continue reading We thank our reviewers for their time taken to review our work and kind and helpful comments. We have fixed the typos in the manuscript. We have also included a bit more detail about the different types of NIHR infrastructure. However, it is a complex funded portfolio, and we did not want to include too much detail which deviates from the methodology and its use that we would like to focus on in this article. Further details of each piece of NIHR infrastructure can be found in the reference links. The Research Excellence Framework (REF) is an exercise that is conduced by Research England, which is a completely separate entity from the NIHR. Participation in the REF is at a university level whereas the majority of NIHR infrastructure is hosted by NHS Trusts which are partnered with UK universities. REF impact case studies are presented from a university’s perspective and represent only a small proportion of carefully selected and curated research being undertaken in any given university. This, combined with the fact that NIHR infrastructure enables and facilitates, through expertise, collaborations and networks, but may not directly or fully fund, research, makes it very difficult to define a linear relationship between NIHR infrastructure supported research and impact. We used citation analysis as a starting point to determine where NIHR infrastructure has played an enabling role in research, and qualitative methods to further understand the pathways to impact. Pathways to impact can be convoluted and messy and impacts can be unintended which also makes it difficult to comment on the relationship between research methodologies and efficiency of research and resulting impacts. We have a provided a bit more detail on our thinking around forward and backward citation analysis and how it has helped us to understand the movement of research between different parts of our funded portfolio. It would be helpful to understand what further insights might be of interest to our readers. The aim of this article is to present our methodologies and learning from using citation analysis to understand pathways to impact through the movement of research within our infrastructure portfolio. The NIHR spends public money on research, and therefore, showing return on investment and being accountable for this spend is crucial. Lessons learnt from this work have fed into thinking about gaps and opportunities for our infrastructure to enable impact at a systems level, rather than funding decisions on individual projects based on impact types. Additionally, NIHR infrastructure is funded to be responsive to policy needs and the needs of the health and care system, for which impact might look like a variety of different things that deviate from traditional impact types.We thank our reviewers for their time taken to review our work and kind and helpful comments.Competing Interests: No competing interests were disclosed. Close We have fixed the typos in the manuscript. We have also included a bit more detail about the different types of NIHR infrastructure. However, it is a complex funded portfolio, and we did not want to include too much detail which deviates from the methodology and its use that we would like to focus on in this article. Further details of each piece of NIHR infrastructure can be found in the reference links. The Research Excellence Framework (REF) is an exercise that is conduced by Research England, which is a completely separate entity from the NIHR. Participation in the REF is at a university level whereas the majority of NIHR infrastructure is hosted by NHS Trusts which are partnered with UK universities. REF impact case studies are presented from a university’s perspective and represent only a small proportion of carefully selected and curated research being undertaken in any given university. This, combined with the fact that NIHR infrastructure enables and facilitates, through expertise, collaborations and networks, but may not directly or fully fund, research, makes it very difficult to define a linear relationship between NIHR infrastructure supported research and impact. We used citation analysis as a starting point to determine where NIHR infrastructure has played an enabling role in research, and qualitative methods to further understand the pathways to impact. Pathways to impact can be convoluted and messy and impacts can be unintended which also makes it difficult to comment on the relationship between research methodologies and efficiency of research and resulting impacts. We have a provided a bit more detail on our thinking around forward and backward citation analysis and how it has helped us to understand the movement of research between different parts of our funded portfolio. It would be helpful to understand what further insights might be of interest to our readers. The aim of this article is to present our methodologies and learning from using citation analysis to understand pathways to impact through the movement of research within our infrastructure portfolio. The NIHR spends public money on research, and therefore, showing return on investment and being accountable for this spend is crucial. Lessons learnt from this work have fed into thinking about gaps and opportunities for our infrastructure to enable impact at a systems level, rather than funding decisions on individual projects based on impact types. Additionally, NIHR infrastructure is funded to be responsive to policy needs and the needs of the health and care system, for which impact might look like a variety of different things that deviate from traditional impact types. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | ||||| |---|---|---|---|---|---| | 1 | 2 | 3 | 4 | 5 | | | Version 3 (revision) 01 Dec 25 | read | read | read | read | | | Version 2 (revision) 06 Oct 25 | read | read | ||| | Version 1 22 Jul 25 | read | read | Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...