Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs)

Co-production, patient involvement, harms, nocebo, placebo, research ethics, adverse events, participant information leaflet ALL Metrics - Views Downloads How to cite this article Jacob N, Howick J, Svobodova M et al. Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.13423.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Research Article [version 1; peer review: 1 approved, 2 approved with reservations] Nina Jacob1, Jeremy Howick1,2, Martina Svobodova https://orcid.org/0000-0001-7949-4039 1, [...] Shaun Treweek3, Katie Gillies3, Adrian Edwards4, Peter Bower5, Jennifer Bostock6, K Hood1Nina Jacob1, Jeremy Howick1,2, [...] Martina Svobodova https://orcid.org/0000-0001-7949-4039 1, Shaun Treweek3, Katie Gillies3, Adrian Edwards4, Peter Bower5, Jennifer Bostock6, K Hood1 PUBLISHED 21 Aug 2023 Author details Author details 1 Centre for Trials Research, Cardiff University, Cardiff, Wales, UK 2 Stoneygate Centre for Empathic Healthcare,, University of Leicester, Leicester, England, UK 3 Health Services Research Unit, University of Aberdeen, Aberdeen, Scotland, UK 4 Division of Population Medicine, Cardiff University, Cardiff, Wales, UK 5 Division of Population Health, Health Services Research & Primary Care, The University of Manchester, Manchester, England, UK 6 Lay Research Advisor, Centre for Trials Research, Cardiff University, Cardiff, Wales, UK 2 Stoneygate Centre for Empathic Healthcare,, University of Leicester, Leicester, England, UK 3 Health Services Research Unit, University of Aberdeen, Aberdeen, Scotland, UK 4 Division of Population Medicine, Cardiff University, Cardiff, Wales, UK 5 Division of Population Health, Health Services Research & Primary Care, The University of Manchester, Manchester, England, UK 6 Lay Research Advisor, Centre for Trials Research, Cardiff University, Cardiff, Wales, UK Nina Jacob Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Jeremy Howick Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Martina Svobodova Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Shaun Treweek Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Katie Gillies Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Adrian Edwards Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Peter Bower Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Jennifer Bostock Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing K Hood Roles: Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing OPEN PEER REVIEW REVIEWER STATUS The way information about potential benefits and harms of trial is presented within participant information leaflets (PILs) varies widely and may cause unnecessary ‘nocebo’ effects. The Medical Research Council (MRC) funded a project that developed seven principles to reduce this variation. However, guidance has not been produced to facilitate the implementation of the principles. Stakeholder involvement is recommended to optimise the way these principles are disseminated and explained. To co-produce recommendations for developing: (1) user-friendly guidance for users of the principles; and (2) resources that support the implementation of the principles. We held a co-production workshop with representation from the following professional groups: the Health Research Authority (HRA), research ethics committee members, and trial managers. Two rounds of discussions focused on generating recommendations for guidance and resources that support the implementation of the seven principles. Extensive low inference style ethnographic notes were taken, and the data were analysed thematically using deductive codes. The data was collected on October 14, 2022. 25 participants attended a hybrid workshop. Participants recommended that both researchers designing PILs and research ethics committee members should use the principles, and that that they should be simple, mention both benefits and harms explicitly, include examples of visual representations, and provide the evidence base for the principles. We were able to co-produce recommendations for developing and implementing the seven principles within PILs. These recommendations can now be implemented to reduce unexplained variation in the way potential benefits and harms are shared within PILs. Trial participants need to know about the potential benefits and harms of trial interventions to make an informed decision about whether to take part in a clinical trial. Yet the way they are told about these benefits and harms varies widely. We developed seven principles that can reduced this variability. We held a workshop with members of ethics committees and other stakeholders to develop the best way to implement the principles. 25 participants attended the workshop, and we were able to make useful recommendations that will improve the way trial participants are told about risks and benefits of trial treatments. Co-production, patient involvement, harms, nocebo, placebo, research ethics, adverse events, participant information leaflet Corresponding Author(s) Jeremy Howick ([email protected]) Grant information: This project is funded by the Medical Research Council (MRA reference MR/V020706/1); and the Health Technology Assessment Programme (15/40/05). Cardiff University is the sponsor for this research. Neither the funder nor the sponsor was involved in any other aspect of the project, such as the design of the project’s protocol and analysis plan, the collection, and analyses. The funder had no input on the interpretation or publication of the study results. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2023 Jacob N et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Jacob N, Howick J, Svobodova M et al. Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.13423.1) First published: 21 Aug 2023, 3:42 (https://doi.org/10.3310/nihropenres.13423.1) Latest published: 21 Aug 2023, 3:42 (https://doi.org/10.3310/nihropenres.13423.1) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Trial participants need to know about the potential benefits and harms of trial interventions to make an informed decision about whether to take part in a clinical trial1,2. However, research has shown that the way such information is shared within participant information leaflets (PILs) varies widely and is often unbalanced, with potential treatment benefits frequently not mentioned at all3. Additionally, overemphasising harms can induce unnecessary information-induced harm (‘nocebo’ effects)4, which in turn may adversely affect trial recruitment5. To help reduce this variation and unnecessary nocebo harms, the Medical Research Council funded a project called ‘Developing and Testing Participant Information Leaflets that Inform and Do Not Cause Harm (PrinciPIL)’6,7. This project included a Delphi process involving a range of stakeholders (participant representatives, ethics committee members, industry representatives, medico-legal experts, applied researchers, research nurses and trial managers) to identify principles that can be used to guide the way in which information about potential benefits and harms of trial interventions is shared within PILs. The principles are as follows: 1. All potential harms of the intervention should be listed. 2. The harms should be separated into serious (life threatening, causing permanent damage) and less serious (like a mild headache that goes away quickly). 3. The fact that not all potential harms are known needs to be explicit. 4. All potential benefits of the intervention should be listed. 5. The potential benefits and harms of a clinical trial need to be compared with what happens if the participant does not take part in the trial. 6. Suitable visual representations are recommended where appropriate to describe potential intervention benefits and harms, such as pictograms of faces. 7. Information about potential benefits and harms should be presented in proximity (for example, on the same page). PILs that are informed by the principles above are called PrinciPILs, and PrinciPILs are designed to have three main benefits. First, they will reduce the variability in the way information about the potential benefits and harms of trial treatments is described within PILs3. Secondly—and this is a consequence of the above—the principles have the potential to reduce research waste arising from different trial and research ethics committee (institutional review board) teams, developing their own ‘best’ way to share information about the potential benefits and harms of trial treatments. Third, they may improve recruitment rates and reduce nocebo effect harms. To test this, PrinciPILs are being compared with standard PILs in a series of studies within a trial (SWATs) to evaluate whether they influence recruitment rates and trial-related clinical outcomes (especially participant-reported harms, which may be nocebo effects)8. Given their likely benefits, it is important to implement the principles with user-friendly guidance that researchers who design PILs and ethics committees who evaluate them understand and can use. When developing guidance, stakeholder engagement is considered crucial for ensuring priority issues are considered and identified9. The importance of involving practitioners and other stakeholders is also considered key to ensuring guidance is adopted, implemented and maintained in the contexts for which it is intended10,11. In accordance with Health Research Authority (HRA) guidance, this study did not require ethical approval as this study was conducted with professionals to discuss their professional opinions. Attendees (all professionals) provided their verbal consent to publish the results of the workshop at the outset of the workshop. The sponsor of the study agreed that oral consent sufficed, given that no patients were involved in the workshop, no patients data was included, and because the workshop attendees were all acting in their professional capacities and providing their views related to their professional work. A patient and public involvement (PPI) representative (JB) was involved in acquiring the funding for this study, question development, research design, and background research. The same PPI representative is involved in our ongoing active dissemination plan for this study. Participants were chosen from the group of stakeholders (patient and public representatives, research ethics committee members, industry representatives, medico-legal experts, psychologists, and trial managers) who had registered their interest in designing, evaluating, or using PILs from a previous study7. We then used a purposeful sampling strategy to select participants, and contacted them via email. A purposeful strategy helped ensure that group members were drawn from a range of networks whilst also ensuring representation of stakeholders from key organisations who examine PILs (HRA and research ethics committee members) and those who design them (trial managers and other clinical trial representatives). A choice of attending virtually or in person was offered to encourage maximum participation, although most stakeholders attended in person. In advance of the face-to-face workshop held on October 14th, 2022, an email was sent to specify that the scope of the workshop was to discuss the guidance required alongside the principles and suggestions for required implementation resources. It was also specified that a discussion of the principles themselves was beyond the scope of the workshop, as they had already been developed with extensive stakeholder input12. Verbal consent to publish the results was obtained at the outset of the workshop. We followed the methodology used in a related project13 The research team presented an overview of the evidence underpinning the principles (JH). A brief general discussion of the PrinciPIL findings followed. Next, small groups (n=3) were chosen at random to discuss the two questions listed below. 1. How can we best present the principles so that they can be easily understood and implemented? 2. What resources (for example, web-based resources) are required to support implementation of the principles? Iterative rounds of feedback and discussion were conducted after each question, with opportunities to raise conflicting opinions provided. One researcher (NJ) was tasked to take detailed notes during the day. This included minuting of the reports from the small groups. Key points were reflected and summarised, and areas of widespread agreement and of disagreement were noted. As part of the co-production workshop, extensive low inference style ethnographic notes were taken and then expanded upon following the meeting. Data were analysed thematically using deductive codes that had been identified prior to the workshop, based on the two main questions asked during the workshop. Additional inductive codes relating more broadly to the seven core principles were added where required14. For the purpose of this study, the analysis concentrated on: (i) feedback on the two main questions asked of the group; (ii) the stakeholders’ broad observations on the utility of the seven core principles; and (iii) how the feedback can be incorporated into the main principles. We were able to meet our target number of attendees (n=25). The attendees came from the East Midlands, Southeast, London, Yorkshire and the Humber, and the Southwest. There were 19 female attendees (76%), and five (20%) were from non-White backgrounds. In discussing how best to present the principles so that they can be easily understood and implemented, five main themes arose. These focused on the need for (i) the guidance to be easily understood, (ii) clarification of the proper use of appendices, (iii) examples of appropriate visual representation, (iv) provision of the rationale for describing potential benefits, and (v) the need to highlight the evidence base underpinning the principles. Guidance should be useable by all researchers designing PILs ethics committee members. Whilst some welcomed a single guidance document understandable for both applicants and reviewers, others questioned whether a one-size-fits-all document would be appropriate. For research teams, this could come in the form of a standard operating procedure (SOP) or similar protocol. For research ethics committees, it was recommended that the PrinciPIL guidance should serve as a template for a conversation with research teams. The usual practice for research ethics committees is to accept the risks and benefits presented to them by study teams. It was suggested that the PrinciPIL guidance could thus serve as a change in practice by encouraging a conversation around what has been included and why. It was also suggested that a different approach might be needed for different trials. It was noted that the guidance had to take into consideration that PILs for different trials and trial populations needed to be different. For example, explaining risks and benefits to children is likely to be very different from explaining risks and benefits to adults. The proper use of appendices needs to be clarified. Appendices are sometimes used to avoid overly lengthy risks and benefits sections in certain trials. The extent to which these are accessed by participants was queried, although it was agreed that their use may sometimes be required to ensure readability. Examples of visual representation should be provided. The types of visual representation of risks that would be appropriate were not clear to the attendees. The attendees recommended that examples of visual representation be provided. One participant queried the use of smiley faces and suggested using pictograms that reference the text as an alternative approach. The rationale for including information about potential benefits should be clear. A great deal of discussion centred on whether the potential benefits of trial treatments should be mentioned at all within PILs. Some of the participants claimed that potential benefits are subjective and not known (whereas many harms were considered to be known). In addition, the group thought it would be useful to think about benefits as direct (from the intervention i.e., therapeutic) and indirect (wider participation in research i.e., non-therapeutic) and to be clear about what the benefits listed are. The evidence base underpinning the principles must be highlighted. Many workshop attendees had not done the suggested background reading and were unaware of the evidence underpinning the principles. Stakeholders therefore considered it useful to be reminded of the evidence base that underpins the seven principles3,4,12. The group was invited to make specific suggestions with regards to the guidance for the principles. Below is a summary of the main points of discussion and will be incorporated into the PrinciPIL guidance (see Table 1). | Category | Suggestion | How we are taking suggestion into account in the development of dissemination of the principles | |---|---|---| | Presenting principles | Guidance should be usable by all ethics committee members and researchers designing PILs | On our website we make it clear that the principles are to be used as a helpful guide and that the way and extent to which they are implemented will vary from trial to trial depending on several factors. | | The proper use of appendices needs to be clarified | We have taken this into account by making it clear in the PrinciPILs that (a) not all potential harms are known, and (where applicable) that (b) a complete list of known potential harms is contained in an appendix and (c) that additional information can be obtained from trial team. | | | Examples of visual representation should be provided | The way we present potential benefits and potential risks includes visual elements such as contrasting colour. On our website, we have also listed the types of visual representations and provided references. | | | The rationale for including information about potential benefits should be clear | We have written a paper citing regulations from the UK, US, and EU where it is clearly stated that mentioning potential benefits of a trial intervention (where they exist) is required15–17. | | | The evidence-base underpinning the principles must be highlighted | On our website we have included a description of our extensive background studies. | | | Specific feedback on principles | The source of potential risks should be made clear | We have included this in our guidance document. | | An individualised approach needs to be adopted when separating the harms into serious and less serious | On our website we make it clear that the principles are to be used as a helpful guide and that the way and extent to which they are implemented will vary from trial to trial depending on several factors. | | | The sources of potential benefits need to be made explicit. | On our website, we have added an explanation of where information about potential benefits is likely to be found. | | | The answer to the question of what the risks of not taking part in the study requires elaboration. | We recommend that, where applicable, the following standard statement be added to the PrinciPILs: ‘if you do not come into this trial, you will get standard care. To learn more about the risks and benefits of standard care please ask the trial team or your physician.’ | | | Resources for implementing principles | Using resources, training, webinar, and a SOP amendment for CTUs | We have developed several resources including a website, a video, and recommendations for amending SOPs. For all of these, take measures to discourage slavish adherence to principles that does not account for the need for adaptation to specific trials and trial populations. | | Keep it simple | We developed infographics for the principles that are the key features of our website. | 1. All potential harms should be listed: The group agreed with the importance of this principle, especially because ethics committee members are seldom qualified in the disciplines they review. For similar reasons, it was suggested that accompanying guidance should include information on appropriate sources of information for non-drug studies. For example, Cancer Research UK (CRUK) has specific support groups to ensure that a variety of different pathways of information are sourced. It was suggested that an appendix or link should be used for risks over a specified amount. 2. The harms should be separated into serious (life threatening, causing permanent damage) and less serious (like a mild headache that goes away quickly): Participants agreed with this principle and noted that an individualised approach needs to be taken during additional consent conversations. 3. The fact that not all potential harms are known needs to be clear: No suggestions were made regarding guidance related to this principle. 4. All potential benefits of the intervention should be listed: It was suggested that clear guidance was required to define what constitutes a benefit and where the evidence for the benefit came from. In addition, participants suggested that direct potential benefits (that might arise from the intervention) be delineated from indirect benefits (wider participation in research). 5. Potential benefits and harms of a clinical trial need to be compared with what happens if the participant does not take part in the trial: A suggestion was made to present this information such as ‘Consequences of taking part and the accompanying risks and benefits.’ 6. Suitable visual representations are recommended where appropriate to describe potential intervention benefits and harms, such as pictograms of faces: No additional suggestions were made regarding this principle. It was reiterated that examples would be useful. 7. Information about potential benefits and harms should not be presented apart by one or more pages: There was unanimous agreement that this was a good idea, and there were no suggestions for guidance related to this principle. When discussing how to support the implementation of the principles, several suggestions were made, including the use of resources such as webinars and SOPs. It was also recommended that we keep things simple, with one participant stating that a simple list of the seven principles may suffice. We have described these below. Use of resources/training/webinars/SOPs for clinical trials units. Participants noted that appropriate implementation of resources is relative to the target audience. For example, researchers need more detailed instructions, whereas ethics committees may require exemplars. When developing resources, it would be useful to test these out with different research ethics committees. Several attendees noted that the implementation resources should not encourage slavish and unthinking adherence to principles. It was suggested that this could be achieved with worked examples. Keep it simple. Participants agreed that a strength of the principles is their brevity. It was even suggested that a single-page explanation of the seven principles could suffice. We implemented all the suggestions made at the workshop to improve the way the principles are disseminated and explained (see Table 1). Whilst the focus of the workshop was guidance and implementation, several wider points were raised during the workshop. We agreed to make a note of these for future research, and they are described below: 1. Importance of the broader consent procedure: The PrinciPIL approach is focussed upon the written communication around consent practices. It was noted that oral communications surrounding consent are at least as important18. Relatedly, the trial participants’ understanding of likelihood and risk was likely to be variable, and an individualised approach needs to be taken during consent conversations. 2. Are all harms and benefits equal? It was thought that it would be beneficial to think beyond the potential benefits and harms of the intervention itself to more general considerations, including time taken, increased monitoring and jumping queues. The potential to consider benefits to organisations (such as research organisations that generate income from research and pharmacological companies) was also raised, although no consensus was achieved on this point. 3. The legality of mentioning potential benefits: A vocal minority of participants stated that European guidance claimed to recommend against mentioning potential trial treatment benefits within PILs. This was disputed in the workshop and was subsequently found to be false. The European Union clinical trials regulations state that mentioning benefits is required: ‘Information given to the subject or, where the subject is not able to give informed consent, his or her legally designated representative for the purposes of obtaining his or her informed consent shall: (a) enable the subject or his or her legally designated representative to understand: (i) the nature, objectives, benefits, implications, risks and inconveniences of the clinical trial.’15 3. Greater definition about what constitutes a risk:The way risks (of trial treatments) are defined varies widely, and there is no consensus around the best way to define or present them19. There was general agreement that natural frequencies should be used (whole numbers rather than percentages), and this needs to be made clear in the guidance. 4. Rationale for taking part in the trial. Some of the attendees considered it essential for the PIL to answer to the question ‘Why should I take part in this trial and what will happen if I don’t?’ at the beginning of the PIL. As this goes beyond the scope of our current project, we did not incorporate it into our plans and instead made note of it for future research. We were able to co-produce several recommendations for PIL guidance and resources to support the implementation of the seven principles that can harmonise the way in which information about potential intervention benefits and harms is shared within PILs. The suggestions included the following: ensuring that guidance was useable by both research ethics committees and researchers; highlighting the evidence base underpinning the principles; the need for clarity regarding the use of appendices; the need for clarity regarding appropriate visual representation; and the need for greater clarity regarding the discussion of potential benefits. Overall, the group considered the simplicity of the principles to be their strength and, as a result, that minimal additional resources were required to implement them. The need to reduce variability in the application of research ethics principles has been recognized20, and calls have been made to harmonize research ethics guidance6,21. Unfortunately, and with few exceptions22, guidance for sharing potential trial intervention benefits and harms is under-researched7. By providing suggestions that can reduce the variability in the way in which potential trial intervention benefits and harms can be shared within PILs, our study addresses the identified need to harmonize research ethics guidance. A limitation of our co-production workshop was that the attendees had not done the required background reading outlining evidence that underpinned the seven principles. This led to tangential discussions, especially about the need to mention potential benefits, which threatened to undermine the groups’ confidence in the value of the principles. Future research should evaluate the extent to which the guidance and resources we produce are used and able to reduce the variability in the way potential benefits and harms are described within PILs. In addition, our methodology can now be expanded. Whereas our principles were applicable to written materials, we can now develop parallel principles that can inform verbal discussions of the potential benefits and harms of trial treatments. Additionally, there is a need to overcome the mistaken belief that mentioning potential trial/intervention benefits within PILs is unlawful. Stakeholders were able to co-produce suggestions for guidance and resources for implementing the seven principles that will harmonise the way information about potential trial intervention benefits and harms are described within PILs. These principles have the potential to help trial teams and research ethics committees consider what risks and benefits should be included, reduce variation in the way these are communicated and reduce research waste arising from both trial teams and research ethics committees having to develop their own optimal way to share this information. The recommendations can now be implemented. The workshop records used in this study are restricted to ensure the anonymity of the participants due to the small sample size. Ethical approval and consent was obtained on the basis that data would remain anonymous. Preserving anonymity requires that additional data not be shared. All unrestricted data is available in the manuscript, any additional data sharing would de-anonymise the sample. To request access to the restricted data, please contact Professor Jeremy Howick ([email protected]). Faculty Opinions recommendedReferences - 1. Faden RR, Beauchamp TL, King NMP: A history and theory of informed consent. New York; Oxford: Oxford University Press, 1986. Reference Source - 2. Howick J: Unethical informed consent caused by overlooking poorly measured nocebo effects. J Med Ethics. 2021; 47(9): 590–594. PubMed Abstract | Publisher Full Text - 3. Kirby N, Shepherd V, Howick J, et al.: Nocebo effects and participant information leaflets: evaluating information provided on adverse effects in UK clinical trials. Trials. 2020; 21(1): 658. PubMed Abstract | Publisher Full Text | Free Full Text - 4. Howick J, Webster R, Kirby N, et al.: Rapid overview of systematic reviews of nocebo effects reported by patients taking placebos in clinical trials. Trials. 2018; 19(1): 674. PubMed Abstract | Publisher Full Text | Free Full Text - 5. Treweek S, Pitkethly M, Cook J, et al.: Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018; 2(2): MR000013. PubMed Abstract | Publisher Full Text | Free Full Text - 6. Centre for Trials Research: PrinciPILs Wales. Cardiff University, 2021; accessed August 19 2022. Reference Source - 7. Svobodova M, Jacob N, Hood K, et al.: Developing principles for sharing information about potential trial intervention benefits and harms with patients: report of a modified Delphi survey. Trials. 2022; 23(1): 863. PubMed Abstract | Publisher Full Text | Free Full Text - 8. Howick J, Svobodova M, Treweek S, et al.: Patient reported outcomes and recruitment rates following the introduction of principled patient information leaflets (PrinciPILs): Protocol for a meta-analysis [version 1; peer review: awaiting peer review]. NIHR Open Res. in progress, 2023; 3: 29. Publisher Full Text - 9. Petkovic J, Riddle A, Akl EA, et al.: Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation. Syst Rev. 2020; 9(1): 21. PubMed Abstract | Publisher Full Text | Free Full Text - 10. Carroll C: Qualitative evidence synthesis to improve implementation of clinical guidelines. BMJ. 2017; 356: j80. PubMed Abstract | Publisher Full Text - 11. Schunemann F, Meerpohl JJ, Schwingshackl L, et al.: [Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise]. Z Evid Fortbild Qual Gesundhwes. 2021; 163: 76–84. PubMed Abstract | Publisher Full Text - 12. Svobodova M, Hale R, Hood K, et al.: Developing Core Principles for Sharing Information about Potential Intervention Benefits and Harms in Patient Information Leaflets Using a Modified Delphi Survey. Open Science Framework. 2021. Publisher Full Text - 13. Bruhn H, Campbell M, Entwistle V, et al.: What, how, when and who of trial results summaries for trial participants: stakeholder-informed guidance from the RECAP project. BMJ Open. 2022; 12(3): e057019. PubMed Abstract | Publisher Full Text | Free Full Text - 14. Braun V, Clarke V: Using thematic analysis in psychology. Qual Res Psychol. 2006; 3(2): 77–101. Publisher Full Text - 15. EUR-Lex: Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC Text with EEA relevance. 2022; accessed 6 January 2023. Reference Source - 16. Legislation.gov.uk: The Medicines for Human Use (Clinical Trials) Regulations 2004. London: The National Archives; 2004; accessed 6 January 2023. Reference Source - 17. Office for Human Research Protections: 2018 Requirements (2018 Common Rule). Rockville MD: Department of Health and Human Services (HHS), 2018; accessed 6 January 2023. Reference Source - 18. Koyfman SA, Reddy CA, Hizlan S, et al.: Informed consent conversations and documents: A quantitative comparison. Cancer. 2016; 122(3): 464–9. PubMed Abstract | Publisher Full Text | Free Full Text - 19. Coyle M, Gillies K: A systematic review of risk communication in clinical trials: How does it influence decisions to participate and what are the best methods to improve understanding in a trial context? PLoS One. 2020; 15(11): e0242239. PubMed Abstract | Publisher Full Text | Free Full Text - 20. Goodyear-Smith F, Lobb B, Davies G, et al.: International variation in ethics committee requirements: comparisons across five Westernised nations. BMC Med Ethics. 2002; 3: E2. PubMed Abstract | Publisher Full Text | Free Full Text - 21. Independent Review of Research Bureaucracy Final Report. In: Department for Business EISaURaI, ed.: Gov.uk, 2022. Reference Source - 22. Schwartz LM, Woloshin S, Welch HG: Using a drug facts box to communicate drug benefits and harms: two randomized trials. Ann Intern Med. 2009; 150(8): 516–27. PubMed Abstract | Publisher Full Text Author details Author details 1 Centre for Trials Research, Cardiff University, Cardiff, Wales, UK 2 Stoneygate Centre for Empathic Healthcare,, University of Leicester, Leicester, England, UK 3 Health Services Research Unit, University of Aberdeen, Aberdeen, Scotland, UK 4 Division of Population Medicine, Cardiff University, Cardiff, Wales, UK 5 Division of Population Health, Health Services Research & Primary Care, The University of Manchester, Manchester, England, UK 6 Lay Research Advisor, Centre for Trials Research, Cardiff University, Cardiff, Wales, UK 2 Stoneygate Centre for Empathic Healthcare,, University of Leicester, Leicester, England, UK 3 Health Services Research Unit, University of Aberdeen, Aberdeen, Scotland, UK 4 Division of Population Medicine, Cardiff University, Cardiff, Wales, UK 5 Division of Population Health, Health Services Research & Primary Care, The University of Manchester, Manchester, England, UK 6 Lay Research Advisor, Centre for Trials Research, Cardiff University, Cardiff, Wales, UK Nina Jacob Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Jeremy Howick Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Martina Svobodova Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Shaun Treweek Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Katie Gillies Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Adrian Edwards Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Peter Bower Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Jennifer Bostock Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing K Hood Roles: Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Funding Acquisition, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Competing interests No competing interests were disclosed. Grant information This project is funded by the Medical Research Council (MRA reference MR/V020706/1); and the Health Technology Assessment Programme (15/40/05). Cardiff University is the sponsor for this research. Neither the funder nor the sponsor was involved in any other aspect of the project, such as the design of the project’s protocol and analysis plan, the collection, and analyses. The funder had no input on the interpretation or publication of the study results. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright © 2023 Jacob N et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article Jacob N, Howick J, Svobodova M et al. Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.13423.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 1 VERSION 1 PUBLISHED 21 Aug 2023 Views 0 How to cite this report: Nair G(. Reviewer Report For: Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.14557.r30610) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30610 https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30610 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 16 Sep 2024 Approved VIEWS 0 This is a very useful study and will be of relevance to those who develop PILs and for ethics committees who review them. The work is clearly presented. However, the literature cited appears to be restricted to ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close The work is clearly presented. However, the literature cited appears to be restricted to ... Continue reading This is a very useful study and will be of relevance to those who develop PILs and for ethics committees who review them. The work is clearly presented. However, the literature cited appears to be restricted to the developed world context. More and more frequently clinical trials are now multisite, spanning both developed and developing countries. To make the recommendations generalizable and globally relevant, perhaps views of stakeholders/experts from developing countries could have been included. Cultural beliefs and practices play a major role in adoption of or adherence to interventions under investigation in trials. This is especially. true for prevention studies/trials among healthy volunteers (e.g non adherence to study product among young women in HIV prevention trials in SSA Africa). The co-production workshop is an appropriate design to solicit feedback on the development of guidance and resources for implementing the 7 principles. However, more information is required on the characteristics of the workshop participants: Were community members or community advisory board members included? Noted also that participants were allowed to join online and it would have been feasible to allow participants from developing countries to join, making the feedback more broadly relevant. The justification of not making source data available to preserve anonymity is acceptable but the researchers could have considered including comments from participants made during the discussions to support the conclusions. For example, what were thoughts on what constitutes a benefit? Should "jumping the queue" or receiving a service at no cost such as contraceptive access or a screening procedure be considered a benefit. Or should a benefit be restricted to that of using/adhering to the experimental drug. Did any of the discussion focus on the access to resources/guidance tools in different settings and how likely busy ethics committee members who are often volunteers would seek out these resources. I agree that the strength is the simplicity of the principles and look forward to seeing the accompanying guidelines and resources. The work is clearly presented. However, the literature cited appears to be restricted to the developed world context. More and more frequently clinical trials are now multisite, spanning both developed and developing countries. To make the recommendations generalizable and globally relevant, perhaps views of stakeholders/experts from developing countries could have been included. Cultural beliefs and practices play a major role in adoption of or adherence to interventions under investigation in trials. This is especially. true for prevention studies/trials among healthy volunteers (e.g non adherence to study product among young women in HIV prevention trials in SSA Africa). The co-production workshop is an appropriate design to solicit feedback on the development of guidance and resources for implementing the 7 principles. However, more information is required on the characteristics of the workshop participants: Were community members or community advisory board members included? Noted also that participants were allowed to join online and it would have been feasible to allow participants from developing countries to join, making the feedback more broadly relevant. The justification of not making source data available to preserve anonymity is acceptable but the researchers could have considered including comments from participants made during the discussions to support the conclusions. For example, what were thoughts on what constitutes a benefit? Should "jumping the queue" or receiving a service at no cost such as contraceptive access or a screening procedure be considered a benefit. Or should a benefit be restricted to that of using/adhering to the experimental drug. Did any of the discussion focus on the access to resources/guidance tools in different settings and how likely busy ethics committee members who are often volunteers would seek out these resources. I agree that the strength is the simplicity of the principles and look forward to seeing the accompanying guidelines and resources. - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Yes - If applicable, is the statistical analysis and its interpretation appropriate? Yes - Are all the source data underlying the results available to ensure full reproducibility? No source data required - Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: HIV and TB treatment and prevention clinical trials. Research Ethics. I have a keen interest in the informed consent process and am currently a member of a research ethics committee. CITE HOW TO CITE THIS REPORT Nair G(. Reviewer Report For: Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.14557.r30610) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30610 https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30610 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Nicholls S. Reviewer Report For: Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.14557.r30404) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30404 https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30404 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 25 Oct 2023 Approved with Reservations VIEWS 0 Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) In the present manuscript the authors provide an overview of a workshop undertaken to inform the development of guidance documents for the implementation of the ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close In the present manuscript the authors provide an overview of a workshop undertaken to inform the development of guidance documents for the implementation of the ... Continue reading Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) In the present manuscript the authors provide an overview of a workshop undertaken to inform the development of guidance documents for the implementation of the PrinciPIL principles. Improving the consent process to ensure potential participants can make informed decisions is a key ethical marker of research and work to improve this process is important. I do, however, have some specific comments for consideration (listed under the section headings) as well as several larger comments for the authors to consider. In terms of the larger comments: While the paper discusses purposeful sampling, this seems to only reflect on role. Were other experiential or demographic characteristics used to inform sampling? Was there any representation, for example, from people with design experience or with expertise in knowledge translation/implementation or other skills that could provide input on best approaches to uptake of information? ANALYSIS What deductive codes were used? How do the final themes relate to the deductive coding vs indictive coding?

Given the noted fact that many had not done the background reading on the evidence base for the principles; to what extent can these views be considered informed or deliberated vs initial or reactionary? I am also not completely sure on the goal of the analysis, which again goes back to the relatively broad aim. In some parts the results seem to reflect a desire to share the range of issues participants deemed relevant, yet in other areas it is more focused, and in some cases quasi-quantitative in the way that there is reference to “unanimous agreement” or a single suggestion being made. Some of the proposals seem somewhat circular to the questions posed. For example, a key question posed was “How can we best present the principles so that they can be easily understood and implemented?” and a theme in response is to make them useable by REC members and researchers. This seems to go without saying doesn’t it? One wouldn’t make them unusable. It seems to me that the more important question is HOW can they be made useable to researchers and how can they be made useable to RECs. Put the other way, it would seem important to identify what might be barriers to their use? The theme of having a rationale for including benefits is interesting as it seems to in part be more about refining the principle itself (e.g. the specification as direct and indirect benefits) as opposed to a way to present the principle so it can be understood. This is consistent with the serious/less serious specification of harms and how that is incorporated into the principles themselves. In addition, there seems to be some overlap between the theme relating to the rationale for including benefits, and the specific comments about listing the potential benefits of an intervention (both for example, discuss direct and indirect benefits). Perhaps these sets of results could be merged rather than duplicated. I am not clear on the concerns about “slavish and unthinking adherence to principles.” Is that not what is exactly desired – that these principles are universally adopted? If not, would that not lead to the variation in content that is lamented in the previous review and the introduction? Indeed, the fact that they are principles and not actions seems to require consideration and thought about how to they are applied in practice. Or was it the case that the comment reflects concern for a focus solely on these principles to the detriment of considering other important information? Perhaps clarifying this or offering perspective would be helpful. There is also variation in the results reporting. For some comments the results are provided solely as description: e.g. a topic was raised. In other areas of the results there is commentary made on the issue (e.g. the legality of mentioning potential benefits). Some are highlighted but then explained as not relevant, so I wasn’t sure of the rationale for raising them in the manuscript results (as opposed to reporting them as issues raised and then addressing them in the discussion as to whether they are misinformed).

The discussion would benefit from greater integration with the literature. While I realise the focus here is on the suggestions around approaches that can be made to improve the uptake of the PrinciPILs principles there has, for example, been lots of work in the area of participant information leaflets and especially in relation to required content (e.g the US regulations stipulate core components: https://www.hhs.gov/ohrp/regulations-and-policy/guidance/faq/informed-consent/index.html ). This is built into regulations, so it would be useful to contrast the regulatory approach to the integration of content, and this principled approach. Similarly, reviewing the literature relating to the topics raised would be helpful – for example, the visual representation of risks. In the present manuscript the authors provide an overview of a workshop undertaken to inform the development of guidance documents for the implementation of the PrinciPIL principles. Improving the consent process to ensure potential participants can make informed decisions is a key ethical marker of research and work to improve this process is important. I do, however, have some specific comments for consideration (listed under the section headings) as well as several larger comments for the authors to consider. In terms of the larger comments: - The workshop is focused on developing user friendly guidance to improve the use and evaluation of PILs. The Principles themselves are framed in terms of impact on trials: Yet data is not presented as to whether PILs designed in accordance with the PrinciPILs actually achieve the proposed benefits: does the PrinciPIL designed PIL reduce nocebo effects, for example? Should one not have evidence that adherence to the 7 principles does indeed have benefits? Seeking to improve uptake or adherence, absent evidence of benefits (or lack of harms) seems pre-emptive and it is notable that this was not raised as a theme (even though evidence of benefits to be listed in the PIL was). - A second aspect I found challenging was that the term ‘user-friendly’ is vague. How does one define or evaluate the ‘user-friendliness’ of the guidance? Was this framed in a specific way for participants? It is relevant as it could relate to language, visuals, format, as well as accessibility. Perhaps the notion of user friendly could be expanded upon or greater specificity in the objectives offered. - Finally, can the authors expand more on their definition of co-production? The term is used throughout but the work presented seems more akin to a consultation, unless the individuals engaged are/were also involved in the actions taken (as detailed in the 3rd column of Table 1). If a definition could be provided, along with how the study and future work meets the definition, that would be helpful. While the paper discusses purposeful sampling, this seems to only reflect on role. Were other experiential or demographic characteristics used to inform sampling? Was there any representation, for example, from people with design experience or with expertise in knowledge translation/implementation or other skills that could provide input on best approaches to uptake of information? ANALYSIS What deductive codes were used? How do the final themes relate to the deductive coding vs indictive coding?

Given the noted fact that many had not done the background reading on the evidence base for the principles; to what extent can these views be considered informed or deliberated vs initial or reactionary? I am also not completely sure on the goal of the analysis, which again goes back to the relatively broad aim. In some parts the results seem to reflect a desire to share the range of issues participants deemed relevant, yet in other areas it is more focused, and in some cases quasi-quantitative in the way that there is reference to “unanimous agreement” or a single suggestion being made. Some of the proposals seem somewhat circular to the questions posed. For example, a key question posed was “How can we best present the principles so that they can be easily understood and implemented?” and a theme in response is to make them useable by REC members and researchers. This seems to go without saying doesn’t it? One wouldn’t make them unusable. It seems to me that the more important question is HOW can they be made useable to researchers and how can they be made useable to RECs. Put the other way, it would seem important to identify what might be barriers to their use? The theme of having a rationale for including benefits is interesting as it seems to in part be more about refining the principle itself (e.g. the specification as direct and indirect benefits) as opposed to a way to present the principle so it can be understood. This is consistent with the serious/less serious specification of harms and how that is incorporated into the principles themselves. In addition, there seems to be some overlap between the theme relating to the rationale for including benefits, and the specific comments about listing the potential benefits of an intervention (both for example, discuss direct and indirect benefits). Perhaps these sets of results could be merged rather than duplicated. I am not clear on the concerns about “slavish and unthinking adherence to principles.” Is that not what is exactly desired – that these principles are universally adopted? If not, would that not lead to the variation in content that is lamented in the previous review and the introduction? Indeed, the fact that they are principles and not actions seems to require consideration and thought about how to they are applied in practice. Or was it the case that the comment reflects concern for a focus solely on these principles to the detriment of considering other important information? Perhaps clarifying this or offering perspective would be helpful. There is also variation in the results reporting. For some comments the results are provided solely as description: e.g. a topic was raised. In other areas of the results there is commentary made on the issue (e.g. the legality of mentioning potential benefits). Some are highlighted but then explained as not relevant, so I wasn’t sure of the rationale for raising them in the manuscript results (as opposed to reporting them as issues raised and then addressing them in the discussion as to whether they are misinformed).

The discussion would benefit from greater integration with the literature. While I realise the focus here is on the suggestions around approaches that can be made to improve the uptake of the PrinciPILs principles there has, for example, been lots of work in the area of participant information leaflets and especially in relation to required content (e.g the US regulations stipulate core components: https://www.hhs.gov/ohrp/regulations-and-policy/guidance/faq/informed-consent/index.html ). This is built into regulations, so it would be useful to contrast the regulatory approach to the integration of content, and this principled approach. Similarly, reviewing the literature relating to the topics raised would be helpful – for example, the visual representation of risks. - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? No - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: Co-author Gillies is a co-applicant with me on a recent submitted grant proposal in another area related to clinical trial design. Reviewer Expertise: Research ethics, patient-oriented research CITE HOW TO CITE THIS REPORT Nicholls S. Reviewer Report For: Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.14557.r30404) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30404 https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30404 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Fleming M and Robinson B. Reviewer Report For: Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.14557.r30351) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30351 https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30351 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 15 Sep 2023 Melanie Fleming, University of Oxford, Oxford, England, UK Barbara Robinson, University of Oxford, Oxford, England, UK Approved with Reservations VIEWS 0 This qualitative study aimed to generate and collate suggestions for guidelines of use for a set of principles which were created to help standardise the way in which harms and benefits are presented in participant information leaflets (PILs). Stakeholders across ... Continue reading We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above. Close This qualitative study aimed to generate and collate suggestions for guidelines of use for a set of principles which were created to help standardise the way in which harms and benefits are presented in participant information leaflets (PILs). Stakeholders across different professional research settings were invited to a workshop to discuss recommendations of guidance based on the principles, and these recommendations were thematically presented. The paper is well written, and does a good job of highlighting the importance and need of having consistent and accurate guidelines that can be used by research and ethics teams to implement the principles laid out, and benefits from considering the viewpoints of a diverse range of stakeholders. I have several minor comments/suggestions for consideration: The paper is well written, and does a good job of highlighting the importance and need of having consistent and accurate guidelines that can be used by research and ethics teams to implement the principles laid out, and benefits from considering the viewpoints of a diverse range of stakeholders. I have several minor comments/suggestions for consideration: - Due to the workshop building on the previous work in which the principles were generated, the narrative within the introduction was a little confusing in terms of what this study was doing, and what had already been done (e.g. when mentioning the Delphi process, I initially thought that was in relation to the current study, rather than the principle generation). The introduction would benefit from some rewording to clarify what was past and what is the current work. - The introduction also presents the SWATS, although this is useful and relevant to the work, this information would be better placed in the discussion section of the paper as it seems like “next steps” rather than work leading up to this part of the project. - The methods note that there was a “purposeful sampling strategy” employed. It would be useful to include more detail as to which criteria was used for this sampling. It is currently unclear whether patient representatives were included or not, as the “patient and public involvement” subsection implies that they were not involved in this workshop, but the “recruitment of the co-production group” does mention patient and public representatives as part of the group of stakeholders from which participants were chosen for this study. - The authors note that many of the workshop attendees did not read the background reading, however there was no description of the background reading provided. Please could details of the suggested reading be included in the methods section. - Under the ‘wider issues’ section of the results, point 2 mentions benefits/harms beyond the intervention itself. However, this point is somewhat covered in the ‘specific feedback relating to core principles’ section already, and I recommend that this therefore be incorporated into this section (specifically points 2 and 4) rather than being separate. - The conclusion points to the guidance being applicable and useable by both researchers creating PILs and ethics review boards. However, given some of the feedback provided in the results (specifically on pg 4: “Whilst some welcomed a single guidance document understandable for both applicants and reviewers, others questioned whether a one-size-fits-all document would be appropriate.”), I am not wholly convinced by this conclusion, and further discussion of this point may be more apt within the paper. - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Yes - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? Partly - Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Research involving human participants, including clinical trials. Ethical oversight. Experience in patient and public involvement in research. CITE HOW TO CITE THIS REPORT Fleming M and Robinson B. Reviewer Report For: Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs) [version 1; peer review: 1 approved, 2 approved with reservations]. NIHR Open Res 2023, 3:42 (https://doi.org/10.3310/nihropenres.14557.r30351) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30351 https://openresearch.nihr.ac.uk/articles/3-42/v1#referee-response-30351 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | ||| |---|---|---|---| | 1 | 2 | 3 | | | Version 1 21 Aug 23 | read | read | read | Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...