{"paper_id":"3691989f-c749-48c5-8dad-5fe752f0a928","body_text":"1\n1 Uncertainty in serious illness: \n2 A national interdisciplinary consensus exercise to identify clinical research priorities\n3\n4\n5 Authors Simon N Etkind1,2*, Stephen Barclay1, Anna Spathis1,2, Sarah A Hopkins1, Ben Bowers1, Jonathan \n6 Koffman 3 \n7\n8 Affiliations\n9 1. Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge\n10 2. Cambridge University Hospitals NHS Foundation Trust\n11 3. Hull York Medical School, University of Hull\n12 * corresponding author\n13\n14\n15 Corresponding author contact details and address\n16 Simon Etkind\n17 Sde23@medschl.cam.ac.uk\n18 Primary Care Unit, Department of Public Health and Primary Care, East Forvie Building, Addenbrookes \n19 Biomedical Campus, CB2 0SR\n20\n21\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \nNOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.\n\n2\n22 Abstract: \n23 Background: Serious illness is characterised by uncertainty, particularly in older age groups. Uncertainty \n24 may be experienced by patients, family carers, and health professionals about a broad variety of issues. \n25 There are many evidence gaps regarding the experience and management of uncertainty. \n26 Aim: We aimed to identify priority research areas concerning uncertainty in serious illness, to ensure that \n27 future research better meets the needs of those affected by uncertainty and reduce research \n28 inefficiencies.\n29 Methods: Rapid prioritisation workshop comprising five focus groups to identify research areas, followed \n30 by a ranking exercise to prioritise them. Participants were healthcare professionals caring for those with \n31 serious illnesses including geriatrics, palliative care, intensive care; researchers; patient/carer \n32 representatives, and policymakers. Descriptive analysis of ranking data and qualitative framework \n33 analysis of focus group transcripts was undertaken. \n34 Results: Thirty-four participants took part; 67% female, mean age 47 (range 33 – 67). The highest priority \n35 was communication of uncertainty, ranked first by 15 participants (overall ranking score 1.59/3). \n36 Subsequent priorities were: 2) How to cope with uncertainty; 3) healthcare professional \n37 education/training; 4) Optimising clinical approaches to uncertainty; and 5) exploring in-depth \n38 experiences of uncertainty. Research related to optimally managing uncertainty was given higher priority \n39 than research focusing on experiences of uncertainty and its impact. \n40 Conclusions: These co-produced, clinically-focused research priorities map out key evidence gaps \n41 concerning uncertainty in serious illness. Managing uncertainty is the most pressing issue, and \n42 researchers should prioritise how to optimally manage uncertainty in order to reduce distress, unlock \n43 decision paralysis and improve illness and care experience. \n44\n45 Key words\n46 Uncertainty; Communication; Serious illness; Palliative care; Qualitative research\n47\n48\n49 Key points\n50  Uncertainty is ubiquitous and distressing in serious illness, and can paralyse decision making\n51  In this consensus exercise, stakeholders identified research priorities for uncertainty in serious \n52 illness\n53  Communication of uncertainty was the highest priority\n54  Participants prioritised research concerning  managing uncertainty above research to understand \n55 experiences of uncertainty\n56\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n3\n57 Background\n58 Uncertainty is ubiquitous in serious illnesses across all health settings, especially when living with long-\n59 term conditions and frailty.[1-5] Encompassing “known unknowns”, uncertainty is characterised as an \n60 inadequate understanding, a sense of incomplete, ambiguous or unreliable information, and conflicting \n61 alternatives.[6, 7] It is inherently a complex concept and situations of uncertainty often result from \n62 several inter-related factors.[8] \n63\n64 Irrespective of its origin, uncertainty matters because when suppressed and ignored, it can profoundly \n65 negatively impact patients and their family.[9, 10] Older adults may be particularly affected as they \n66 commonly experience complex and unpredictable illness, associated with irreducible uncertainties.  \n67 Uncertainty may precipitate extensive psychological and existential distress, potentially culminating in an \n68 experience of ‘Total Uncertainty’ which may threaten an individual’s sense-of-self.[11] \n69\n70 If uncertainty is not addressed it may impact patient safety, adverse events, healthcare interactions and \n71 relationships.[12, 13] One metric where this is recorded are complaints levelled at healthcare, \n72 particularly in hospital settings.[14] Uncertainty can also limit patient participation in decision-making, \n73 leading to ‘decision paralysis’,[15, 16]  which may contribute to sub-optimal care and has repercussions \n74 for the allocation of scarce health resources, including hospital admissions and longer inpatient stays.[17, \n75 18] \n76\n77 It is not just patients who are affected by uncertainty. Despite uncertainty in medicine dating back to \n78 Hippocrates, there exists a deeply rooted aversion to it in empirical medicine, where acknowledging \n79 uncertainty can have connotations of failure.[18, 19] If poorly tolerated by health professionals, \n80 uncertainty can adversely impact their confidence and competence, increasing the risk of moral injury, \n81 burnout and depression.[20-23] This was particularly evident during the Covid-19 pandemic.[24, 25]  \n82\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n4\n83 Uncertainty in illness is not necessarily distressing; it may be appraised negatively, neutrally, or in some \n84 individuals, illnesses, and care circumstances, positively; for example holding onto prognostic uncertainty \n85 can in some circumstances enable people to retain hope.[26-28]  The distress caused by uncertainty, and \n86 hence the block to decision making is therefore not inevitable; the negative impacts of uncertainty can \n87 be at least partially ameliorated if it is addressed and communicated sensitively.[29-31] \n88\n89 There are innumerable possible situations of uncertainty, each of which may have its own optimal \n90 approach. We still do not know how best to approach and address it in older patients living with long-\n91 term or life-threatening illnesses in a way that best supports the individual and those involved in their \n92 care;[32, 33] although previous work has explored how to support communication of uncertainty, [34] \n93 uncertainty management,[12, 32] and shared decision making.[35] We do know that there can be no \n94 one-size-fits-all approach to uncertainty, as it is experienced differently in different clinical contexts, by \n95 different individuals.[6, 36] Uncertainties in some contexts may be more distressing than others and may \n96 require different approaches.[37] Despite the importance of uncertainty for patients and clinicians alike \n97 there has, to date, been no attempt to prioritise the most pressing areas to focus applied research on to \n98 improve care. \n99\n100 We aimed to identify stakeholder priority research areas concerning uncertainty in serious illness, [38] to \n101 enable future research to more effectively meet the needs of those affected by uncertainty and to \n102 reduce research inefficiencies.[39, 40]  \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n5\n103 Methods\n104 Design\n105 National interdisciplinary one-day prioritisation workshop with a range of stakeholders: patient and carer \n106 representatives, clinicians, researchers, and policymakers. Reporting according to REPRISE guidance (see \n107 appendix).[41] \n108\n109 Patient and public involvement\n110 A Patient and public involvement group supported development of the study aims and methods; 3 public \n111 contributors participated in the workshop, contributing to focus groups and the ranking exercise, and \n112 commented on the findings.\n113\n114 Participants and participant identification\n115 Participants were clinicians, researchers, policymakers, people with lived experience of serious illness \n116 and their informal carers. Researchers and policymakers were eligible if they were interested in the area. \n117 Clinicians were from any profession or specialty with experience in providing care to people with serious \n118 illnesses. The workshop invitation was disseminated widely through clinical and research networks and \n119 social media, including: Applied Research Collaborative (ARC) East of England, UK uncertainty in serious \n120 illness specialist interest group, the UK-wide Community Nursing Research Forum, regional and national \n121 palliative care and gerontology contacts. Invitations were sent from December 2022, and registration \n122 was open until 27 th February 2023. The workshop was held on 28th February 2023. Workshop attendees \n123 were informed of the research component in advance. \n124\n125 Workshop process\n126 The workshop drew on existing approaches to prioritisation, incorporating idea generation, \n127 consolidation, and ranking stages.[42] Written informed consent and self-reported demographic data \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n6\n128 were collected from workshop participants at the start of the day, including age, gender, ethnicity, \n129 participants’ status as a researcher, clinician, policymaker, or patient representative, and details of their \n130 field where relevant. To encourage debate, the workshop began with presentations on the “state of the \n131 science” concerning uncertainty in serious illness. Models of uncertainty and evidence gaps from \n132 relevant literature reviews were outlined.[11, 32, 34, 43, 44] \n133 We then held focus groups with participants to explore their views on uncertainty and identify key areas \n134 for future research. This approach mirrored the idea generation stage of the nominal group technique \n135 and the first round of a Delphi process.[42, 45] The topic guide was developed by the research team and \n136 was informed by literature review: it focused on experiences of uncertainty, views on desired outcomes \n137 when addressing uncertainty, and ideas for research questions concerning uncertainty in serious illness. \n138 Focus groups were led by clinicians and researchers with expertise in facilitation. Conversations were \n139 audio recorded and a scribe within each group recorded the research questions identified by \n140 participants.\n141 Research areas from the focus groups were collated into a summary list during the day. In the final \n142 session, this summary list was presented to participants who were invited to anonymously rank the top \n143 three areas in order of priority using the online ranking tool “Slido” (© 2023 Cisco Systems, Inc.). Items \n144 were presented in random order. See appendix for full workshop programme and topic guide.\n145\n146 Data analysis\n147 Analysis of focus group lists: The lists generated by focus groups were reviewed by two researchers (SE & \n148 JK) during the day, taking into account both researchers’ existing knowledge of evidence gaps and \n149 previously expressed areas for future research from literature reviews.(9, 28) The researchers combined \n150 the lists produced by each focus group by removing duplicates and arranging similar questions under \n151 “umbrella terms” to produce a single summary list of priority research areas for use in the subsequent \n152 ranking exercise. \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n7\n153\n154 Analysis of ranking exercise data: Individual-level ranking scores were exported from Slido. We analysed \n155 the scores descriptively and reported the number of participants ranking each item within their top \n156 three. We calculated an average ranking score by allocating points to each item as follows: the item \n157 ranked first received three points; the item ranked second received two points; the item ranked third \n158 received one point; all other items received zero points. The average ranking score was calculated by \n159 adding the points for each item and dividing by the number of participants.  The maximum any item \n160 could score was three if every participant ranked it as their top priority and the minimum was zero if no \n161 participant ranked it in their top three. \n162\n163 Analysis of focus group transcripts: Following the workshop the focus group recordings were transcribed \n164 verbatim, anonymised and analysed using a framework approach.[46] This stage aimed to identify \n165 detailed research questions within the priority areas, as well as any additional areas discussed that were \n166 relevant for future research. We used the research priority areas identified during the workshop as a \n167 coding framework and one researcher (SE) coded text in the transcripts that described research \n168 questions or participants’ views about these areas. A second researcher (JK) independently reviewed one \n169 focus group transcript. Coding was reviewed, and where there were differences, these issues were \n170 reconsidered and debated by both researchers until consensus was achieved.[46] To avoid making \n171 unwarranted claims about patterns and regularities in the data, we examined and coded unusual or non-\n172 confirmatory views that did not fit easily into the original framework.[46] The framework was \n173 condensed, summarised and discussed with the wider research team to refine it. Anonymised excerpts \n174 from the transcripts are presented to illustrate themes and represent a range of views. \n175\n176 Ethical approval: This study received approval from the University of Cambridge Psychology Research \n177 Ethics Committee [Reference:PRE.2022.125]. \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n8\n178 Results\n179 Participant details\n180 Thirty three participants took part in the focus groups, and 34 in the ranking survey, of whom 30 \n181 provided demographic information. The average age was 47 years (range 33 – 67), and 67% were female. \n182 80% were of white ethnicity, 10% Asian, and 10% from mixed or multiple ethnic groups. 70% were \n183 clinicians, 43% researchers, 10% patient or carer representatives, and 7% policymakers (participants \n184 could state multiple roles). Of the clinician participants, twelve had a background in palliative care, three \n185 geriatrics, two nursing and one each of intensive care, general practice, psychology, and physiotherapy.\n186\n187 Item generation and ranking\n188 Five focus groups with six to seven participants in each were of 53 to 64 minutes’ duration. The groups \n189 generated 61 research questions, which we condensed to produce the 10 priority areas that were then \n190 ranked by participants (Table 1). Communication of uncertainty was the highest-ranked item, scored first \n191 by 15 participants. Participants ranked the next four priorities almost equally: coping with uncertainty; \n192 training health professionals; optimising clinical approaches to uncertainty; understanding in-depth \n193 experiences of uncertainty. The other areas received lower priority scores. \n194\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n9\n195 Table 1. 10 priority areas for future uncertainty research in serious illness (n = 34):\nNumber of participants \nranking item in each top \n3 positions (%)\nOverall \nRanking\nPriority\n#1 #2 #3\nPercentage \nof \nparticipants \nranking \nitems in top \nthree\nRanking \nscore*\n1 Communication of uncertainty 15 (44) 3 (9) 3 (9) 62 1.59\n2 How to cope with uncertainty 6 (18) 5 (15) 3 (9) 41 0.91\n3 Education/training of healthcare \nprofessionals\n2 (6) 7 (21) 9 (26) 53 0.85\n4 Optimising clinical approaches to \nuncertainty\n6 (18) 2 (6) 4 (12) 35\n0.76\n5\nUnderstanding patient/carer \nexperiences of uncertainty in \ndepth\n2 (6) 7 (26) 5 (15) 41\n0.74\n6\nVariation in experience/response \nto uncertainty between different \nindividuals, groups, professions\n1 (3) 4 (12) 1 (3) 18\n0.35\n7 Explore positive aspects of \nuncertainty\n1 (3) 1 (3) 4 (12) 18 0.26\n8 Impact of uncertainty on \nbereavement\n1 (3) 2 (6) 1 (3) 12 0.24\n9 Uncertainty in specific \nconditions/clinical situations\n0 (0) 3 (9) 1 (3) 12 0.21\n10 Factors associated with different \nuncertainty experiences\n0 (0) 0 (0) 1 (3) 9 0.09\n196 *Calculated as follows: item ranked first receives 3 points; item ranked 2nd receives 2 points; item ranked 3rd \n197 receives 1 point; all other items receive 0 points. The total points for each item are added and divided by the \n198 number of participants. \n199\n200 Detailed research priorities\n201 We explored the 10 priority areas raised by workshop participants during qualitative analysis, and \n202 identified detailed sub-questions within each area (Table 2).\n203\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n10\n204 The highest priority research areas were related to managing uncertainty, by communication, supporting \n205 individual coping mechanisms, or providing training. Participants particularly focused on how and when \n206 to have conversations about uncertainty, and how to individualise communication to promote \n207 psychological wellbeing:\n208 “Then there’s a really important research need around understanding more about how we \n209 communicate uncertainty and the consequences of how we communicate uncertainty and how \n210 we can talk about uncertainty in a way which is more patient-centred and supportive and \n211 considers sort of psychological wellbeing.” Social scientist, focus group 3. \n212 Training of health professionals at varying stages was a recognised priority. This included training \n213 professionals on toleration of their uncertainty as well as how to address the uncertainty experienced by \n214 others. One participant highlighted that inadequate training of doctors to manage uncertainty has been \n215 a longstanding issue and queried the best timing of such training:\n216 “We’ve been inadequate at teaching them [medical students] how to actually manage this \n217 complexity for a long time, although you could argue that it’s quite difficult to teach until they \n218 really get into the nitty-gritty of practising.” Geriatrician, focus group 1\n219 Participants recognised it is unlikely a single intervention can address all the nuanced multilevel aspects \n220 of uncertainty, but nevertheless, some felt interventions could play a role in addressing uncertainty, as \n221 long as they were situated in a broader societal context:\n222 “There are interventions that are being developed out there but there are no interventions that \n223 deal with all the different types and layers of uncertainty and they can’t by nature. And I think \n224 there is that kind of relational uncertainty, the kind of organisational uncertainties, there are \n225 uncertainties on macro, meso, micro levels and you can deal with one component, but you can’t \n226 deal with all the different components, and it’s how those interventions work within the broader \n227 societal context of uncertainty.” Occupational therapist, focus group 4\n228 Older frail patients were seen as a priority for future research into how uncertainty can be managed. \n229 Participants noted a lack of knowledge about approaching uncertainty in the context of frailty: \n230 “For old and frail elderly there’s no support you know, for that anxiety, fear, so it’s how do we \n231 look at that, you know, and how do we maybe look at how we can support, supporting that \n232 frailty and older.” Professor of end-of-life care, focus group 5\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n11\n233\n234 Participants identified research questions concerning experiences of uncertainty, though these were \n235 generally given lower priority than questions relating to managing uncertainty. Participants identified \n236 illness contexts where uncertainty experiences could be explored further, including critical illness and \n237 frailty. They also acknowledged it is important to explore how the experience of uncertainty varies \n238 depending on perspective (patient, carer or health professional), or individual characteristics:  \n239 “We’d want to look at different groups such as sort of learning disability, neurodiversity, sort of \n240 hard-to-reach areas whether deprivation, LGBTQ+, sort of that kind of differences you might get.” \n241 Palliative care consultant, focus group 1. \n242 Participants identified potential positive impacts of uncertainty and its utility in certain situations. They \n243 suggested it was important to understand why some health professionals thrive when required to \n244 manage uncertainty, whereas others exhibit a lower tolerance. Participants noted the relationship \n245 between uncertainty and hope and reasoned that positive aspects of uncertainty should be explored \n246 further:\n247 “I wonder….whether there’s something about learning to cope with uncertainty or to tolerate \n248 uncertainty and whether people, patients and families can see that as a positive as well.” \n249 Bereavement practitioner, focus group 4\n250\n251 Additional areas for future research  \n252 Some questions identified from focus group transcripts were not identified as research priority areas by \n253 participants. Additional areas included consideration of the legal and regulatory implications of \n254 uncertainty and its management, the link between uncertainty and patient safety, how to prepare the \n255 public for serious illness uncertainty, and the resource impacts of different levels of tolerance to \n256 uncertainty:\n257 “Have we studied the use of resources around uncertainty? Because I’m sure loads of tests are \n258 done completely unnecessarily because people just want to be sure.” Palliative care consultant, \n259 group 1\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n260 Table 2. Detailed research priorities from framework analysis*\nTheme Priority area Detail of priority area\nCommunication of uncertainty \n(research on how best to communicate \nabout uncertainty)\n Understand more about the detailed processes for uncertainty communication: who should be involved, \noptimum setting & timing, what phrases to use; how much uncertainty to share; how are such conversations \nreceived by patients \n How to communicate uncertainty openly whilst maintaining a trusting relationship? \n Interprofessional communication \n How to individualise communication of uncertainty to different settings, situations, individual characteristics?\n How does the communication of uncertainty affect decision-making and other outcomes?\nHow to cope with uncertainty\n(how can individuals best be supported to \ncope with their uncertainties)\n How do health professionals cope when they feel uncertain and how do they achieve resilience/tolerance to it? \n How can patients and carers be supported to cope with their uncertainty and develop resilience? \n Who copes well and why?\n How to cope with the long-term impact of decisions made under conditions of uncertainty? \n What is the role of hope in coping with uncertainty?\nEducation/training of healthcare \nprofessionals (how can we train health \nprofessionals to approach uncertainty)\n How to teach uncertainty management to medical students, what would be the goals of such training?\n How to train HCPs at different levels to recognise/tolerate/hold their own uncertainty? \n How to transfer expertise from areas where uncertainty is well managed?\n Development of training interventions including psychological training\nUncertainty \nmanagemen\nt\nOptimising clinical approaches to \nuncertainty\n(how can the uncertainty of others best be \nmanaged and addressed, what are optimal \napproaches)\n How to individualise the management of uncertainty depending on patient experiences, response to \nuncertainty, and how to identify how much uncertainty is tolerable to an individual?\n Who should 'hold' uncertainty and how to find a balance in terms of information sharing and decision-making\n How to maintain trust and manage expectations?\n How to gauge the readiness of people to discuss uncertainty and time conversations?\n How can technology be used to distil information and reduce/address uncertainty relating to complexity/ \ninformation overload?\n What is the role of uncertainty management interventions and models of care to address uncertainty, and how \neffective are these?\n What are the key outcomes we should be aiming for when seeking to approach and manage uncertainty? \n What are the barriers to addressing uncertainty?\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n How to manage uncertainty as an MDT and maintain continuity of approach\nUnderstanding patient/carer experiences \nof uncertainty in depth\n(what are patient and carer experiences of \nuncertainty)\n How does the experience of uncertainty affect decision-making?\n What is the lived experience of uncertainty amongst patients and families (what is helpful when faced with \nuncertainty and what is harmful? How do the multiple layers of uncertainty interact within an individual’s \nexperience (mapping uncertainty)? Are the experiences/impacts of different types of uncertainty different? \nHow do past life or healthcare experiences affect current uncertainty experiences.)? \n How is uncertainty transmissible i.e. how can HCP uncertainty be picked up by patients and vice versa and what \nis the effect of this?\nVariation in experience/ response to \nuncertainty \n(how does the experience of uncertainty vary \nbetween different groups)\n How does experience and response to uncertainty vary between different medical specialties and health \nprofessional groups?\n How does experience and response to uncertainty change over time as the illness progresses.\n How do individual characteristics e.g. culture, LGBTQ status, neurodiversity, age, affect experience and \nresponse to uncertainty?\n How does career stage and knowledge/experience in a clinical role affect health professionals experience?\n What are the long-term effects of uncertainty experiences in ITU on ITU survivors?\nExplore positive aspects of uncertainty\n(what aspects of uncertainty are positive and \nhow can these be promoted)\n What is the relationship between sharing uncertainty and maintaining hope?\n Explore positive utility of uncertainty e.g. as a way to promote a quest for knowledge or changing ways of \nthinking\n Explore why some people thrive with uncertainty\nImpact of uncertainty on bereavement\n(how does uncertainty in serious illness \nimpact on bereavement experiences)\n How does uncertainty in serious illness and at the end of life affect experiences and outcomes of \nbereavement?\nUncertainty \nexperiences\nUncertainty in specific conditions/clinical \nsituations\n(the uncertainties that are experienced in a \nparticular clinical situation such as in ITU)\n How do different clinical situations change how uncertainty is experienced and responded to?  \n Are there situations where the expression of uncertainty is not appropriate?\n How to make good decisions in the context of uncertainty in the intensive care unit?\n How is uncertainty experienced in children with complex neuro-disability? \n How to approach uncertainty in adults with neurodegenerative disease and uncertain illness trajectory?\n How to approach uncertainty in frailty with uncertain prognosis?\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n How to communicate uncertainty at transitions of care\nFactors associated with different \nuncertainty experiences \n(what factors are associated with different \nexperiences of or responses to uncertainty.)\n What system factors underlie the challenges of uncertainty?\n What factors are associated with uncertainty experiences and tolerance e.g. age, life experience, culture?\n How did the COVID-19 pandemic affect experiences and response to uncertainty?\n How does trust in clinicians affect uncertainty experiences?\n261 *Priority areas are listed in the order they were ranked\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n262 Discussion\n263 Summary of findings\n264 This study co-produced clinically-focused research priorities to address known evidence gaps \n265 concerning uncertainty in serious illness. Optimising communication of uncertainty was the top \n266 research priority. Research into managing uncertainty was considered higher priority than research \n267 investigating experiences of uncertainty.\n268\n269 Discussion of main findings\n270 Communicating uncertainty was the top priority for participants, reflecting key evidence gaps and \n271 recommendations in this field.[32, 47] In their narrative review of uncertainty communication, \n272 Simpkin et al identified a number of evidence gaps, including identifying individuals' communication \n273 preferences and tailoring communication to those preferences.[34] The question of how to maintain \n274 hope whilst communicating uncertainty was noted as a priority; this has been explored in cancer \n275 care,[48] but remains a key question in other serious illnesses. Additionally, participants raised \n276 several sub-questions in terms of how to discuss uncertainty, reflecting the need for \n277 implementation-focused communication research. \n278 After communication, participants prioritised other aspects of managing uncertainty: identifying \n279 how individuals can be supported to cope with their own uncertainty; investigating how we can \n280 optimise clinical approaches to uncertainty; understanding more about how to equip health \n281 professionals to deal with uncertainty through training. Though management of uncertainty has \n282 been recognised as a core component of medical training for decades, curricula still make limited \n283 reference to uncertainty, and filling this gap should be a priority.(2) Whilst we have an improving \n284 understanding of how physicians manage uncertainty,[31] the evidence base for other professional \n285 groups is very limited, yet nurses and allied healthcare professionals often lead the clinical care and \n286 support of older people and their families. To date, uncertainty management and communication \n287 interventions have had variable impact in serious illness,[32] and often prove  challenging to \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n288 evaluate.[33, 49] There is scope for further intervention development,[50] especially work that \n289 focuses on evaluation of uncertainty-focused interventions. \n290 Whilst many of the questions identified by participants related to investigating experiences of \n291 uncertainty, these were usually considered lower priority, perhaps because much is already known \n292 about uncertainty experiences.[7, 51] For example, conceptual taxonomies of uncertainty are well \n293 developed,(6) and there have been evidence syntheses of experience in some specific areas e.g. \n294 multimorbidity.[11] However, there are still evidence gaps concerning how uncertainty affects \n295 individuals in other clinical contexts, in particular older patients living with frailty. Given the rapidly \n296 increasing complexity of the healthcare system and unpredictability of the frailty trajectory, this is an \n297 area that warrants urgent exploration.[34] Understanding more about the impacts of uncertainty on \n298 experiences of illness, care, and bereavement would enable us to develop interventions focused on \n299 the real-world problems uncertainty can cause.\n300\n301 Strengths and limitations\n302 The rapid prioritisation approach we used enabled a diverse group of interested individuals to \n303 generate and rank research priorities in a single day. Those ranking the priorities had spent the \n304 entire day considering uncertainty in serious illness and were well placed to express considered \n305 views when ranking the list presented to them. By identifying evidence gaps before the workshop \n306 and communicating these to participants during initial presentations, we were able to focus on areas \n307 where more research is needed and incorporate the key stages of a traditional prioritisation \n308 exercise. By additionally incorporating formal qualitative analysis we increased rigour and developed \n309 a robust priority list. This approach was a more feasible and pragmatic alternative to lengthier \n310 methodologies such as the Delphi process[52, 53]. However, the rapid nature of the prioritisation \n311 process meant there was limited time to condense the findings of the focus group discussions, which \n312 risked a loss of accuracy and ranking was limited to broad research areas. We ameliorated this by \n313 subsequent analysis of focus group transcripts, which enabled us to identify detailed research \n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n314 questions identified by participants, including areas that participants mentioned even when they \n315 weren’t identified as priorities at the time.\n316 We incorporated patient and carer experiences, but most participants were healthcare professionals \n317 or researchers, thus these groups were not equally represented in the ranking process. Whilst not \n318 specifically excluded, social care professionals did not attend this workshop, which limits the findings \n319 to healthcare. We recruited a UK-wide sample, but this was not an international study, and future \n320 work should explore if these findings hold internationally, though literature from 17 countries \n321 reported consistent findings on a similar topic.[11] The anonymous nature of the ranking means we \n322 could not adjust for the background of participants when analysing ranking data. The largest group \n323 of clinical participants were from palliative care backgrounds which may have shaped their views; \n324 however, a broad range of health professionals and researchers were represented, and the degree \n325 of agreement, particularly regarding the top priority of communication suggests the findings \n326 represent true consensus. \n327\n328\n329 Conclusion\n330 Through a rapid prioritisation workshop, we have identified 10 ranked priority areas for clinically \n331 focused research on uncertainty in serious illness. There was consensus that further research into \n332 managing uncertainty, particularly communication, was of higher priority than research to \n333 investigate experiences of uncertainty. Future targeted research could result in interventions to \n334 reduce the distress associated with uncertainty, unlock decision paralysis and improve illness and \n335 care experience.\n336\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n337 Acknowledgements\n338 Thank you to those who facilitated focus groups: Stephan Barclay, Jonathan Koffman, Anna Spathis, \n339 Ben Bowers, Sarah Hopkins, Debbie Critoph, Markus Schichtel, Ikumi Okamoto\n340 We would also like to thank our patient and public involvement representatives: Roberta Lovick, \n341 Sarah Dixon, Rashmi Kumar\n342 Many thanks to the workshop organiser Angela Harper & the professional services team at the \n343 Primary Care Unit, University of Cambridge for ensuring the smooth running of this study\n344 Thank you to Zoe Fritz for your advice during drafting of the manuscript\n345\n346 Funding:\n347 This study and SB are supported by the National Institute for Health and Care Research (NIHR) \n348 Applied Research Collaboration East of England (NIHR ARC EoE) at Cambridgeshire and Peterborough \n349 NHS Foundation Trust. BB is supported by the Wellcome Trust [225577/Z/22/Z]. SAH is jointly \n350 funded by The Dunhill Medical Trust and British Geriatrics Society [Grant ref. JBGS20\\5].\n351 The views expressed are those of the author(s) and not necessarily those of the NIHR or the \n352 Department of Health and Social Care.’\n353\n354 Contributions:\n355 Study design: SE, SB, AS, JK\n356 Securing funding: SE, SB\n357 Data collection: SE, SB, AS, SAH, BB, JK\n358 Analysis: SE JK\n359 Paper drafting: SE, SB, AS, SAH, BB, JK\n360 Approval of final version: SE, SB, AS, SAH, BB, JK\n361\n362 Conflicts of Interest: \n363 The authors declare that they have no conflicts of interest.\n364\n365\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n366 References\n367 1. Fox RC. The evolution of medical uncertainty. Milbank Memorial Fund Quarterly, Health and \n368 Society. 1980;58(1):1-49.\n369 2. Etkind SN, Koffman J. Approaches to managing uncertainty in people with life-limiting \n370 conditions: role of communication and palliative care. Postgraduate Medical Journal. 2016.\n371 3. Etkind SN, Bristowe K, Bailey K, Selman LE, Murtagh FEM. How does uncertainty shape \n372 patient experience in advanced illness? A secondary analysis of qualitative data. Palliative Medicine. \n373 2016;31(2):171-80.\n374 4. Cox CL, Miller BM, Kuhn I, Fritz Z. Diagnostic uncertainty in primary care: what is known \n375 about its communication, and what are the associated ethical issues? Family Practice. \n376 2021;38(5):654-68. doi: 10.1093/fampra/cmab023.\n377 5. Higginson IJ, Rumble C, Shipman C, Koffman J, Sleeman KE, Morgan M, et al. The value of \n378 uncertainty in critical illness? An ethnographic study of patterns and conflicts in care and decision-\n379 making trajectories. BMC Anesthesiol. 2016;16:11. Epub 2016/02/11. doi: 10.1186/s12871-016-\n380 0177-2. PubMed PMID: 26860461; PubMed Central PMCID: PMCPMC4746769.\n381 6. Han PK, Klein WM, Arora NK. Varieties of Uncertainty in Health Care A Conceptual \n382 Taxonomy. Medical Decision Making. 2011;31(6):828-38.\n383 7. Etkind SN. Uncertainty in multimorbidity: a shared experience we should recognise, \n384 acknowledge and communicate. British journal of community nursing. 2022;27(11):540-4. doi: \n385 10.12968/bjcn.2022.27.11.540.\n386 8. Lipshitz R, Strauss O. Coping with uncertainty: A naturalistic decision-making analysis. Organ \n387 Behav Hum Decis Process 1997;69:149–63.\n388 9. Andersen HE, Hoeck B, Nielsen DS, Ryg J, Delmar C. A phenomenological-hermeneutic study \n389 exploring caring responsibility for a chronically ill, older parent with frailty. Nursing Open. \n390 2020;7(4):951-60. PubMed PMID: rayyan-114136678.\n391 10. Nanton V, Munday D, Dale J, Mason B, Kendall M, Murray S. The threatened self: \n392 Considerations of time, place, and uncertainty in advanced illness. British Journal of Health \n393 Psychology. 2015:n/a-n/a. doi: 10.1111/bjhp.12172.\n394 11. Etkind SN, Li J, Louca J, Hopkins SA, Kuhn I, Spathis A, et al. Total Uncertainty: A systematic \n395 review and thematic synthesis of experiences of uncertainty in older people with advanced \n396 multimorbidity, their informal carers, and health professionals. . Age and Ageing 2022;51(8).\n397 12. Carey I, Shouls S, Bristowe K, Morris M, Briant L, Robinson C, et al. Improving care for \n398 patients whose recovery is uncertain. The AMBER care bundle: design and implementation. BMJ \n399 supportive & palliative care. 2015;5(4):405-11.\n400 13. Yardley I, Yardley S, Williams H, Carson-Stevens A, Donaldson LJ. Patient safety in palliative \n401 care: a mixed-methods study of reports to a national database of serious incidents. Palliative \n402 medicine. 2018;32(8):1353-62.\n403 14. Ombudsman PaHS. Dying without dignity Investigations by the Parliamentary  and Health \n404 Service Ombudsman into complaints about end of life care. London Parliamentary and Health \n405 Service Ombudsman, 2015.\n406 15. Sellars M, Chung O, Nolte L, Tong A, Pond D, Fetherstonhaugh D, et al. Perspectives of \n407 people with dementia and carers on advance care planning and end-of-life care: A systematic review \n408 and thematic synthesis of qualitative studies. Palliative medicine. 2019;33(3):274-90.\n409 16. Oksavik; JD, Solbjør; M, Kirchhoff; R, Sogstad MKR. Games with uncertainty: the participation \n410 of older patients with multimorbidity in care planning meetings – a qualitative study. BMC Geriatrics. \n411 2021;21:242.\n412 17. Bristowe K, Carey I, Hopper A, Shouls S, Prentice W, Caulkin R, et al. Patient and carer \n413 experiences of clinical uncertainty and deterioration, in the face of limited reversibility: A \n414 comparative observational study of the AMBER care bundle. Palliat Med. 2015;29(9):797-807. Epub \n415 2015/04/02. doi: 10.1177/0269216315578990. PubMed PMID: 25829443; PubMed Central PMCID: \n416 PMCPMC4572938.\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n417 18. Simpkin AL, Schwartzstein RM. Tolerating Uncertainty - The Next Medical Revolution? The \n418 New England journal of medicine. 2016;375(18):1713-5. Epub 2016/11/03. doi: \n419 10.1056/NEJMp1606402. PubMed PMID: 27806221.\n420 19. Gheihman G, Johnson M, Simpkin AL. Twelve tips for thriving in the face of clinical \n421 uncertainty. Medical Teacher. 2020;42(5):493-9. doi: 10.1080/0142159X.2019.1579308.\n422 20. Čartolovni A, Stolt M, Scott PA, Suhonen R. Moral injury in healthcare professionals: A \n423 scoping review and discussion. Nursing ethics. 2021;28(5):590-602.\n424 21. Di Trani M, Mariani R, Ferri R, De Berardinis D, Frigo MG. From resilience to burnout in \n425 healthcare workers during the COVID-19 emergency: the role of the ability to tolerate uncertainty. \n426 Frontiers in Psychology. 2021;12:646435.\n427 22. van Iersel MB, Brantjes E, de Visser M, Looman N, Bazelmans E, van Asselt D. Tolerance of \n428 clinical uncertainty by geriatric residents: a qualitative study. European Geriatric Medicine. \n429 2019;10(3):517-22.\n430 23. Begin AS, Hidrue M, Lehrhoff S, Del Carmen MG, Armstrong K, Wasfy JH. Factors Associated \n431 with Physician Tolerance of Uncertainty: an Observational Study. Journal of general internal \n432 medicine. 2022;37(6):1415-21. Epub 2021/04/28. doi: 10.1007/s11606-021-06776-8. PubMed PMID: \n433 33904030; PubMed Central PMCID: PMCPMC8074695.\n434 24. Koffman J, Etkind SN, Gross J, Selman L. Uncertainty and Covid-19: How are we to respond? \n435 Journal of the Royal Society of Medicine. 2020:1–6 doi: DOI: 10.1177/0141076820930665.\n436 25. Davey Smith G, Blastland M, Munafò M. Covid-19’s known unknowns. BMJ (Clinical research \n437 ed). 2020;371:m3979. doi: 10.1136/bmj.m3979.\n438 26. Mishel MH. Reconceptualization of the Uncertainty in Illness Theory. Image: the Journal of \n439 Nursing Scholarship. 1990;22(4):256-62. doi: 10.1111/j.1547-5069.1990.tb00225.x.\n440 27. Gough N, Ross JR, Riley J, Judson I, Koffman J. 'When something is this rare … how do you \n441 know bad really is bad…?'-views on prognostic discussions from patients with advanced soft tissue \n442 sarcoma. BMJ Support Palliat Care. 2019;9(1):100-7. Epub 2015/12/31. doi: 10.1136/bmjspcare-\n443 2015-000898. PubMed PMID: 26715566.\n444 28. Brashers DE. Communication and Uncertainty Management. Journal of Communication. \n445 2001;51(3):477-97. doi: 10.1111/j.1460-2466.2001.tb02892.x.\n446 29. Krawczyk M, Gallagher R. Communicating prognostic uncertainty in potential end-of-life \n447 contexts: experiences of family members. BMC palliative care. 2016;15(1):59. doi: 10.1186/s12904-\n448 016-0133-4.\n449 30. Meranius MS, Engstrom G. Experience of self-management of medications among older \n450 people with multimorbidity. Journal of Clinical Nursing. 2015;24(19):2757-64. PubMed PMID: \n451 rayyan-114139203.\n452 31. Han PK, Strout TD, Gutheil C, Germann C, King B, Ofstad E, et al. How physicians manage \n453 medical uncertainty: a qualitative study and conceptual taxonomy. Medical decision making. \n454 2021;41(3):275-91.\n455 32. Ellis-Smith C, Dawkins M, Gao W, Higginson IJ, Evans CJ. Managing clinical uncertainty in \n456 older people towards the end of life: a systematic review of person-centred tools. BMC palliative \n457 care. 2021;20(1):1-41.\n458 33. Koffman J, Yorganci E, Yi D, Gao W, Murtagh F, Pickles A, et al. Managing uncertain recovery \n459 for patients nearing the end of life in hospital: a mixed-methods feasibility cluster randomised \n460 controlled trial of the AMBER care bundle. Trials. 2019;20(1):1-18.\n461 34. Simpkin AL, Armstrong KA. Communicating Uncertainty: a Narrative Review and Framework \n462 for Future Research. Journal of general internal medicine. 2019;34(11):2586-91. doi: \n463 10.1007/s11606-019-04860-8.\n464 35. Politi MC, Clark MA, Ombao H, Dizon D, Elwyn G. Communicating uncertainty can lead to less \n465 decision satisfaction: a necessary cost of involving patients in shared decision making? Health \n466 Expectations. 2011;14(1):84-91. doi: 10.1111/j.1369-7625.2010.00626.x.\n467 36. Hill DL, Walter JK, Szymczak JE, DiDomenico C, Parikh S, Feudtner C. Seven Types of \n468 Uncertainty When Clinicians Care for Pediatric Patients With Advanced Cancer. J Pain Symptom \n469 Manage. 2020;59(1):86-94. Epub 2019/08/20. doi: 10.1016/j.jpainsymman.2019.08.010. PubMed \n470 PMID: 31425822; PubMed Central PMCID: PMCPMC6942218.\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint \n\n471 37. Ghosh AK, Joshi S. Tools to manage medical uncertainty. Diabetes & Metabolic Syndrome: \n472 Clinical Research & Reviews. 2020;14(5):1529-33. doi: https://doi.org/10.1016/j.dsx.2020.07.055.\n473 38. Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? \n474 Descriptions and Recommendations for Future Consensus Research. Academic Medicine. \n475 2016;91(5).\n476 39. Sibbald SL, Singer PA, Upshur R, Martin DK. Priority setting: what constitutes success? A \n477 conceptual framework for successful priority setting. BMC Health Services Research. 2009;9(1):43. \n478 doi: 10.1186/1472-6963-9-43.\n479 40. El-Harakeh A, Lotfi T, Ahmad A, Morsi RZ, Fadlallah R, Bou-Karroum L, et al. The \n480 implementation of prioritization exercises in the development and update of health practice \n481 guidelines: A scoping review. PLoS ONE. 2020;15(3):e0229249. doi: 10.1371/journal.pone.0229249.\n482 41. Tong A, Synnot A, Crowe S, Hill S, Matus A, Scholes-Robertson N, et al. Reporting guideline \n483 for priority setting of health research (REPRISE). BMC Medical Research Methodology. \n484 2019;19(1):243. doi: 10.1186/s12874-019-0889-3.\n485 42. James Lind Alliance Guidebook 2021 [cited 2023 25th April]. Available from: \n486 https://www.jla.nihr.ac.uk/jla-guidebook/.\n487 43. Lund S, Richardson A, May C. Barriers to Advance Care Planning at the End of Life: An \n488 Explanatory Systematic Review of Implementation Studies. PLoS ONE. 2015;10(2):e0116629. doi: \n489 10.1371/journal.pone.0116629.\n490 44. Tulsky JA, Beach MC, Butow PN, Hickman SE, Mack JW, Morrison RS, et al. A Research \n491 Agenda for Communication Between Health Care Professionals and Patients Living With Serious \n492 Illness. JAMA Internal Medicine. 2017;177(9):1361-6. doi: 10.1001/jamainternmed.2017.2005.\n493 45. Harvey N, Holmes CA. Nominal group technique: An effective method for obtaining group \n494 consensus. International Journal of Nursing Practice. 2012;18(2):188-94. doi: \n495 https://doi.org/10.1111/j.1440-172X.2012.02017.x.\n496 46. Spencer L, Ritchie J, Lewis J. Quality in qualitative evaluation: a framework for assessing \n497 research evidence. London: 2003 2003. Report No.: Occasional Series No 2.\n498 47. Berger Z. Navigating the unknown: shared decision-making in the face of uncertainty. \n499 Journal of general internal medicine. 2015;30:675-8.\n500 48. van Vliet LM. Balancing explicit with general information and realism with hope. \n501 Communication at the transition to palliative breast cancer care: Netherlands Institute for Health \n502 Services Research; 2013.\n503 49. Cox C, Fritz Z. What is in the toolkit (and what are the tools)? How to approach the study of \n504 doctor–patient communication. Postgraduate Medical Journal. 2022.\n505 50. Tunnard I, Yi D, Ellis-Smith C, Dawkins M, Higginson IJ, Evans CJ. Preferences and priorities to \n506 manage clinical uncertainty for older people with frailty and multimorbidity: a discrete choice \n507 experiment and stakeholder consultations. BMC Geriatrics. 2021;21(1). doi: 10.1186/s12877-021-\n508 02480-8.\n509 51. Connolly T, Coats H, DeSanto K, Jones J. The experience of uncertainty for patients, families \n510 and healthcare providers in post-stroke palliative and end-of-life care: a qualitative meta-synthesis. \n511 Age and Ageing. 2020. doi: 10.1093/ageing/afaa229.\n512 52. Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting \n513 DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic \n514 review. Palliat Med. 2017;31(8):684-706. Epub 2017/02/14. doi: 10.1177/0269216317690685. \n515 PubMed PMID: 28190381.\n516 53. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv \n517 Nurs. 2000;32(4):1008-15. Epub 2000/11/30. PubMed PMID: 11095242.\n518\n . CC-BY 4.0 International licenseIt is made available under a \n is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)\nThe copyright holder for this preprint this version posted July 24, 2023. ; https://doi.org/10.1101/2023.07.21.23293007doi: medRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}