A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment

Multiple Criteria Decision Analysis, Emergency Medical Services, Point of Care Testing ALL Metrics - Views Downloads How to cite this article Kirby K, Coggins J, Gibson A et al. A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.13580.2) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Research Article Revised A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved] Kim Kirby https://orcid.org/0000-0002-8092-7978 1,2, Jessica Coggins https://orcid.org/0000-0002-6663-6816 1, Andy Gibson1, [...] Cathy Liddiard1, Theresa H.M. Moore3, Jelena Savović https://orcid.org/0000-0002-2861-0578 3,4, Kimberley Mitchell1, Alexander Thompson5, Jonathan Benger https://orcid.org/0000-0001-6131-0916 1, Richard Body https://orcid.org/0000-0001-9089-8130 5Kim Kirby https://orcid.org/0000-0002-8092-7978 1,2, Jessica Coggins https://orcid.org/0000-0002-6663-6816 1, [...] Andy Gibson1, Cathy Liddiard1, Theresa H.M. Moore3, Jelena Savović https://orcid.org/0000-0002-2861-0578 3,4, Kimberley Mitchell1, Alexander Thompson5, Jonathan Benger https://orcid.org/0000-0001-6131-0916 1, Richard Body https://orcid.org/0000-0001-9089-8130 5 PUBLISHED 23 Jul 2024 Author details Author details 1 University of the West of England, Bristol, BS16 ODD, UK 2 South Western Ambulance Service NHS Foundation Trust, Exeter, EX2 7HY, UK 3 The National Institute for Health and Care Research Applied Research Collaboration West, Bristol, BS1 2NT, UK 4 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK 5 University of Manchester, Manchester, M13 9PL, UK 2 South Western Ambulance Service NHS Foundation Trust, Exeter, EX2 7HY, UK 3 The National Institute for Health and Care Research Applied Research Collaboration West, Bristol, BS1 2NT, UK 4 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK 5 University of Manchester, Manchester, M13 9PL, UK Kim Kirby Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Jessica Coggins Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Andy Gibson Roles: Investigation, Project Administration, Writing – Review & Editing Roles: Investigation, Project Administration, Writing – Review & Editing Cathy Liddiard Roles: Investigation, Project Administration, Writing – Review & Editing Roles: Investigation, Project Administration, Writing – Review & Editing Theresa H.M. Moore Roles: Data Curation, Formal Analysis, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Jelena Savović Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing Kimberley Mitchell Roles: Data Curation, Formal Analysis, Investigation Roles: Data Curation, Formal Analysis, Investigation Alexander Thompson Roles: Data Curation, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Jonathan Benger Roles: Conceptualization, Data Curation, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Richard Body Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing OPEN PEER REVIEW REVIEWER STATUS There are increasing demands on Emergency Medical Services. More efficient treatment pathways are required to support conveyance decision making and patient referral in prehospital care. Point of Care testing is increasingly available and utilised across the NHS to support optimal ways of working. We aimed to design and conduct a Multiple Criteria Decision Analysis to prioritise in vitro point of care tests and use cases for inclusion in a platform trial of in vitro point of care testing in UK Emergency Medical Services. We designed a Multiple Criteria Decision Analysis that included systematic scoping reviews stakeholder recruitment, two stakeholder surveys and two stakeholder workshops to scope the use cases, explore criteria and map use cases, evaluate the criteria and measure the use cases against the criteria. We recruited 32 stakeholders. We developed a scoring matrix with 4 criteria for scoring the use cases and 8 criteria for scoring the point of care tests and applied weighting determined from survey results. Use cases were scored by the stakeholders against 4 criteria. The 3 highest scoring use cases were point of care troponin testing in: possible Acute Myocardial Infarction, lactate testing in suspected sepsis and in trauma. We developed the process for scoring the point of care tests to be completed close to a proposed trial to allow for a changes in technology. We successfully designed a Multiple Criteria Decision Analysis to identify use cases and candidate tests for inclusion in a future platform trial of in vitro point of care testing in UK Emergency Medical Services. We identified 3 use cases for evaluation in a platform trial of in vitro point of care testing: troponin testing in possible acute myocardial infarction, lactate testing in suspected sepsis and lactate testing to identify occult haemorrhage in trauma. Plain Language Summary Aim of the research To identify the most useful bedside tests and patient conditions appropriate for bedside testing in ambulance services.

to the research Tests called point of care tests can be conducted at a patient’s side and quickly give clinicians information that can help with decision-making about the right treatment and pathway for patients. There are many of these tests available and we wanted to complete a trial to understand if tests are suitable for use in ambulance services and are the right tests for patients to have. Before we could conduct a trial we needed to understand which patient conditions are amenable to point of care testing in ambulance services and which tests are appropriate to use in this environment. We conducted research to design and complete a Multiple Criteria Decision Analysis (MCDA). MCDA is a method that allowed us to identify the most appropriate tests and conditions to enter into a trial of point of care testing in ambulance services. Design and methods used We reviewed the relevant literature to understand which tests were available and which conditions were appropriate for point of care testing in ambulance services. We completed surveys and workshops with stakeholders to identify criteria and a scoring system to score the tests and conditions that those tests can be used.

We scored the conditions in which point of care tests could be used in ambulance services. We identified that point of care tests were most suitable for patients that might be suffering a heart attack, patients that have suffered a severe injury and patients that may have a severe infection. Aim of the research To identify the most useful bedside tests and patient conditions appropriate for bedside testing in ambulance services.

to the research Tests called point of care tests can be conducted at a patient’s side and quickly give clinicians information that can help with decision-making about the right treatment and pathway for patients. There are many of these tests available and we wanted to complete a trial to understand if tests are suitable for use in ambulance services and are the right tests for patients to have. Before we could conduct a trial we needed to understand which patient conditions are amenable to point of care testing in ambulance services and which tests are appropriate to use in this environment. We conducted research to design and complete a Multiple Criteria Decision Analysis (MCDA). MCDA is a method that allowed us to identify the most appropriate tests and conditions to enter into a trial of point of care testing in ambulance services. Design and methods used We reviewed the relevant literature to understand which tests were available and which conditions were appropriate for point of care testing in ambulance services. We completed surveys and workshops with stakeholders to identify criteria and a scoring system to score the tests and conditions that those tests can be used.

We scored the conditions in which point of care tests could be used in ambulance services. We identified that point of care tests were most suitable for patients that might be suffering a heart attack, patients that have suffered a severe injury and patients that may have a severe infection. Multiple Criteria Decision Analysis, Emergency Medical Services, Point of Care Testing Corresponding Author(s) Kim Kirby ([email protected]) Grant information: This project is funded by the National Institute for Health and Care Research (NIHR) under its ‘Health Technology Assessment’ (Grant Reference Number NIHR156550). THM Moore’s, and J Savović’s time was supported by the NIHR Applied Research Collaboration West (NIHR ARC West). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2024 Kirby K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Kirby K, Coggins J, Gibson A et al. A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.13580.2) First published: 09 May 2024, 4:29 (https://doi.org/10.3310/nihropenres.13580.1) Latest published: 23 Jul 2024, 4:29 (https://doi.org/10.3310/nihropenres.13580.2) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Thank you for taking the time to review this paper and for your helpful comments. We have added more context to the reference to Rohr et al's paper in the Background section. Thank you for highlighting the point regarding the gold standard for two reviewers to review articles for inclusion in a scoping review. We have added an additional paragraph to the discussion section to address this point. The study team included clinical leaders in prehospital and emergency care, and also clinicians and researchers with expertise in prehospital POCT. Via personal networks and snowball sampling we identified and recruited stakeholders that we believe were satisfactorily representative, though we accept that it is challenging to determine whether a sample is truly representative and includes the widest possible range of expertise. We have therefore added the following to the “limitations” section. "Whilst we aimed to recruit a genuinely representative group of stakeholders, using personal networks and snowball techniques, the participants are unlikely to wholly reflect the full range of expertise relevant to pre-hospital POC testing, and as a result some views and areas of knowledge will have been under-represented in this research". We agree that limitations regarding evidence for POCTs could be more clearly emphasised, and have added the following text to the Discussion section. “POC testing is a relatively new development in EMS, and as a result the quantity and quality of evidence is limited currently. Even the POC tests that scored most highly in this research, such as troponin in chest pain, have relatively little evidence of their clinical effectiveness and more trials are required[24]. This field is anticipated to develop rapidly over the next few years[4], and we have established a methodology by which POC tests may continue to be evaluated and selected for further assessment in research”. Thank you for taking the time to review this paper and for your helpful comments. We have added more context to the reference to Rohr et al's paper in the Background section. Thank you for highlighting the point regarding the gold standard for two reviewers to review articles for inclusion in a scoping review. We have added an additional paragraph to the discussion section to address this point. The study team included clinical leaders in prehospital and emergency care, and also clinicians and researchers with expertise in prehospital POCT. Via personal networks and snowball sampling we identified and recruited stakeholders that we believe were satisfactorily representative, though we accept that it is challenging to determine whether a sample is truly representative and includes the widest possible range of expertise. We have therefore added the following to the “limitations” section. "Whilst we aimed to recruit a genuinely representative group of stakeholders, using personal networks and snowball techniques, the participants are unlikely to wholly reflect the full range of expertise relevant to pre-hospital POC testing, and as a result some views and areas of knowledge will have been under-represented in this research". We agree that limitations regarding evidence for POCTs could be more clearly emphasised, and have added the following text to the Discussion section. “POC testing is a relatively new development in EMS, and as a result the quantity and quality of evidence is limited currently. Even the POC tests that scored most highly in this research, such as troponin in chest pain, have relatively little evidence of their clinical effectiveness and more trials are required[24]. This field is anticipated to develop rapidly over the next few years[4], and we have established a methodology by which POC tests may continue to be evaluated and selected for further assessment in research”. We have added more context to the reference to Rohr et al's paper in the Background section. Thank you for highlighting the point regarding the gold standard for two reviewers to review articles for inclusion in a scoping review. We have added an additional paragraph to the discussion section to address this point. The study team included clinical leaders in prehospital and emergency care, and also clinicians and researchers with expertise in prehospital POCT. Via personal networks and snowball sampling we identified and recruited stakeholders that we believe were satisfactorily representative, though we accept that it is challenging to determine whether a sample is truly representative and includes the widest possible range of expertise. We have therefore added the following to the “limitations” section. "Whilst we aimed to recruit a genuinely representative group of stakeholders, using personal networks and snowball techniques, the participants are unlikely to wholly reflect the full range of expertise relevant to pre-hospital POC testing, and as a result some views and areas of knowledge will have been under-represented in this research". We agree that limitations regarding evidence for POCTs could be more clearly emphasised, and have added the following text to the Discussion section. “POC testing is a relatively new development in EMS, and as a result the quantity and quality of evidence is limited currently. Even the POC tests that scored most highly in this research, such as troponin in chest pain, have relatively little evidence of their clinical effectiveness and more trials are required[24]. This field is anticipated to develop rapidly over the next few years[4], and we have established a methodology by which POC tests may continue to be evaluated and selected for further assessment in research”. To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table. Demand on Emergency Medical (Ambulance) Services (EMS) continues to increase1. Lengthy delays in patient handover at crowded Emergency Departments (EDs) compromises the ability of EMS to respond to new emergency calls, leading to prolonged ambulance response times2. Strategies to improve efficiency by maximising the safe avoidance of patient conveyance to hospital wherever possible, while also ensuring patients who are transported receive the best care and reach the most appropriate facility, are paramount. An interview study with medical cardiologists and oncologists identified that laboratory testing guides 60–70% of clinical decisions in these contexts3. Emergency Departments utilise in vitro diagnostics for swift identification of patient conditions and to support decision making. In vitro Point of Care (POC) testing could enhance EMS by achieving shorter response times, less hospital transfers and reduced ED waits, with potential for substantial cost savings. POC testing is increasingly available and utilised across the NHS. Rapid growth in the POC testing market is predicted and may rival the pharmaceutical industry within 20 years4. Platform trials are designed to test multiple interventions at the same time5 and in 2023 the UK National Institute of Health and Care Research (NIHR) encouraged funding applications for its Health Technology Assessment (HTA) programme for platform trials of strategic priority6. While planning a platform randomised evaluation of clinical and cost effectiveness of in vitro POC testing in EMS in the UK, we required a robust and transparent process to triage POC tests and associated “use cases” for inclusion in a platform trial. Multiple criteria decision analysis (MCDA) is a commonly used method to guide complex decision making in HTA7 when there are multiple options and multiple relevant criteria to be taken into account during the decision-making process8. In particular, MCDA has been most frequently used within HTA to guide resource priority decision-making9. This paper describes the process and outcomes of MCDA methodology in the prioritisation of candidate in vitro POC tests and use cases for potential inclusion in a proposed future platform randomised controlled trial of in vitro POC testing in UK EMS. In this paper the term POC tests refers to in vitro POC testing. This work benefitted from a PPI co-applicant who had prior experience of relevant high-quality point of care testing research and who was involved in all aspects of the research. A wider group of Public Advisory Group members were involved in the following ways: Public Contributors (PCs) helped us to understand the range of variation in service provision and patient circumstances. This included looking at the potential alternative care pathways that can be accessed by testing and describing the facilitators and barriers to pre-hospital testing from a patient/carer perspective. They also advised on the pre-requisites that make pre-hospital testing and alternative care pathways acceptable from a patient and carer perspective. PCs were involved in the development of the Stakeholder Survey 1 to create the longlist of ‘criteria’ for assessment and the Stakeholder Survey 2 to define the weighting of criteria. PCs also participated fully in both stakeholder workshops. We conducted an MCDA to prioritise candidate in vitro POC tests and use cases for inclusion in a platform trial of POC testing in UK ambulance services. The MCDA was conducted in accordance with the principles described in the good practice guidelines for the conduct of MCDA published by the International Society for Health Economics and Outcomes Research (ISPOR)10. The MCDA was conducted between March and September 2023 and the process is illustrated in Figure 1. Identification of potential use cases The options (use cases) were identified through systematic literature search and selection process following principles of scoping review11. We did not assess the bias in the studies identified, nor did we attempt any formal synthesis of the evidence as these are outside of the methods used for scoping reviews12. Searching for relevant studies To identify POC tests used within, or with potential for use in the EMS we searched three electronic bibliographic databases (Medline and EMBASE on Ovid and CINAHL on Ebscohost). The search was developed by an information specialist (SD) and systematic reviewers (TM,JS) in liaison with the rest of the team. The strategy combined terms for point of care tests with terms for EMS. We did not apply a filter to restrict types of study design. The search strategies are available in an online data repository13. We searched reference lists of key systematic reviews. We restricted to English language and research published after 2000 and focused the search on adults. Following an initial broad search for POC tests in the EMS, additional supplementary searches for specific potential use cases were completed between April and August 2023. A table summarising the searches is presented in Extended Data File 1. Titles and abstracts of records were screened by one author (TM) with all excluded records checked for exclusion by a second member of the wider team (see acknowledgements). Full-text versions of the records marked “retrieve” by either reviewer were screened by one author (TM) with all excluded papers checked for exclusion by a second member of the wider team. Strict inclusion and exclusion criteria were used to guide selection, criteria included: reporting on a POC test used in the EMS, setting was the EMS, POC test was using bodily fluids (saliva, blood, serum, breath, sweat). We excluded studies where the focus was use of equipment to obtain a diagnosis from a body scan, such as ultrasound. We excluded tests used in specialist services such as mobile stroke units because of the reliance on specialist personnel and scanning equipment in conjunction with any POC tests. Search data were managed using Rayyan and Endnote software14,15. Screening was managed using the Microsoft software Access and Excel16,17. Preparation of stakeholder summaries Data extraction from studies identified in the scoping exercise was done by one researcher (TM) and checked by a second (KM,RP). Data extracted included: Type of test(s), test device name and manufacturer, study design, condition the test was being used for, biomarkers being tested. Information for potential use cases was organised into summaries for stakeholders into 6 types of information, 1) systematic review of primary research, 2) Economic evaluation of use of POC tests in the EMS, 3) ability to make causal inference for a change in patient pathway, from a) experimental studies with interventions assigned at random, or b) from non-randomised studies with a comparator group or controlled before and after studies 4) demonstration of the POC test being used in the EMS from observational studies describing the use of a POC test (single arm observation studies) 5) Views of paramedics or clinicians or patients or hospital staff on the use of POC tests in the EMS to change the patient pathway (qualitative studies or surveys). Studies investigating the diagnostic test accuracy of POC tests were documented and listed but not summarised as this information is not sufficient to ascertain if a change in patient pathway has potential. The number of records for each use case, and the type of information identified for each were organised into tables and stakeholder summaries that were provided at two workshops. The summaries presented at Workshop 2 are listed in (Extended Data File 1). Stakeholder recruitment The MCDA focused on UK EMS care, but recruited national and international stakeholders with the expertise to meet the inclusion criteria described in Figure 1. Stakeholders were identified and recruited during June and July 2023. Eligible stakeholders, who had been identified as lead authors of research found during the scoping review, or whose work and expertise was familiar to the study team, were contacted. Snowball sampling was then employed to recruit additional stakeholders. Initial contact was via an email providing information about the study. Where stakeholders expressed an interest in participating, they were provided with a participant information sheet and a consent form and asked to consent to participate in the study. In addition to seeking topic experts, we also recruited a sample of patient and public stakeholders from an established patient and public research advisory group. Public contributor workshops To support the full engagement of public contributors in the MCDA process we convened two virtual public contributor workshops. The initial workshop was designed to provide contributors with training on background information about the topic, the proposed platform trial, and the MCDA methodology. A member of the study team (KK) and an expert in Patient and Public Engagement (AG) presented information to the group and answered questions. Participants were also provided with written information. A subsequent workshop was convened and supported by JC, HN and AG to inform public contributors about the process that would be used in Workshop 2, and to ensure there was adequate support to enable patient and public contributors to engage fully in all activities. Stakeholder Survey 1 Stakeholder Survey 1 (Extended Data File 2) was disseminated to non-public contributor stakeholders to explore candidate criteria that should be considered when deciding whether a POC test is suitable for use in EMS. The survey contained questions focused on ‘a priori’ themes of promise, plausibility, risk, costs, the nature of the condition being investigated, the technology, the adopter system, the wider institutional and societal context, the evidence quality and other criteria which we selected from reviews of pertinent research by Campbel and Knox18 and Greenhalgh and colleagues19. Stakeholders were asked to rate the overall importance of each of the criteria using a five-point Likert scale but were also asked to provide free text responses to elaborate upon their answers. Responses to the survey were collated and reviewed by the research team. Likert scale responses were summarised using frequencies. Survey 1 results informed an update of a priori themes, producing a new shortlist of criteria to take forward into Stakeholder Workshop 1. Stakeholder Workshop 1 Prior to the workshop summaries of the evidence available for different use cases were provided to the stakeholders. Responses to Survey 1 were visually summarised using an online interface (Miro board20) and formed the basis of a subsequent discussion. During the online workshop, stakeholders were invited to map clinical pathways for candidate use cases (mapping both the current care pathway and the proposed amendments to the care pathway that introduction of a POC test would allow), discuss the importance of each shortlisted criterion and make suggestions or alterations to the shortlist. All ideas were captured on the Miro board and displayed to the stakeholders, who were asked to agree on the final content of the Miro board by the end of the workshop. The workshop was recorded using Microsoft Teams. We then compared the refined shortlist of criteria against the Evidence and Value: Impact on Decision Making (EVIDEM) framework21, which has previously been used for MCDA involving health technologies22. Following an iterative process of coding and refinement within the study team, we reached a final consensus that the list of criteria identified was truly representative of stakeholder opinions, while being sufficiently specific and measurable to use in the MCDA process. The study team also prepared a draft rating system with which to score each criterion. We searched for published examples of established scoring systems and used these wherever possible. Where that was not possible, the team agreed three- or four-point scoring systems to put to the stakeholders. Stakeholder Survey 2 To validate and finalise the list of criteria to be used for scoring and to determine the weightings that would be used to score those criteria, we disseminated a second survey (Extended Data File 2) to non-public contributor stakeholders. Each of the shortlisted criteria was presented back to the stakeholders. They were asked three questions about each criterion. First, they were asked to score the importance of the criterion on an 11-point scale. This would later be used to assign weightings to the final criteria. Second, they were asked to provide feedback on the proposed rating system. They were asked if they felt that the rating system was appropriate. If not, we asked stakeholders to explain why not and to propose alternatives. Finally, we asked the stakeholders if any of the categories should be used to exclude a POC test from being considered in a future platform trial. This allowed us to identify the ‘deal breakers’ or satisfice criteria, which would exclude a POC test from further consideration. The responses to the survey were collated and analysed within the study team. The list of criteria was finalised and stratified into two sections: criteria for prioritising use cases; and criteria for prioritising individual POC tests or assays within those use cases. Scoring systems and satisfice criteria were finalised based on consensus of the stakeholders, with review within the study team for every instance of disagreement. Outputs were taken forward into Stakeholder Workshop 2. Stakeholder Workshop 2 Both expert and public contributor stakeholders were invited to participate in the online Stakeholder Workshop 2. Prior to the workshop, all stakeholders were provided with the final list of options (use cases) identified alongside their associated care pathways (current and POC test-driven), which had been mapped earlier in the project (Extended Data File 4). A summary of the evidence identified in the updated systematic scoping exercise was also provided for each use case (Extended Data File 1). During the first half of the second workshop, the final list of criteria for scoring use cases was presented back to all stakeholders, who were given an opportunity to make any final adjustments. Once consensus had been achieved, we proceeded to the second stage of the workshop, during which the options (candidate use cases) were presented and scored. The relevant care pathways were explained to stakeholders, and further comment was invited. The summary of evidence for each use case was then presented. The summary was provided in language that was comprehensible to a lay person, and there was an opportunity for all stakeholders to ask questions. Once we had confirmed that the stakeholders had understood the information and after answering any questions, we proceeded to invite the stakeholders to score the options against each of the agreed criteria using the scoring systems developed earlier in the project. This was done via Mentimeter23. Average scores from Survey 2 provided the weighting figure assigned to each criteria. The pre-determined weightings and satisfice criteria were used to create a template scoring matrix prior to the workshop. Once stakeholders had provided a score, this was averaged (mean) and added to the matrix by a study team member during the workshop. The weighting and average figures were multiplied for each criteria, and this provided a final score for each use case. By collating the scores contemporaneously, we were then able to present the final results back to the stakeholders and ensure that consensus had been achieved about the final scores and the ranking of use cases. Having ranked the use cases, we selected the top three scoring use cases for inclusion in a future platform trial. Achieving consensus on scoring POC tests Prior to completing Stakeholder Workshop 2, we also presented the agreed criteria for scoring in vitro POC tests and scoring systems for rating individual tests/assays within the relevant use cases. Consensus was achieved for each prior to the end of the workshop. We then identified all eligible commercially supplied, CE- or UKCA-marked assays for the relevant test. This list will be updated and POC tests scored prior to a platform trial of in vitro POC tests in EMS to allow for industry advancements and for the study team to systematically review each device before scoring. Options (use case) - Identification Systematic scoping reviews of POC tests in EMS supplemented by surveys of industry organisations, identified the candidate use cases shown in Table 1. Stakeholder recruitment We recruited 32 stakeholders to participate in the study and their expertise and country of origin is shown in Table 2. Stakeholder Survey 1 Fifteen “expert” participants (54% response rate) completed Stakeholder Survey 1 in May and June 2023. Charts displaying the results from the Likert questions are shown in Figure 2. Free text responses to Survey 1 are included in Extended Data File 3. Stakeholder Workshop 1 Stakeholder Workshop 1 was conducted on June 7th 2023 with 13 stakeholder participants. The findings of Survey 1 were discussed during the workshop and a list of criteria to continue into Survey 2 for scoring and weighting was discussed and developed. These are shown in Table 3. Use cases were mapped and discussed in Stakeholder Workshop 1 and through consensus the most appropriate use cases for further analysis were identified, based principally on available technology and pathway readiness. Identified use cases were ketone testing in diabetic ketoacidosis, NT-proBNP testing in heart failure, lactate testing in trauma, lactate testing in sepsis, bacterial or viral testing in respiratory tract infection, biomarker testing in traumatic head injury and troponin testing in chest pain. Use case mapping is included in Extended Data File 4. Stakeholder Survey 2 Fifteen expert participants (54% response rate) completed Survey 2 in August 2023. The results are included in Extended Data File 6. Table 4 details the scoring and satisfice results of Survey 2. Options meeting satisfice criteria (coagulation testing in trauma, blood gas testing in COPD, NT-proBNP in heart failure and sepsis, C-reactive protein in sepsis, Beta-HCG in pregnancy, biomarkers of traumatic brain injury) were automatically excluded from entering the platform trial because they met the satisfice criteria. This left five use cases to be scored during Stakeholder Workshop 2. Stakeholder Workshop 2 There were 14 participants in Stakeholder Workshop 2. Participants used Mentimeter23 to score each of the criteria for the five included use cases, this is detailed further in Table 5. Weighting was applied to the criteria: Potential to alter the clinical pathway; Certainty of evidence for diagnostic accuracy; Certainty of evidence for clinical effectiveness; Size of population affected. The weighting applied was the average ‘importance’ scoring given to the criteria by participants in Survey 2. The full Mentimeter23 results are included in Extended Data File 7 and Table 6 illustrates the results of the use case scoring. The three highest scoring use cases; POC troponin testing in possible Acute Myocardial Infarction (AMI), POC lactate testing in suspected sepsis and POC lactate testing in trauma, were prioritised for evaluation within a future platform trial of POC testing in EMS. Achieving consensus on scoring POC tests The satisfice criteria for evaluating the POC tests detailed was agreed with the stakeholders during Stakeholder Workshop 2 and this is detailed within Table 4. We developed an MCDA process for identifying the most appropriate options (use cases and in vitro POC tests) to enter into a platform trial of POC testing in UK EMS. The process consisted of identifying relevant use cases and POC tests through systematic searching of the literature and industry engagement. We identified a relevant stakeholder group and worked with these stakeholders to: scope the use cases; explore the criteria and map the use cases; evaluate the criteria; measure the use cases against the criteria and achieve consensus on scoring in vitro POC tests. Following this process, we identified that the most appropriate use cases to progress into a subsequent platform trial are POC troponin testing in possible AMI, POC lactate testing in suspected sepsis and POC lactate testing in trauma. POC testing is a relatively new development in EMS, and as a result the quantity and quality of evidence is limited currently. Even the POC tests that scored most highly in this research, such as troponin in chest pain, have relatively little evidence of their clinical effectiveness and more trials are required24. This field is anticipated to develop rapidly over the next few years4, and we have established a methodology by which POC tests may continue to be evaluated and selected for further assessment in research. There is a general recognition that there are few resources to support organisations in evaluating POC tests for implementation, and that often POC tests are developed that appear to meet clinical requirements, but then fail to be adopted successfully into practice25. There are no studies that investigate an MCDA in relation to POC testing specifically. This study has developed a formal and reproducible methodology to evaluate alternatives and priorities for use cases and POC tests that may be included in a platform trial of POC testing in UK EMS, facilitating transparency in decision-making. Our study presents a quantitative MCDA as opposed to qualitative, or MCDA with decision rules10. Quantitative MCDA is considered more robust because the use of scores and weightings improves the consistency of recommendations. There are well-documented benefits of using MCDA for HTA. These include a sound structuring of the decision problem, avoidance of bias in complex decision-making, and a transparent, consistent and reproducible process26. The method is particularly attractive when there are many important criteria to consider when triaging the available options. These qualities are important in the prioritisation of POC tests to enter into a platform trial of POC testing in UK EMS. In addition, our process of MCDA has included a wide range of stakeholders, including public contributors allowing greater participation in decision-making. An argument against using MCDA in HTA, and a factor that we recognise as a potential limitation in our methods, is that the management of the scoring and weighting of factors by researchers can inhibit debate, rather than stimulate it26. However, our stakeholders were informed of all decision-making and indicated throughout the process that they were content with the methods used and the outcomes. Whilst we aimed to recruit a genuinely representative group of stakeholders, using personal networks and snowball techniques, the participants are unlikely to wholly reflect the full range of expertise relevant to pre-hospital POC testing, and as a result some views and areas of knowledge will have been under-represented in this research. Due to resource constraints, during the scoping review of the literature only one reviewer screened articles for inclusion. We recognise that the gold standard for a scoping review is to have two, or more reviewers screening articles for inclusion27. Moving forward we will further refine and develop this MCDA process and continue to utilise it to determine which POC tests are appropriate for inclusion in an EMS platform trial. This research will also be informative for healthcare organisations and guideline decision makers on the appropriateness of POC tests for EMS and related settings, recognising that this MCDA was completed with UK EMS in mind and may not be generalisable to international settings. Continuation of research in this area in a future trial of POC testing in EMS is warranted to understand the factors that influence successful adoption and implementation of POC testing in UK EMS. We report on the development and use of MCDA methodology to identify candidate tests for inclusion in a future platform trial of POC testing in UK EMS. After identifying stakeholders and candidate tests (options) through systematic scoping reviews and surveys of industry organisations, we used a series of stakeholder surveys and workshops to define the criteria for selection of use cases and in vitro POC tests, and determine the appropriate weighting and scoring criteria. Through this process, we identified three use cases for evaluation in a platform trial: troponin testing in possible AMI, lactate testing in suspected sepsis and lactate testing to identify occult haemorrhage in trauma. Ethics committee approval was granted on 2nd June 2023 by the University of the West of England, Faculty of Health and Applied Sciences Faculty Research Ethics Committee, reference: UWE REC REF No: HAS.23.04.101. All participants in this research provided written informed consent prior to their participation. Open Science Framework: PrePOCTED Multiple Criteria Decision Analysis Extended Data. https://doi.org/10.17605/OSF.IO/3YUBA28. This project contains the following underlying data: Extended Data file 1. (Search plan for point of care tests in the emergency medical services, and evidence summaries presented to stakeholder workshops) Extended Data file 2. (Stakeholder Survey 1) Extended Data file 3. (Stakeholder Survey 1 results) Extended Data file 4. (Clinical Pathway Mapping) Extended Data file 5. (Stakeholder Survey 2) Extended Data file 6. (Stakeholder Survey 2 Results) Extended Data file 7. (Workshop 2 Meni-Meter results) Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication). KK - data curation, formal analysis, investigation, project administration, supervision, writing – original draft preparation, writing – review and editing; JC - data curation, formal analysis, investigation, project administration, writing – original draft preparation, writing – review and editing; AG - investigation, project administration, writing – review and editing; CL - investigation, project administration, writing – review and editing; THM – methodology, data curation, formal analysis, investigation, supervision, writing – original draft , writing – review and editing; JS – methodology, project administration, supervision, writing – review and editing; KM - data curation, formal analysis, investigation; AT - investigation, methodology, data curation, supervision, writing – original draft preparation, writing – review and editing; JB conceptualization, funding acquisition, methodology, investigation, supervision, writing – original draft preparation, writing – review and editing; RB – conceptualization, funding acquisition, Data curation, methodology, formal analysis, investigation, supervision, writing – original draft preparation, writing – review and editing. We would like to acknowledge the work of Sarah Dawson of NIHR ARC West and Population Health Sciences, University of Bristol, in designing and running the search strategies; Yetunde Adepoju a student who helped with screening titles and abstracts and data extraction; Dr Helen Nicholson (University of the West of England), Dr Jac Dinnes, University of Birmingham and Dr Rachel Perry of NIHR ARC West and Population Health Sciences, University of Bristol for help with screening titles and abstracts, data extraction and preparation of stakeholder summaries. We would also like to acknowledge our stakeholders for contributions to the research. Our stakeholders include: Dr Ankur Gupta-Wright (Consultant in Infectious Diseases and General Internal Medicine, North Bristol NHS Trust, UK), Dr Kristin Aakre (Consultant, Associate Professor, Department of Medical Biochemistry and Pharmacology, Haukeland University Hospital, University of Bergen, Norway), Dr Virginia Newcombe (Intensive Care Medicine and Emergency Physician and Clinician Scientist, University of Cambridge, UK), Dr Jason Stopyra (Associate Professor Department of Emergency Medicine, Wake Forest University School of Medicine, North Carlina, US; EMS Medical Director, US), Clare Davenport (Clinical Associate Professor, University of Birmingham, UK), Amanda Threfall (Public Contributor), Samina Begum (Public Contributor), Dr Larissa Prothero (Research Paramedic, East of England Ambulance Service NHS Trust, UK), Chloe Rowlinson (Clinical Lead, South Western Ambulance Service NHS Foundation Trust). Finally, we would like to acknowledge the support of our wider group of public contributors in the design, development and delivery of this research. Faculty Opinions recommendedReferences - 1. Public Services Committee: House of Lords - Emergency healthcare: a national emergency - public services committee. 2023; Accessed January 3, 2024. Reference Source - 2. Association of Ambulance Chief Executives: National ambulance handover delays. National handover delays. 2023; Accessed January 3, 2024. Reference Source - 3. Rohr UP, Binder C, Dieterle T, et al.: The value of in vitro diagnostic testing in medical practice: a status report. PLoS One. 2016; 11(3): e0149856. PubMed Abstract | Publisher Full Text | Free Full Text - 4. Global Market Insights: Point of care testing market size. 2023. Reference Source - 5. Burki T: Platform trials: the future of medical research? Lancet Respir Med. 2023; 11(3): 232–233. PubMed Abstract | Publisher Full Text | Free Full Text - 6. National Institute for health and Care Research: 22/104 HTA application acceleration award – platform studies in areas considered strategic priorities commissioning brief. 2022; Accessed January 3, 2024. Reference Source - 7. Marsh KD, Sculpher M, Caro JJ, et al.: The use of MCDA in HTA: great potential, but more effort needed. Value Health. 2018; 21(4): 394–397. PubMed Abstract | Publisher Full Text - 8. Baran-Kooiker A, Czech M, Kooiker C: Multi-Criteria Decision Analysis (MCDA) models in Health Technology Assessment of orphan drugs—a systematic literature review. next steps in methodology development? Front Public Health. 2018; 6: 287. PubMed Abstract | Publisher Full Text | Free Full Text - 9. Gongora-Salazar P, Rocks S, Fahr P, et al.: The use of multicriteria decision analysis to support decision making in healthcare: an updated systematic literature review. Value Health. 2023; 26(5): 780–790. PubMed Abstract | Publisher Full Text - 10. Thokala P, Devlin N, Marsh K, et al.: Multiple Criteria Decision Analysis for health care decision making--an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016; 19(1): 1–13. PubMed Abstract | Publisher Full Text - 11. Arksey H, O’Malley L: Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005; 8(1): 19–32. Publisher Full Text - 12. Peters MDJ, Godfrey CM, Khalil H, et al.: Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015; 13(3): 141–146. PubMed Abstract | Publisher Full Text - 13. Moore T, Dawson S, Savovic J, et al.: POC tests in EMS - search strategies and search history. Published online 2023. Publisher Full Text - 14. Ouzzani M, Hammady H, Fedorowicz Z, et al.: Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016; 5(1): 210. PubMed Abstract | Publisher Full Text | Free Full Text - 15. The EndNote Team: EndNote. Published online 2013. - 16. Microsoft: Microsoft access. Published online 2021. Reference Source - 17. Microsoft: Microsoft excel. Published online 2023. Reference Source - 18. Campbell B, Knox P: Promise and plausibility: health technology adoption decisions with limited evidence. Int J Technol Assess Health Care. 2016; 32(3): 122–125. PubMed Abstract | Publisher Full Text - 19. Greenhalgh T, Wherton J, Papoutsi C, et al.: Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017; 19(11): e367. PubMed Abstract | Publisher Full Text | Free Full Text - 20. MIRO: Miro | The visual workspace for innovation. 2023; Accessed January 3, 2024. Reference Source - 21. Goetghebeur MM, Wagner M, Khoury H, et al.: Evidence and value: impact on DEcisionMaking – the EVIDEM framework and potential applications. BMC Health Serv Res. 2008; 8(1): 270. PubMed Abstract | Publisher Full Text | Free Full Text - 22. Öztürk N, Tozan H, Vayvay Ö: A New decision model approach for Health Technology Assessment and a case study for dialysis alternatives in Turkey. Int J Environ Res Public Health. 2020; 17(10): 3608. PubMed Abstract | Publisher Full Text | Free Full Text - 23. Mentimeter: Interactive presentation software. Mentimeter. Accessed January 3, 2024. Reference Source - 24. Vrints CJM: Are we ready for prehospital troponin testing by paramedics? Evid Based Nurs. 2024; 27(2): 64. PubMed Abstract | Publisher Full Text - 25. Korte BJ, Rompalo A, Manabe YC, et al.: Overcoming challenges with the adoption of point-of-care testing: from technology push and clinical needs to value propositions. Point Care. 2020; 19(3): 77–83. PubMed Abstract | Publisher Full Text | Free Full Text - 26. Hansen P, Devlin N: Multi-Criteria Decision Analysis (MCDA) in healthcare decision-making. In: Oxford Research Encyclopedia of Economics and Finance. 2019. Publisher Full Text - 27. Peters MDJ, Godfrey C, McInerney P, et al.: Scoping Reviews. In: JBI Manual for Evidence Synthesis. 2020; Accessed July 11, 2024. Publisher Full Text - 28. Kirby K, Coggins J, Gibson A, et al.: PrePOCTED multiple criteria decision analysis extended data. [Dataset]. 2024; Accessed April 29, 2024. https://osf.io/3yuba/ Author details Author details 1 University of the West of England, Bristol, BS16 ODD, UK 2 South Western Ambulance Service NHS Foundation Trust, Exeter, EX2 7HY, UK 3 The National Institute for Health and Care Research Applied Research Collaboration West, Bristol, BS1 2NT, UK 4 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK 5 University of Manchester, Manchester, M13 9PL, UK 2 South Western Ambulance Service NHS Foundation Trust, Exeter, EX2 7HY, UK 3 The National Institute for Health and Care Research Applied Research Collaboration West, Bristol, BS1 2NT, UK 4 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK 5 University of Manchester, Manchester, M13 9PL, UK Kim Kirby Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Jessica Coggins Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Andy Gibson Roles: Investigation, Project Administration, Writing – Review & Editing Roles: Investigation, Project Administration, Writing – Review & Editing Cathy Liddiard Roles: Investigation, Project Administration, Writing – Review & Editing Roles: Investigation, Project Administration, Writing – Review & Editing Theresa H.M. Moore Roles: Data Curation, Formal Analysis, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Jelena Savović Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing Kimberley Mitchell Roles: Data Curation, Formal Analysis, Investigation Roles: Data Curation, Formal Analysis, Investigation Alexander Thompson Roles: Data Curation, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Jonathan Benger Roles: Conceptualization, Data Curation, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Richard Body Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Competing interests AT reports consultancy fees from Siemens Healthineers and Perspectum. RB reports research grants with Roche Diagnostics, Siemens Healthineers and Abbott Point of Care; consultancy with Roche Diagnostics, Siemens Healthineers, Abbott Point of Care, Beckman Coulter, Radiometer, LumiraDx, Aptamer Group and Psyros Diagnostics (money to institution); speaker fees with EMCREG (Emergency Medicine Cardiac Research and Education Group) International (personal honoraria). support for travel/accommodation to conferences: Siemens Healthineers, Roche Diagnostics, Astra Zeneca. No other competing interests were disclosed. Grant information This project is funded by the National Institute for Health and Care Research (NIHR) under its ‘Health Technology Assessment’ (Grant Reference Number NIHR156550). THM Moore’s, and J Savović’s time was supported by the NIHR Applied Research Collaboration West (NIHR ARC West). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Article Versions (2) Copyright © 2024 Kirby K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article Kirby K, Coggins J, Gibson A et al. A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.13580.2) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 2 VERSION 2 PUBLISHED 23 Jul 2024 Revised Views 0 How to cite this report: Tolsma RT. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14854.r32407) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32407 https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32407 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 22 Aug 2024 Approved VIEWS 0 No ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close No new comments Competing Interests: No competing interests were disclosed. Reviewer Expertise: Prehospital care, riskstratification, point of care testing in healthcare CITE HOW TO CITE THIS REPORT Tolsma RT. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14854.r32407) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32407 https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32407 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Ebbs D. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14854.r32406) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32406 https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32406 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 07 Aug 2024 Approved VIEWS 0 Authors addressed concerns adequately ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close Authors addressed concerns adequately for full approval for indexing. Competing Interests: No competing interests were disclosed. Reviewer Expertise: Prehospital emergency services, Prehospital POCTs, Prehospital Predictive Modeling CITE HOW TO CITE THIS REPORT Ebbs D. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14854.r32406) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32406 https://openresearch.nihr.ac.uk/articles/4-29/v2#referee-response-32406 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Version 1 VERSION 1 PUBLISHED 09 May 2024 Views 0 How to cite this report: Ebbs D. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14741.r32010) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-32010 https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-32010 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 27 Jun 2024 Approved with Reservations VIEWS 0 Thank you for this opportunity to review your article, A Multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close Thank you for this opportunity to review your article, A Multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment. This is an important and timely study. The use of POCTs has the potential to serve as early triage for several medical conditions as discussed in this article. This manuscript aims to design an MCDA to set priority to potential relevant POCTs in the prehospital setting which can guide future stakeholders for implementation. Intro: Reliance on in vitro lab for medical decision making is highly over-exaggerated, as referenced by Rohr in “The Value of In Vitro Diagnostics..” in this introduction. This one citation is used as a supporting argument in the introduction, which interviews highly sub-specialized physicians (cardiologist/oncologists) who rely on POCTs for very specific chronic medical conditions. Although medical providers are highly reliant on these diagnostics, 60-70% is extreme, and this is stated without any context.

The use of multiple criteria decision analysis (MCDA) is both innovative and appropriate for this study. Appropriate guidelines followed via ISPOR. For article review, the gold standard for scoping reviews as outlined by more contemporary sources, such as Joanna Brigg Institute, typically requires two reviewers to review articles for inclusion; having one reviewer select all inclusion can introduce bias which should be noted in discussion. It is noted that a second reviewer was utilized for exclusion review. The stakeholder survey methods and results are not representative of experts; the breadth of POCTs in the prehospital setting, particularly among EMS, is novel and new to the last decade, and there is limited data in the form of studies, as you noted (cited in extended file 1) , on most prehospital POCTs. Please expand on how reaching out to random authors who publish on this or applying snowball methods can be synonymous to expertise in the field. Please expand on the limitation section that describes stakeholders.

should also discuss serious limitations in clinical utility/effectiveness of each POCT and discuss the lack in quality evidence overall in the impact assessment. For example, Troponin, which scored upward of 3 on the evidence of clinical effectiveness, has only three RCTs, and limited overall evidence towards its benefit; yet it is cited among experts that this is above moderate. Intro: Reliance on in vitro lab for medical decision making is highly over-exaggerated, as referenced by Rohr in “The Value of In Vitro Diagnostics..” in this introduction. This one citation is used as a supporting argument in the introduction, which interviews highly sub-specialized physicians (cardiologist/oncologists) who rely on POCTs for very specific chronic medical conditions. Although medical providers are highly reliant on these diagnostics, 60-70% is extreme, and this is stated without any context.

The use of multiple criteria decision analysis (MCDA) is both innovative and appropriate for this study. Appropriate guidelines followed via ISPOR. For article review, the gold standard for scoping reviews as outlined by more contemporary sources, such as Joanna Brigg Institute, typically requires two reviewers to review articles for inclusion; having one reviewer select all inclusion can introduce bias which should be noted in discussion. It is noted that a second reviewer was utilized for exclusion review. The stakeholder survey methods and results are not representative of experts; the breadth of POCTs in the prehospital setting, particularly among EMS, is novel and new to the last decade, and there is limited data in the form of studies, as you noted (cited in extended file 1) , on most prehospital POCTs. Please expand on how reaching out to random authors who publish on this or applying snowball methods can be synonymous to expertise in the field. Please expand on the limitation section that describes stakeholders.

should also discuss serious limitations in clinical utility/effectiveness of each POCT and discuss the lack in quality evidence overall in the impact assessment. For example, Troponin, which scored upward of 3 on the evidence of clinical effectiveness, has only three RCTs, and limited overall evidence towards its benefit; yet it is cited among experts that this is above moderate. - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Partly - Are all the source data underlying the results available to ensure full reproducibility? Yes - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: Prehospital emergency services, Prehospital POCTs, Prehospital Predictive Modeling CITE HOW TO CITE THIS REPORT Ebbs D. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14741.r32010) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-32010 https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-32010 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Tolsma RT. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14741.r31901) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-31901 https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-31901 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 11 Jun 2024 Approved VIEWS 0 Thank you for the opportunity to review this article. The researchers did a very thorough job and have also given an excellent coverage of this in this article. In developing an MDCA, the researchers, with input from ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Close The researchers did a very thorough job and have also given an excellent coverage of this in this article. In developing an MDCA, the researchers, with input from ... Continue reading Thank you for the opportunity to review this article. The researchers did a very thorough job and have also given an excellent coverage of this in this article. In developing an MDCA, the researchers, with input from Public Contributors and stakeholders, have been able to identify 3 key use cases that could be further explored in a platform trial in the (near) future. The topic focuses on important developments in prehospital care, namely the use of point-of-care diagnostics. The use of these diagnostics helps in the decision-making of healthcare professionals, strives for the right care in the right place for patients and offers opportunities to reduce overcrowding in emergency departments and improve ambulance availability. The researchers did a very thorough job and have also given an excellent coverage of this in this article. In developing an MDCA, the researchers, with input from Public Contributors and stakeholders, have been able to identify 3 key use cases that could be further explored in a platform trial in the (near) future. The topic focuses on important developments in prehospital care, namely the use of point-of-care diagnostics. The use of these diagnostics helps in the decision-making of healthcare professionals, strives for the right care in the right place for patients and offers opportunities to reduce overcrowding in emergency departments and improve ambulance availability. - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Yes - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? Yes - Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Prehospital care, riskstratification, point of care testing in healthcare CITE HOW TO CITE THIS REPORT Tolsma RT. Reviewer Report For: A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment [version 2; peer review: 2 approved]. NIHR Open Res 2024, 4:29 (https://doi.org/10.3310/nihropenres.14741.r31901) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-31901 https://openresearch.nihr.ac.uk/articles/4-29/v1#referee-response-31901 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | || |---|---|---| | 1 | 2 | | | Version 2 (revision) 23 Jul 24 | read | read | | Version 1 09 May 24 | read | read | Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...