Protocol for a trial-based economic evaluation analysis of a complex digital health intervention including a computerised decision support tool: the iFraP intervention

Digital Health Intervention, Cost-effectiveness, Value of Information, Osteoporosis ALL Metrics - Views Downloads How to cite this article Siciliano M, Bathers S, Bentley I et al. Protocol for a trial-based economic evaluation analysis of a complex digital health intervention including a computerised decision support tool: the iFraP intervention [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:15 (https://doi.org/10.3310/nihropenres.13575.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Study Protocol [version 1; peer review: 1 approved with reservations] Michele Siciliano https://orcid.org/0009-0000-7354-5368 1, Sarah Bathers2, Ida Bentley3, [...] Laurna Bullock https://orcid.org/0000-0002-4193-1835 4, Andrea Cherrington5, Emma Clark https://orcid.org/0000-0001-8332-9052 6, Jane Fleming https://orcid.org/0000-0002-8127-2061 7,8, Clare Jinks4, Sarah Lewis https://orcid.org/0000-0003-3992-6938 2, Christian Mallen https://orcid.org/0000-0002-2677-1028 4, Elaine Nicholls4, Terence O'Neill9,10, Jo Smith https://orcid.org/0009-0007-9030-4110 2, David Webb https://orcid.org/0009-0009-9313-385X 2, Zoe Paskins https://orcid.org/0000-0002-7783-2986 4,11, Cynthia Iglesias-Urrutia1,12Michele Siciliano https://orcid.org/0009-0000-7354-5368 1, Sarah Bathers2, [...] Ida Bentley3, Laurna Bullock https://orcid.org/0000-0002-4193-1835 4, Andrea Cherrington5, Emma Clark https://orcid.org/0000-0001-8332-9052 6, Jane Fleming https://orcid.org/0000-0002-8127-2061 7,8, Clare Jinks4, Sarah Lewis https://orcid.org/0000-0003-3992-6938 2, Christian Mallen https://orcid.org/0000-0002-2677-1028 4, Elaine Nicholls4, Terence O'Neill9,10, Jo Smith https://orcid.org/0009-0007-9030-4110 2, David Webb https://orcid.org/0009-0009-9313-385X 2, Zoe Paskins https://orcid.org/0000-0002-7783-2986 4,11, Cynthia Iglesias-Urrutia1,12 PUBLISHED 03 Apr 2024 Author details Author details 1 Department of Health Sciences, University of York, York, England, UK 2 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 3 School of Medicine Research User Group, Keele University, Staffordshire, England, UK 4 Centre for Musculoskeletal Health Research, School of Medicine, Keele University, Staffordshire, England, UK 5 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 6 Bristol Medical School, Faculty of Health Sciences, University of Bristol, Bristol, England, UK 7 Cambridge Public Health, University of Cambridge, Cambridge, England, UK 8 Addenbrooke’s Hospital Fracture Liaison Service, Cambridge University Hospitals NHS Trust, Cambridge, England, UK 9 Centre for Epidemiology Versus Arthritis, The University of Manchester, Manchester, England, UK 10 NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Sciences Centre, Manchester, England, UK 11 Midlands Partnership University NHS Foundation Trust, Stoke-on-Trent, England, UK 12 Danish Centre for Healthcare Improvements (CHI), Aalborg University, Aalborg, North Denmark Region, Denmark 2 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 3 School of Medicine Research User Group, Keele University, Staffordshire, England, UK 4 Centre for Musculoskeletal Health Research, School of Medicine, Keele University, Staffordshire, England, UK 5 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 6 Bristol Medical School, Faculty of Health Sciences, University of Bristol, Bristol, England, UK 7 Cambridge Public Health, University of Cambridge, Cambridge, England, UK 8 Addenbrooke’s Hospital Fracture Liaison Service, Cambridge University Hospitals NHS Trust, Cambridge, England, UK 9 Centre for Epidemiology Versus Arthritis, The University of Manchester, Manchester, England, UK 10 NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Sciences Centre, Manchester, England, UK 11 Midlands Partnership University NHS Foundation Trust, Stoke-on-Trent, England, UK 12 Danish Centre for Healthcare Improvements (CHI), Aalborg University, Aalborg, North Denmark Region, Denmark Michele Siciliano Roles: Formal Analysis, Writing – Review & Editing Roles: Formal Analysis, Writing – Review & Editing Sarah Bathers Roles: Methodology, Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Ida Bentley Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Laurna Bullock Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Andrea Cherrington Roles: Methodology, Project Administration, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Project Administration, Software, Writing – Original Draft Preparation, Writing – Review & Editing Emma Clark Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Jane Fleming Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Clare Jinks Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Sarah Lewis Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Christian Mallen Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Elaine Nicholls Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Resources, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Resources, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Terence O'Neill Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Jo Smith Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing David Webb Roles: Investigation, Project Administration, Software, Writing – Review & Editing Roles: Investigation, Project Administration, Software, Writing – Review & Editing Zoe Paskins Roles: Conceptualization, Funding Acquisition, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Cynthia Iglesias-Urrutia Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing OPEN PEER REVIEW REVIEWER STATUS Digital health interventions (DHI) are associated with significant promise. In recent years, the need to assess the value of these healthcare technologies has motivated a debate regarding the suitability of existing economic evaluation methods in the context of DHI evaluation. Some have argued that robust economic evaluation methods may not be capable of capturing relevant DHI’s characteristics. Others consider that assessing the value of DHI might not be feasible. This protocol paper challenges that view. More specifically, it describes early Health Technology Assessments (HTA) methods to rigorously assess the value for money of a complex intervention including a digital decision support tool i.e., Improving uptake of Fracture Prevention drug treatments (iFraP) as a tracer intervention. iFraP is a complex intervention consisting of a computerised decision support tool, a clinician training package, and information resources to facilitate shared decision-making, increase informed medicine initiation and reduce levels of medicine discontinuation. iFraP’s development was motivated by a view that good quality shared decision-making conversations have the potential to improve levels of osteoporosis medicine uptake. An early economic evaluation of the iFraP intervention was designed to identify, measure, and evaluate the costs and health benefits of iFraP compared to usual practice in Fracture Liaison Services (FLSs). A within-trial cost-effectiveness from the perspective of the National Health Service and Personal Social Service in England will be conducted using patient’s self-reported health related quality of life (HRQoL) and resource use from the iFraP randomised controlled trial. Microanalysis will be used to estimate iFraP’s intervention cost. Finally, Bayesian Value of Information analysis will allow us to estimate an upper bound for the potential health benefits gained from reducing uncertainty on the impact of the iFraP intervention to support uptake and adherence with osteoporosis medicines. ISRCTN10606407 - https://doi.org/10.1186/ISRCTN10606407 In the last decade, the promise associated with Digital Health Interventions has been gaining traction. Differences and peculiarities associated with these complex interventions make their economic evaluation difficult. However, we challenge this view and provide a protocol for the economic evaluation of iFraP, a package of resources including a digital health intervention that aims to improve shared decision-making and uptake of to drug treatment in people with osteoporosis. The iFraP intervention includes a computerised Decision Support Tool (DST), clinician training package, and information resources, for use in UK Fracture Liaison Service consultations. This paper describes the methods that will be used to conduct a rigorous analysis of the value for money of the iFraP intervention compared with usual Fracture Liaison Service practice. In addition, we will explore the value of conducting further primary research to reduce any remaining uncertainty associated with iFraP’s value for money. To the best of our knowledge, this is the first protocol of a trial-based economic evaluation study of a Digital Health Intervention. Digital Health Intervention, Cost-effectiveness, Value of Information, Osteoporosis Corresponding Author(s) Cynthia Iglesias-Urrutia ([email protected]) Grant information: This trial was funded by the National Institute for Health and Care Research (NIHR) [Clinician Scientist Award (CS-2018-18-ST2-010)/NIHR Academy] to Professor Zoe Paskins. The views expressed are those of the author(s) and not necessarily those of the NIHR, NHS or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2024 Siciliano M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Siciliano M, Bathers S, Bentley I et al. Protocol for a trial-based economic evaluation analysis of a complex digital health intervention including a computerised decision support tool: the iFraP intervention [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:15 (https://doi.org/10.3310/nihropenres.13575.1) First published: 03 Apr 2024, 4:15 (https://doi.org/10.3310/nihropenres.13575.1) Latest published: 03 Apr 2024, 4:15 (https://doi.org/10.3310/nihropenres.13575.1) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. In recent years, digital transformation of health has been gaining considerable importance in the policy space. Digital health has been identified as a priority in national health system programmes1, continental projects2 and global initiatives3. Digital Health encompasses a broad set of medical technologies and interventions designed to improve health4. Different types of products fall under the category of digital health interventions (DHI), ranging from mHealth devices, Artificial Intelligence applications, genomics and, computerised decision support tools (CDSTs). Digital health technologies (DHT) - and the DHIs as part of which DHT are delivered - have been associated with a number of potential advantages, e.g., improving effectiveness, efficiency, accessibility, safety, and personalization5. However, DHIs also present some peculiarities that make their clinical and economical evaluation more nuanced than traditional health innovations from pharma or medical devices5–11. Some challenges include: Digital health’s novelty and heterogeneity, which have contributed to a lack of established methodologies for the economic evaluation of such tools8. The key role of user involvement in determining the efficacy of the intervention, which can critically determine the efficacy of a DHI (e.g., user / health technology interaction)6. The interventions can have a complex impact, i.e., one that simultaneously affects multiple outcomes6–10. In contrast, the traditional tools of cost-utility and cost-effectiveness analysis are meant to capture one dimension at a time. The consequences associated with the use of DHI can fall outside the targeted population. This can question the appropriateness of a National Health Service perspective to capture changes brought about by a Digital Health Tool (DHT) and associated intervention6,8. Often DHI are subject to several changes and updates throughout their development phase (i.e., they are incrementally developed). This makes traditional evaluation tools such as RCT not particularly suited for an evaluation and arguably, unable to match the more dynamic nature of digital health innovations7,10. Decision Support Tools (DSTs) – also called decision aids or conversation aids - are a family of digital health interventions which are key to support shared decision-making in pursue of personalised medicine, as highlighted in the National Institute for Health care and Excellence (NICE) guidelines on Shared Decision Making12. CDSTs go beyond being simple repositories of information, as they have the potential for individualized content, a high degree of interaction, and scalability13. Moreover, the information contained can be specifically curated for patients with varying degree of health literacy. Consequently, the patient is able to engage more with the clinician, moving from a traditional principal agent relationship to something closer to a partnership14. Nevertheless, the challenges connected to the evaluation of DHI also affect DSTs, as highlighted by Trenaman et al.15. In this paper we describe the protocol for the within-trial healthcare economic evaluation of a complex intervention including a digital DST: the iFraP intervention16. To the best of our knowledge, our protocol is the first one to illustrate how a within-trial cost-effectiveness and value of information (VoI) analysis will be conducted alongside a randomised controlled trial (RCT) of a DHI. This paper aims to contribute to the literature on how to design a healthcare economic evaluation study to rigorously estimate the value for money of a DHI. Osteoporosis is a disease that compromises bone structure, making bones weak and more likely to break17. The drug treatments for osteoporosis revolve around fracture risk reduction through bone strengthening. There are different types of osteoporosis medicine given either by tablet or injection (including bisphosphonates, RANK ligand inhibitor; and anabolic agents such as recombinant parathyroid hormone and anti-sclerostin monoclonal antibodies) with calcium and/or vitamin D supplements acting as treatment adjunct18. Despite the considerable burden of the disease and the numerous treatment options, there is a considerable treatment gap19 (i.e. proportion of people in whom osteoporosis medicine is recommended by clinical guidelines, but do not receive treatment). The treatment gap arises because people at risk are either not identified or offered treatment, or are offered treatment, but decide not to take it. The care gap is a newly proposed term that emerged during the development of the iFraP intervention to address the implicit assumption that all patients recommended medicine should take it regardless of their preferences19. This term highlights the gap (and possible solutions) as a clinician problem, not a patient problem, and puts emphasis on the importance of clinician behaviours to facilitate informed shared decision-making between healthcare professionals and patients. The reasons behind osteoporosis undertreatment are multiple: concerns about long term bisphosphonates efficacy18; safety20; uncertainty about what medicine actually achieves21; and, potential overconfidence with routinely used risk calculators known -under certain circumstances- to be associated with a risk of underestimating the risk of fractures22. To an extent, promoting better clinician-patient communication on bone health and treatment options, could contribute to address some of these challenges. Some osteoporosis decision tools have been developed to improve clinician-patient communication. However, these have not met the required international quality criteria and have as yet, failed to demonstrate effectiveness at improving drug initiation and/or persistence (collectively described as adherence23)24. The Improving uptake of Fracture Prevention drug treatments (iFraP), a complex digital health intervention, features a DST with an interactive representation of individual fracture risk, information about risk factors for osteoporosis, bone density, and accessible explanations of bone health and treatment risk/benefit profile, and has been co-developed to achieve this need. A detailed description iFraP’s development is provided elsewere16. Briefly, the iFraP intervention includes a CDST to support the patient-clinician discussion, a training package for the clinician on how to use the tool and consultation skills enhancement, and additional information resources for the patients, along with a print-out from the CDST; most elements of the complex intervention support the use of (training), or are a by-product (print-out), of the CDST as a DHI. All the resources have been conceptualized and developed following the MRC complex intervention development framework25, comprising i) an evidence synthesis including an evaluation/assessment of the available online patient information26; ii) an assessment of existing decision tools in osteoporosis24; iii) a Delphi study to inform the model content of the FLS consultations27 and iv) focus groups and individual interviews with patients and health professionals28. iFraP was developed following a collaborative approach including bone specialists, nurses, people with lived experience, GPs, behavioural psychologists, representatives from Health Literacy UK, the Royal Osteoporosis Society, academics, and health economists. The iFraP team designed the study to evaluate the effect of the iFraP intervention on patient-reported ease in decision-making about osteoporosis medicine, to examine the experience of care and effectiveness, and to assess the within-trial cost-effectiveness and VoI of the iFraP intervention compared with usual Fracture Liaison Service (FLS) practice. FLSs are services which enact secondary fracture prevention, and assess people who have low trauma fractures for osteoporosis. This is a two sided multicentre individual randomised controlled trial (RCT), with parallel process evaluation and health economic evaluation. Participants are randomised in a 1:1 ratio, using blocked randomisation stratified by FLS. The planned sample size is 380 patients who have sustained a fragility fracture (following a fall from standing height or less); have been referred to FLS; and, are set to receive a consultation from an FLS29. Adult patients aged 50 years and over who are eligible for and FLS consultation based on having a previous fragility fracture(s). The intervention is a consultation delivered by FLS clinicians with the aid of a dynamic, interactive, and patient-tailored CDST to communicate individual fracture risk, bone health and treatment recommendations to patients. The clinicians in the intervention arm will receive a dedicated training course and will partake in a 3-hour role play session with experts to familiarize with the new consultation. Lastly, additional information resources will be given to both the patient and the GP after the consultation. The comparator is the usual FLS NHS care, which does not involve the use of CDST to support patient-clinician discussion. The clinicians in the comparator arm will not have access to the iFraP training and resources. The data collection started on April 2023 and it is planned to end in July 2024, in line with a three-month follow-up duration for the trial. Within trial cost-effectiveness analysis of iFraP intervention compared with usual FLS practice and VoI analysis. The perspective for the analysis will be that of the English National Health Service and Personal Social Service. The costing year will be 2023. Main sources will be the 85th volume of the British National Formulary (BNF) (Covering the March-September 2023 period) and the 2023 edition of the Unit Costs of Health and Social Care manual (PSSRU). Due to the within-trial nature of the analysis, we will employ a three-month time horizon, equivalent to the length of the trial follow-up. Given that our time horizon will cover less than a year, we will not discount costs or health outcomes. The EQ5D-5L questionnaire is a standardised tool developed by the EuroQoL group, used to estimate generic health-related quality of life (HRQoL)30. In the questionnaire, respondents are asked to rate their severity level in the five areas of life (dimensions) of mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. In this version of the questionnaire, there are 5 possible answers to each question, equating to (1) no problems, (2) slight problems, (3) moderate problems, (4) severe problems, or (5) inability in the domain. The final profile of the respondents is a sequence of 5 digits (e.g. 12322), corresponding to the answers in the 5 dimensions. Patient participants will self report their health status at baseline, 2-week, and 3-month follow-ups. There are four main categories of costs: health care (services) utilisation, medications use, iFraP intervention, and medication side effects. Self-reported healthcare utilisation and medication use data will be collected via a patient questionnaire administered at baseline and 3 months. Osteoporosis medication uptake and adherence will be cross-referenced with hospital medical record review data. Further details of this can be found in the clinical protocol (submitted to NIHR Open Journal). Micro-costing will be used to estimate the unit cost associated with the development and delivery of the iFraP intervention. This will include decision support tool development cost and training costs. Consultation costs for iFraP and usual FLS consultations will not be estimated as this is the same for both study groups. This analysis will evaluate the Expected Value of Perfect Information (EVPI) which captures the hypothetical value of simultaneously eliminating all uncertainty from the economic evaluation. Our VoI analysis will be conceptualized as rapid VoI, in a framework similar to the one used in research prioritization31. After examining the data, we will evaluate eventual missing data strategies to implement, following the recommendations presented in Faria et al.32. More precisely, we expect to see a relatively low amount of missing data at baseline, making it possible to impute the missing values by using the group mean for any missing covariate. We will test the assumption of missing-at-random mechanism as described in Cro et al.33 and then inspect the missingness pattern in covariates and outcomes. If a monotonic pattern is found in the missing data over time, we will rely on Inverse Probability Weighting (IPW). If not, we will use multiple imputation (MI) methods. Currently, NICE recommends the conversion of EQ5D-5L into EQ5D-3L using one of the available mapping functions34. In line with this, we will perform our analysis employing the methods described by van Hout et al.35. In the sensitivity analysis, the mapping function used will be taken from Hernandez et al.36. The cost-effectiveness analysis will compute the intervention Incremental Cost effectiveness Ratio (ICER) and compare it with appropriate cost-effectiveness thresholds. The utility values will be described using beta-based regression models to account for the multimodality and limited acceptable range of values37. The costs will be modelled using GLM models, to account for the characteristic skewness of cost data, as suggested in Mihaylova et al.38. In order to account for potential correlation between costs and effectiveness, the ICER will be estimated by bootstrap techniques, which will also be used for the corresponding 95% Confidence Intervals (CI)39,40. Analyses will be conducted on the statistical software R. In short, the rapid VoI evaluation revolves around describing the intervention(s) net benefit(s) without specifying a full economic model, but only comparing the relative effectiveness of the intervention and its costs. Then, the uncertainty in relative effect and baseline event rates are described by suitable statistical distributions, constructed either by referring to the literature or by expert elicitation. Afterwards, a large number of individual values (i.e., realisations) are sampled from said distribution to estimate the health benefit distributions. Analysing the curves enables to estimate the consequences of uncertainty for the intervention. Following the procedures described in Glynn31, we present in Table 1 the minimum evidence required to perform a rapid VoI analysis. | Primary outcome measure | Treatment adherence* rate at 3 months. | |---|---| | Relative effectiveness | Variation in treatment adherence* rate at 3 months. | | Baseline event rate | Treatment adherence* at 3 months | | Annual incidence | Number of individuals subject to frailty fractures | | Minimum clinical difference (MCD) | Minimum variation in treatment compliance rate to produce a clinically relevant change. | | Cost of the study | In GBP £ | | Duration of the study | In months | | Length of time for which the new evidence is expected to be valuable | In months | | Discount rate | Yearly discount rate for costs and utilities | Osteoporosis impacts around 3.75 million individuals in the UK. The estimated prevalence in the over 50 age group is 21.9% among women and 6.7% among men. In 2019, the cost of osteoporotic fractures in the UK accounted for approximately 2.4% of public health national expenditure29. Bisphosphonates are an effective and cost-effective osteoporosis treaetment41. To date, limited uptake and adherence has prevented realising the promise associated with this bone therapy more fully. A computerised decision tool and consultation model were developed to improve shared decision making to enhance bisphosphonate uptake and adherence (iFraP). This manuscript describes the protocol for the economic evaluation of the iFraP intervention. Its findings will illustrate how a robust evaluation of the value for money of digital health interventions can be conducted in the context of randomised controlled studies. The paper shows how to go about designing a within trial economic evaluation of a digital health intervention (i.e. a decision support tool) and supporting resources. This trial-based evaluation is expected to be associated with high internal validity and minimise the impact of selection bias on our estimates of uptake of bisphosphonate therapy. To explore the decision uncertainty associated with the cost-effectiveness of iFraP to enhance bisphosphonate adherence we will conduct a VoI analysis. An electronic search of trial-based economic evaluation of digital health interventions study protocols identified only three study protocols and neither of these included a trial based VoI analysis42–44. No VoI analysis was found when searching the published literature for protocols of cost-effectiveness analyses of digital support tools. Evaluation of the cost-effectiveness of improving bisphosphonates uptake and adherence present both a short term and a life-time decision problem. The within trial cost effectiveness analysis described here will allow us to robustly explore the short-term impact of the iFraP intervention on osteoporosis medicine uptake, principally bisphosphonates. Estimating the value associated with improving adherence with bisphosphonate therapy over a life time horizon in England would require designing a model based economic evaluation. Funding restrictions limited our ability to design and implement a model based evaluation. Future research may focus on re-evaluating an existing osteoporosis decision analytic model to explore the long-term impact of the iFraP intervention on adherence with bisphosphonate bone health therapies. Ethical approval was obtained from East of Scotland Research Ethics Service (EoSRES) (22/ES/0038). Following initial approval from the Research Ethics Committee (REC), they will continually be informed of all substantial changes to the management of the study. Routine reporting will take place in line with REC requirements. Dissemination and knowledge mobilisation will be facilitated through national bodies and networks such as the ROS, journal papers and conference presentations. The results of this study will be made widely and freely available to all stakeholders; a summary of the results will be published on the Keele University and ROS website. Patient Advisory Group (PAG) members will advise on how to translate these into easily understandable messages and on how best to disseminate the results to the wider public. In addition to publications in open-access peer-reviewed journals, we will use NHS networks and links to professional bodies to support dissemination of the findings to all stakeholders and will use social media to promote the findings via our dedicated Twitter and Facebook feeds. The osteoporosis Research User Group (RUG) at Keele University comprises people with experience of osteoporosis and/or fragility fractures, or carers. These RUG members had substantial involvement in a previous study to identify patient and public priorities for research in osteoporosis, which provided the starting point for iFraP. Furthermore, the study-specific Patient Advisory Groups (PAG) informed and agreed how public contributors will be involved throughout the iFraP programme at the outset. PAG meetings facilitated the development of the iFraP intervention and PAG members specifically commented on the importance of evaluating economic effectiveness of decision support tools and shared decision making interventions16. Furthermore, PAG members informed the design of the iFraP randomised controlled trial including choice of outcome measures and piloting questionnaires, although they did not directly inform the methods of economic analysis. Future meetings with the PAG will contribute to the analysis and interpretation of the iFraP trial results. The authors would like to thank all those that supported, funded, and participated in, the iFraP intervention development (iFraP-D) work, including the National Institute for Health and Care Research (NIHR), the Royal Osteoporosis Society, Haywood Foundation, Study Management Group members, stakeholders, and participants. We would also like to thank the public contributors that helped to design the trial as well as the iFraP Trial Steering Committee and Data Monitoring Committee members Professor Neil Gittoes, Professor Alicia O’Cathain, Dr Sara Muller, Professor Lee Shepstone Professor Gretl McHugh, and Dr Kenneth Poole. Faculty Opinions recommendedReferences - 1. NHS: NHS Long Term Plan. 2019. Reference Source - 2. European Commission: Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Commitee of th e Regions Empty on enabling the digital transformation of health and care in the Digital Single Market: empowering citizens and building a healthier society. 2018; (accessed). Reference Source - 3. World Health Organization: Global strategy on digital health 2020-2025. 2021. Reference Source - 4. World Health Organization: Digital health. (accessed). Reference Source - 5. Murray E, Hekler EB, Andersson G, et al.: Evaluating digital health interventions: key questions and approaches. Am J Prev Med. In: Elsevier; 2016; 51(5): 843–51. PubMed Abstract | Publisher Full Text | Free Full Text - 6. Gomes M, Murray E, Raftery J: Economic Evaluation of Digital Health Interventions: Methodological Issues and Recommendations for Practice. PharmacoEconomics. 2022; 40(4): 367–78. PubMed Abstract | Publisher Full Text | Free Full Text - 7. Guo C, Ashrafian H, Ghafur S, et al.: Challenges for the evaluation of digital health solutions—A call for innovative evidence generation approaches. NPJ Digit Med. 2020; 3: 110. PubMed Abstract | Publisher Full Text | Free Full Text - 8. Kolasa K, Kozinski G: How to value digital health interventions? A systematic literature review. Int J Environ Res Public Health. 2020; 17(6): 2119. PubMed Abstract | Publisher Full Text | Free Full Text - 9. Luzi D, Pecoraro F, Tamburis O: Economic evaluation of health IT. Stud Health Technol Inform. 2016; 222: 165–80. PubMed Abstract - 10. McNamee P, Murray E, Kelly MP, et al.: Designing and undertaking a health economics study of digital health interventions. Am J Prev Med. 2016; 51(5): 852–60. PubMed Abstract | Publisher Full Text - 11. Wilkinson T, Wang M, Friedman J, et al.: A Framework for the Economic Evaluation of Digital Health Interventions. Washington, DC: World Bank; 2023. Reference Source - 12. NICE: Shared Decision Making Guidelines. (accessed). Reference Source - 13. Staszewska A, Zaki P, Lee J: Computerized Decision Aids for Shared Decision Making in Serious Illness: Systematic Review. JMIR Med Inform. 2017; 5(4): e36. PubMed Abstract | Publisher Full Text | Free Full Text - 14. NICE: Shared decision making. (accessed). Reference Source - 15. Trenaman L, Bryan S, Bansback N: The cost-effectiveness of patient decision aids: A systematic review. Healthc (Amst). 2014; 2(4): 251–7. PubMed Abstract | Publisher Full Text - 16. Paskins Z, Bullock L, Crawford-Manning F, et al.: Improving uptake of Fracture Prevention drug treatments: a protocol for Development of a consultation intervention (iFraP-D). BMJ Open. 2021; 11(8): e048811. PubMed Abstract | Publisher Full Text | Free Full Text - 17. NOGG: Clinical guideline for the prevention and treatment of osteoporosis. 2021; (accessed). Reference Source - 18. Ayub N, Faraj M, Ghatan S, et al.: The treatment gap in osteoporosis. J Clin Med. 2021; 10(13): 3002. PubMed Abstract | Publisher Full Text | Free Full Text - 19. Ralston KAP, Hauser B, Paskins Z, et al.: Effective Communication and the Osteoporosis Care Gap. J Bone Miner Res. 2022; 37(11): 2049–54. PubMed Abstract | Publisher Full Text - 20. Solomon DH, Johnston SS, Boytsov NN, et al.: Osteoporosis medication use after hip fracture in U.S. patients between 2002 and 2011. J Bone Miner Res. 2014; 29(9): 1929–37. PubMed Abstract | Publisher Full Text | Free Full Text - 21. Paskins Z, Crawford-Manning F, Cottrell E, et al.: Acceptability of bisphosphonates among patients, clinicians and managers: a systematic review and framework synthesis. BMJ Open. 2020; 10(11): e040634. PubMed Abstract | Publisher Full Text | Free Full Text - 22. van Geel TACM, Eisman JA, Geusens PP, et al.: The utility of absolute risk prediction using FRAX® and Garvan Fracture Risk Calculator in daily practice. Maturitas. 2014; 77(2): 174–9. PubMed Abstract | Publisher Full Text - 23. Vrijens B, De Geest S, Hughes DA, et al.: A new taxonomy for describing and defining adherence to medications. Br J Clin Pharmacol. 2012; 73(5): 691–705. PubMed Abstract | Publisher Full Text | Free Full Text - 24. Paskins Z, Torres Roldan VD, Hawarden AW, et al.: Quality and effectiveness of osteoporosis treatment decision aids: a systematic review and environmental scan. Osteoporos Int. 2020; 31(10): 1837–51. PubMed Abstract | Publisher Full Text - 25. Skivington K, Matthews L, Simpson SA, et al.: A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021; 374: n2061. PubMed Abstract | Publisher Full Text | Free Full Text - 26. Crawford-Manning F, Greenall C, Hawarden A, et al.: Evaluation of quality and readability of online patient information on osteoporosis and osteoporosis drug treatment and recommendations for improvement. Osteoporos Int. 2021; 32(8): 1567–1584. PubMed Abstract | Publisher Full Text | Free Full Text - 27. Bullock L, Crawford-Manning F, Cottrell E, et al.: Developing a model Fracture Liaison Service consultation with patients, carers and clinicians: a Delphi survey to inform content of the iFraP complex consultation intervention. Arch Osteoporos. 2021; 16(1): 58. PubMed Abstract | Publisher Full Text | Free Full Text - 28. Bullock L, Manning F, Hawarden A, et al.: Shared decision-making about osteoporosis medicines in Fracture Liaison Services: a qualitative exploration of current practice and anticipated barriers to iFraP intervention use. Archives of Osteoporosis - Submission stage. 2024. - 29. IOF: Osteoporosis in Europe: A Compendium of country-specific reports - United Kingdom. 2021. - 30. Herdman M, Gudex C, Lloyd A, et al.: Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011; 20(10): 1727–36. PubMed Abstract | Publisher Full Text | Free Full Text - 31. Glynn D: Practical Methods to Support Research Prioritisation Decisions: Rapid Assessment of the Need for Evidence. University of York; 2019. Reference Source - 32. Faria R, Gomes M, Epstein D, et al.: A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials. Pharmacoeconomics. 2014; 32(12): 1157–70. PubMed Abstract | Publisher Full Text | Free Full Text - 33. Cro S, Morris TP, Kenward MG, et al.: Sensitivity analysis for clinical trials with missing continuous outcome data using controlled multiple imputation: A practical guide. Stat Med. 2020; 39(21): 2815–42. PubMed Abstract | Publisher Full Text - 34. NICE: Position statement on use of the Position statement on use of the EQ-5D-5L value set for England. 7 September, 2023. - 35. van Hout B, Janssen MF, Feng YS, et al.: Interim scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L value sets. Value Health. 2012; 15(5): 708–15. PubMed Abstract | Publisher Full Text - 36. Alava MH, Pudney S, Wailoo A: Estimating the relationship between EQ-5D-5L and EQ-5D-3L: results from an English Population Study. The University of Sheffield, 2020. Reference Source - 37. Gray LA, Alava MH: A command for fitting mixture regression models for bounded dependent variables using the beta distribution. Stata J. 2018; 18(1): 51–75. Publisher Full Text - 38. Mihaylova B, Briggs A, O'Hagan A, et al.: Review of statistical methods for analysing healthcare resources and costs. Health Econ. 2011; 20(8): 897–916. PubMed Abstract | Publisher Full Text | Free Full Text - 39. Campbell MK, Torgerson DJ: Bootstrapping: estimating confidence intervals for cost-effectiveness ratios. Qjm. 1999; 92(3): 177–82. PubMed Abstract | Publisher Full Text - 40. Barber JA, Thompson SG: Analysis of cost data in randomized trials: an application of the non-parametric bootstrap. Stat Med. 2000; 19(23): 3219–36. PubMed Abstract | Publisher Full Text - 41. NICE: Bisphosphonates for treating osteoporosis - TA464. In: 2019. Reference Source - 42. Gonçalves ASO, Mayer I, Schulz RS, et al.: Protocol for an economic evaluation of a tele-neurologic intervention alongside a stepped wedge randomised controlled trial (NeTKoH). BMC Health Serv Res. 2023; 23(1): 1021. PubMed Abstract | Publisher Full Text | Free Full Text - 43. Krishnan SM, Gc VS, Sandhu HK, et al.: Protocol for an economic analysis of the randomised controlled trial of Improving the Well-being of people with Opioid Treated CHronic pain: I-WOTCH Study. BMJ open. 2020; 10(11): e037243. PubMed Abstract | Publisher Full Text | Free Full Text - 44. Quayyum Z, Briggs A, Robles-Zurita J, et al.: Protocol for an economic evaluation of the randomised controlled trial of culprit lesion only PCI versus immediate multivessel PCI in acute myocardial infarction complicated by cardiogenic shock: CULPRIT-SHOCK trial. BMJ open. 2017; 7(8): e014849. PubMed Abstract | Publisher Full Text | Free Full Text Author details Author details 1 Department of Health Sciences, University of York, York, England, UK 2 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 3 School of Medicine Research User Group, Keele University, Staffordshire, England, UK 4 Centre for Musculoskeletal Health Research, School of Medicine, Keele University, Staffordshire, England, UK 5 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 6 Bristol Medical School, Faculty of Health Sciences, University of Bristol, Bristol, England, UK 7 Cambridge Public Health, University of Cambridge, Cambridge, England, UK 8 Addenbrooke’s Hospital Fracture Liaison Service, Cambridge University Hospitals NHS Trust, Cambridge, England, UK 9 Centre for Epidemiology Versus Arthritis, The University of Manchester, Manchester, England, UK 10 NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Sciences Centre, Manchester, England, UK 11 Midlands Partnership University NHS Foundation Trust, Stoke-on-Trent, England, UK 12 Danish Centre for Healthcare Improvements (CHI), Aalborg University, Aalborg, North Denmark Region, Denmark 2 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 3 School of Medicine Research User Group, Keele University, Staffordshire, England, UK 4 Centre for Musculoskeletal Health Research, School of Medicine, Keele University, Staffordshire, England, UK 5 Keele Clinical Trials Unit, Keele University, Staffordshire, England, UK 6 Bristol Medical School, Faculty of Health Sciences, University of Bristol, Bristol, England, UK 7 Cambridge Public Health, University of Cambridge, Cambridge, England, UK 8 Addenbrooke’s Hospital Fracture Liaison Service, Cambridge University Hospitals NHS Trust, Cambridge, England, UK 9 Centre for Epidemiology Versus Arthritis, The University of Manchester, Manchester, England, UK 10 NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Sciences Centre, Manchester, England, UK 11 Midlands Partnership University NHS Foundation Trust, Stoke-on-Trent, England, UK 12 Danish Centre for Healthcare Improvements (CHI), Aalborg University, Aalborg, North Denmark Region, Denmark Michele Siciliano Roles: Formal Analysis, Writing – Review & Editing Roles: Formal Analysis, Writing – Review & Editing Sarah Bathers Roles: Methodology, Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Ida Bentley Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Laurna Bullock Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Andrea Cherrington Roles: Methodology, Project Administration, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Project Administration, Software, Writing – Original Draft Preparation, Writing – Review & Editing Emma Clark Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Jane Fleming Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Clare Jinks Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Sarah Lewis Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Christian Mallen Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Elaine Nicholls Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Resources, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Resources, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Terence O'Neill Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing Jo Smith Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing David Webb Roles: Investigation, Project Administration, Software, Writing – Review & Editing Roles: Investigation, Project Administration, Software, Writing – Review & Editing Zoe Paskins Roles: Conceptualization, Funding Acquisition, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Cynthia Iglesias-Urrutia Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing Roles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing Competing interests ZP reports grants from the NIHR Clinician Scientist Award (CS-2018-18-ST2-010), Royal Osteoporosis Society and the Haywood Rheumatology Research and Development Foundation to develop the iFraP intervention. ZP has received consultancy fees from UCB Pharma and Keele University has received sponsorship from UCB Pharma. TON is supported by the NIHR Manchester Biomedical Research Centre. CM is funded by the NIHR Applied Research Collaboration West Midlands and the NIHR School for Primary Care Research. CJ is part funded by NIHR Applied Research Collaboration West Midlands Other authors declare no other competing interests Grant information This trial was funded by the National Institute for Health and Care Research (NIHR) [Clinician Scientist Award (CS-2018-18-ST2-010)/NIHR Academy] to Professor Zoe Paskins. The views expressed are those of the author(s) and not necessarily those of the NIHR, NHS or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright © 2024 Siciliano M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article Siciliano M, Bathers S, Bentley I et al. Protocol for a trial-based economic evaluation analysis of a complex digital health intervention including a computerised decision support tool: the iFraP intervention [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:15 (https://doi.org/10.3310/nihropenres.13575.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 1 VERSION 1 PUBLISHED 03 Apr 2024 Views 0 How to cite this report: Neal D. Reviewer Report For: Protocol for a trial-based economic evaluation analysis of a complex digital health intervention including a computerised decision support tool: the iFraP intervention [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:15 (https://doi.org/10.3310/nihropenres.14736.r36795) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-15/v1#referee-response-36795 https://openresearch.nihr.ac.uk/articles/4-15/v1#referee-response-36795 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 23 Aug 2025 Approved with Reservations VIEWS 0 Thank you for inviting me to review this protocol for a trial-based cost-effectiveness evaluation of the iFraP intervention, an example of a complex hybrid digital health intervention. This study addresses important methodological issues around the evaluation of (complex) digital health ... Continue reading 2. Neal D, Kucera M, van Munster B, Ettema T, et al.: Cost-effectiveness of the FindMyApps eHealth intervention vs. digital care as usual: results from a randomised controlled trial. Aging & Mental Health. 2024; 28 (11): 1457-1470 Publisher Full Text 3. Witteman H, Maki K, Vaisson G, Finderup J, et al.: Systematic Development of Patient Decision Aids: An Update from the IPDAS Collaboration. Medical Decision Making. 2021; 41 (7): 736-754 Publisher Full Text 4. Shaw C, Longworth L, Bennett B, McEntee-Richardson L, et al.: A Review of the Use of EQ-5D for Clinical Outcome Assessment in Health Technology Assessment, Regulatory Claims, and Published Literature. The Patient - Patient-Centered Outcomes Research. 2024; 17 (3): 239-249 Publisher Full Text I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close Thank you for inviting me to review this protocol for a trial-based cost-effectiveness evaluation of the iFraP intervention, an example of a complex hybrid digital health intervention. This study addresses important methodological issues around the evaluation of (complex) digital health interventions. Please find my commentary on the paper and suggestions for improvement below:

1) The background provides a clear overview of some of the most important challenges in the evaluation of complex DHI. However, the authors then abruptly move on to an introduction of DST. Could the authors please provide examples to address how previous studies of the effectiveness and cost-effectiveness of DHI have been conducted, in light of the challenges described? 2) Following on from the above, I could draw the authors' attention to the one example with which I am familiar, which is my own protocol for, and results from, an RCT with a complex hybrid digital health intervention, which included a trial-based cost-effectiveness evalution (but not a VoI analysis, therefore not to detract from the authors' claim in the penultimate sentence regarding the novelty of the present paper). Please see Neal et al. 2021 (Ref 1) and Neal et al. 2024 (Ref 2), respectively.

3) "The reasons behind osteoporosis undertreatment are multiple: concerns about long term bisphosphonates efficacy18; safety20; uncertainty about what medicine actually achieves21" Whose concerns are these? Patients or clinicians? 4) In addition to the MRC Framework, has iFraP development followed the IPDAS approach, or some alternative systematic approach? See: Witteman H et al. 2021 (Ref 3) 5) "This is a two sided multicentre individual randomised controlled trial (RCT),..." Does this mean two trial sites or two arms? Could the authors please clarify. 6) "The data collection started on April 2023, and it is planned to end in July 2024, in line with a three-month follow-up duration for the trial" Was the data collection already finished, in July 2023, or July 2024, or is data collection ongoing? Please clarify. If the data collection was completed in July 2023 or 2024, can the authors please address why this protocol is only now being published? 7) The EQ-5D-5L is an important measure for economic evaluations but has many limitations. In the context of the current study, with a three month time horizon, it isn't immediately clear to me how the EQ-5D-5L would be sufficiently sensitive to capture effects of the intervention. EUnetHTA recommends the use of disease-specific (but could be intervention-specific) questionnaires in addition to the EQ-5D-5L. Are the authors able to complement the use of the EQ-5D-5L for this study with additional health outcome measure(s) collected as part of this RCT? If not, please address this substantial limitation of the study in the discussion. See for issues around EQ-5D-5L: Shaw C et al 2024 (Ref 4) 8) I see one mention of sensitivity analysis but it is not clear to me precisely what the authors plan to do in terms of sensitivity analyes. Is it possible to elaborate on this?

9) The first three, if not four, sentences of the discussion feel like they would be better placed in the first part of the methods, in which osteoporosis and iFraP are introduced. 10) "To explore the decision uncertainty associated with the cost-effectiveness of iFraP to enhance bisphosphonate adherence we will conduct a VoI analysis." The purpose of a VoI analysis is to determine the expected value of--in the context of this study--a larger and crucially longer-term study, or a modelling study. Since the authors note in the limitations that they lacked funding at the present time for a modelling study, they should explicitly make the link to the VoI analysis. 11) As noted regarding the choice of health outcome measures, at present the study design does not seem to capture "The short-term impact of the iFraP intervention on osteoporosis medicine uptake, principally bisphosphonates." Rather, it investigates the short-term impact of the iFraP intervention on quality of life of those living with osteoporosis, and is in fact silent on medication uptake per se. If the authors are unable to add additional outcomes to complement the EQ-5D-5L, then please ensure that it is clearly explained in the background, methods and limitations, what this study in fact investigates, and the associated limitations. Overarching comments 12) I spotted a number of minor grammatical errors. I would suggest to give the paper another careful read, with the help of a grammar checker, before finalizing. 13) Please be consistent in terminology: patient-clinician communication is, I believe, generally preferred to clinician-patient. Currently, both are used.

1) The background provides a clear overview of some of the most important challenges in the evaluation of complex DHI. However, the authors then abruptly move on to an introduction of DST. Could the authors please provide examples to address how previous studies of the effectiveness and cost-effectiveness of DHI have been conducted, in light of the challenges described? 2) Following on from the above, I could draw the authors' attention to the one example with which I am familiar, which is my own protocol for, and results from, an RCT with a complex hybrid digital health intervention, which included a trial-based cost-effectiveness evalution (but not a VoI analysis, therefore not to detract from the authors' claim in the penultimate sentence regarding the novelty of the present paper). Please see Neal et al. 2021 (Ref 1) and Neal et al. 2024 (Ref 2), respectively.

3) "The reasons behind osteoporosis undertreatment are multiple: concerns about long term bisphosphonates efficacy18; safety20; uncertainty about what medicine actually achieves21" Whose concerns are these? Patients or clinicians? 4) In addition to the MRC Framework, has iFraP development followed the IPDAS approach, or some alternative systematic approach? See: Witteman H et al. 2021 (Ref 3) 5) "This is a two sided multicentre individual randomised controlled trial (RCT),..." Does this mean two trial sites or two arms? Could the authors please clarify. 6) "The data collection started on April 2023, and it is planned to end in July 2024, in line with a three-month follow-up duration for the trial" Was the data collection already finished, in July 2023, or July 2024, or is data collection ongoing? Please clarify. If the data collection was completed in July 2023 or 2024, can the authors please address why this protocol is only now being published? 7) The EQ-5D-5L is an important measure for economic evaluations but has many limitations. In the context of the current study, with a three month time horizon, it isn't immediately clear to me how the EQ-5D-5L would be sufficiently sensitive to capture effects of the intervention. EUnetHTA recommends the use of disease-specific (but could be intervention-specific) questionnaires in addition to the EQ-5D-5L. Are the authors able to complement the use of the EQ-5D-5L for this study with additional health outcome measure(s) collected as part of this RCT? If not, please address this substantial limitation of the study in the discussion. See for issues around EQ-5D-5L: Shaw C et al 2024 (Ref 4) 8) I see one mention of sensitivity analysis but it is not clear to me precisely what the authors plan to do in terms of sensitivity analyes. Is it possible to elaborate on this?

9) The first three, if not four, sentences of the discussion feel like they would be better placed in the first part of the methods, in which osteoporosis and iFraP are introduced. 10) "To explore the decision uncertainty associated with the cost-effectiveness of iFraP to enhance bisphosphonate adherence we will conduct a VoI analysis." The purpose of a VoI analysis is to determine the expected value of--in the context of this study--a larger and crucially longer-term study, or a modelling study. Since the authors note in the limitations that they lacked funding at the present time for a modelling study, they should explicitly make the link to the VoI analysis. 11) As noted regarding the choice of health outcome measures, at present the study design does not seem to capture "The short-term impact of the iFraP intervention on osteoporosis medicine uptake, principally bisphosphonates." Rather, it investigates the short-term impact of the iFraP intervention on quality of life of those living with osteoporosis, and is in fact silent on medication uptake per se. If the authors are unable to add additional outcomes to complement the EQ-5D-5L, then please ensure that it is clearly explained in the background, methods and limitations, what this study in fact investigates, and the associated limitations. Overarching comments 12) I spotted a number of minor grammatical errors. I would suggest to give the paper another careful read, with the help of a grammar checker, before finalizing. 13) Please be consistent in terminology: patient-clinician communication is, I believe, generally preferred to clinician-patient. Currently, both are used. - Is the rationale for, and objectives of, the study clearly described? Yes - Is the study design appropriate for the research question? Partly - Are sufficient details of the methods provided to allow replication by others? Partly - Are the datasets clearly presented in a useable and accessible format? Not applicable

1. Neal D, Kerkhof Y, Ettema T, Muller M, et al.: Evaluation of FindMyApps: protocol for a randomized controlled trial of the effectiveness and cost-effectiveness of a tablet-based intervention to improve self-management and social participation of community-dwelling people with mild dementia, compared to usual tablet use. BMC Geriatrics. 2021; 21 (1). Publisher Full Text2. Neal D, Kucera M, van Munster B, Ettema T, et al.: Cost-effectiveness of the FindMyApps eHealth intervention vs. digital care as usual: results from a randomised controlled trial. Aging & Mental Health. 2024; 28 (11): 1457-1470 Publisher Full Text 3. Witteman H, Maki K, Vaisson G, Finderup J, et al.: Systematic Development of Patient Decision Aids: An Update from the IPDAS Collaboration. Medical Decision Making. 2021; 41 (7): 736-754 Publisher Full Text 4. Shaw C, Longworth L, Bennett B, McEntee-Richardson L, et al.: A Review of the Use of EQ-5D for Clinical Outcome Assessment in Health Technology Assessment, Regulatory Claims, and Published Literature. The Patient - Patient-Centered Outcomes Research. 2024; 17 (3): 239-249 Publisher Full Text Competing Interests: No competing interests were disclosed. Reviewer Expertise: My research concerns the evaluation of digital health interventions from a person-centred and health-care system perspective. This includes the application of human factors engineering methods for usability research, early health technology assessment, and clinical evaluations. Relevant to this paper, I conducted a trial-based cost-effectiveness evaluation of a complex digital health intervention as part of my PhD research. CITE HOW TO CITE THIS REPORT Neal D. Reviewer Report For: Protocol for a trial-based economic evaluation analysis of a complex digital health intervention including a computerised decision support tool: the iFraP intervention [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:15 (https://doi.org/10.3310/nihropenres.14736.r36795) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/4-15/v1#referee-response-36795 https://openresearch.nihr.ac.uk/articles/4-15/v1#referee-response-36795 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | | |---|---| | 1 | | | Version 1 03 Apr 24 | read | Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...