Co-developing a comprehensive disease policy model with stakeholders: the case of malaria during pregnancy

1 1 Co-developing a comprehensive disease 2 policy model with stakeholders: the case 3 of malaria during pregnancy 4 Authors: Fernandes, Silke; Briggs Andrew; Hanson, Kara 5 Abbreviations: 6 CEA=Cost effectiveness analysis 7 DALY=disability-adjusted life years 8 Total word count: 3200 words 9 10 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice. 2 1 Abstract 2 Introduction: Understanding the holistic impact of malaria during pregnancy is essential for 3 improving maternal and child outcomes in malaria endemic settings. To be able to design 4 appropriate research and conduct robust policy analyses, a comprehensive model of the 5 underlying disease, representing the current understanding of mechanisms and consequences 6 is essential. This study aimed to illustrate a methodology to co-develop a disease model with 7 expert stakeholders using malaria during pregnancy as a case study. 8 Methods: An initial steering group was convened to develop a first model of malaria during 9 pregnancy and its consequences for mother and child based on their understanding of the 10 literature. Subsequently, this model was refined using a Delphi process to gain consensus 11 amongst twelve experts, representing the disciplines of health economics, mathematical 12 modelling, epidemiology and clinical medicine, working in the field of malaria during pregnancy. 13 Experts reviewed drafts of the conceptual model and provided feedback in two rounds of semi- 14 structured questionnaires with the aim of identifying the most important health outcomes and 15 relationships in both mother and child as well as the most relevant stratifiers for the model. 16 Consensus on any final disagreement was reached after two consensus meetings. 17 Results: The final model is a comprehensive disease model of malaria during pregnancy, 18 including ten maternal and ten child outcomes with four stratifiers. The model developed in 19 this study should be of value to malaria researchers, funders, evaluators and decision makers, 20 though some adaptation will be required for each specific context and purpose. In addition, the 21 methodology and process followed in this study is replicable and can guide researchers aiming 22 to develop a conceptual model for other conditions. 23 Discussion & Conclusion: The model resulting from this study highlights the complexity 24 required to depict appropriately the consequences of malaria during pregnancy for both the 25 mother and the child. It also demonstrates how to conduct a rigorous process to develop a 26 disease model. In addition the study has helped to identify a number of areas with scarce data 27 and need for further research. 28 Funding: This study from part of the IMPROVE and IMPROVE-2 studies, which received financial 29 support from the EDCTP2 programme under Horizon 2020 (TRIA.2015-1076, TRIA.2015-1076b); 30 the UK Department of Health and Social Care, the UK Foreign Commonwealth and 31 Development Office, the UK Medical Research Council, and Wellcome Trust, through the Joint 32 Global Health Trials scheme (MR/P006922/1); and the Swedish International Development 33 Cooperation Agency. 34 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 3 1 Introduction 2 Most diseases work via complex biological processes that create observable interrelated health 3 outcomes. Understanding these relationships is essential for many types of research and the 4 embedded policy analyses that are informed by that research. For example, trials of 5 comparative effectiveness and associated cost-effectiveness analysis would benefit from an 6 understanding of all the relevant outcomes, and the interconnections among them, both in 7 designing the most appropriate research study and in analysing the results of that study to 8 understand which treatment options are most appropriate in a given context. 9 10 Cost effectiveness analyses (CEA) are required by many global and national bodies to inform 11 policy change. Some health interventions have multiple health effects (for example, both 12 morbidity and mortality); and some have effects on more than one population group. To 13 capture such disparate benefits in a way that facilitates comparison between alternative uses of 14 scare resources, health benefits of interventions are translated into a common metric such as a 15 Quality Adjusted Life Year (QALY) or Disability Adjusted Life Year (DALY). 16 17 Complexity is increased when a disease such as malaria is combined with pregnancy, as the 18 disease and its treatments now have the potential to impact both mother and child. In this 19 paper, we use malaria in pregnancy as a case study to illustrate how a structured approach to 20 co-developing a disease model with relevant stakeholders can result in a more robust model, 21 that will carry greater influence with the scientific community because of the multi-disciplinary 22 input into its development. 23 24 In 2012 a taskforce recommended the development of a conceptual model as the foundation 25 for developing an economic model (1). A conceptual model entails a systematic approach to 26 provide a visual framework for analysis that shows how specific health outcomes and pathways 27 relate and interact with each other (1). Documented approaches to the development of 28 conceptual frameworks include literature reviews, consultation with stakeholders (qualitative 29 and quantitative), methods of incorporating stakeholder views and piloting to refine the 30 framework (2, 3). 31 32 Economic evaluations of interventions can be complex and require contributions from a broad 33 range of disciplines. Models based on a particular viewpoint can lead to poor validity and 34 credibility. A review of outcomes included in published economic models of malaria in . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 4 1 pregnancy interventions reveals a disparate picture of outcomes included in DALY estimation 2 (4-12). Only four out of nine CEAs incorporated clinical malaria, maternal anaemia and low 3 birth weight (5, 6, 9, 11), which are commonly measured in clinical trials, in the DALY 4 estimation. One CEA did not include any child health outcome (8) and another no maternal 5 health outcome (4). 6 The Delphi consultation method is a well-established and tested approach used in research to 7 elicit information from experts and has been used extensively in the social sciences (13-16). 8 It is a particularly suitable method to incorporate a range of stakeholder views, leading to 9 improved quality and acceptance of an economic evaluation model and its findings(17) (1, 18, 10 19) by seeking consensus amongst experts, avoiding the pitfall of only including outcomes and 11 relationships measured in trials. 12 13 The aim of this study was to co-develop a conceptual model of prevention of P.falciparum 14 malaria during pregnancy for pregnant women and their babies using a Delphi consensus study 15 with experts in the field of malaria during pregnancy. The expert panel’s task was to identify 16 the most important health outcomes and relationships in both mother and child and the most 17 relevant stratifiers for the model. In doing so, the study demonstrates that co-production of 18 holistic disease models with expert stakeholders, representing the current understanding of a 19 disease and potential treatment pathways, is feasible and represents a more robust approach 20 than ad hoc model construction by individual academic teams. 21 22 Methods 23 This study used the Delphi methodology to co-develop a policy model of malaria during 24 pregnancy with expert stakeholders. The expert panel in the Delphi methodology consists of 25 people with relevant insight into the subject to be explored and can include technical experts, 26 health providers, policy makers, patients or other suitable panellists. It is a very useful 27 technique to gather input from various stakeholders in a time-efficient manner through a series 28 of questionnaires. Responses from each round are collated, analysed and incorporated into the 29 subsequent rounds of questions until consensus between the panellists has been reached, 30 usually after two-to-three rounds, which is often followed by a final consensus meeting with 31 stakeholders to resolve any final points (15, 16, 20). The experts remain anonymous in the 32 process up until the final meeting if applicable, which promotes equal contribution . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 5 1 independent of status and other factors and removes less favourable forces of group dynamics 2 (15, 16). The 3 4 The different stages and methods in this study are illustrated in figure 1 and summarized 5 below. Full details of the approach are described in the supplementary materials. Experts in 6 this study were first approached on 31 August 2022, all experts had consented by 5th of 7 October 2022. The final consensus meeting took place on 8th September 2023. 8 9 Stage one: preparation 10 During an initial preparation stage, a steering group with collective experience in health 11 economics, conceptual modelling and epidemiology of malaria in pregnancy was convened. Its 12 task was to short-list experts to be approached to be part of a Delphi panel as well as to advise 13 on the preparation of a first draft of the conceptual model and questionnaire based on their 14 understanding of the literature, ongoing research and natural history of malaria in pregnancy. 15 Potential candidates for the Delphi panel were purposively selected for their varied expertise, 16 knowledge of the literature and current research in malaria during pregnancy and approached 17 by email. The authors paid particular attention to having a well-balanced panel with experts 18 representing both maternal and child health, early and later exposure to malaria during 19 pregnancy and various endemicity contexts. The study team aimed to include eight to ten 20 experts in the Delphi panel, a group size shown to be effective and reliable for the Delphi 21 method (15, 16). Experts received no incentive or financial reimbursement for their time 22 participating in this study. 23 24 Stage two: Delphi consultation 25 Twelve expert agreed to take part in the Delphi study and provided written informed consent 26 (online). During the Delphi consultation stage, they were asked to refine the draft model using 27 an a priori undetermined number of rounds of consultations until consensus in most questions 28 was reached. The threshold for consensus for individual questions was set at 70%, consistent 29 with previous Delphi studies (18, 21). In each round, the panel members were provided with a 30 current draft of the model and were asked to perform the following tasks: 31 1) Assess importance of the outcomes in the model, 32 2) suggest additional outcomes that were missing, 33 3) evaluate the accuracy of relationships between outcomes, 34 4) suggest additional relationships between outcomes that were missing, . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 6 1 5) order stratifiers for subgroup analysis by importance (a stratifier is defined as a 2 variable which can partition the population in the model into subpopulations, e.g. by gravidity 3 or HIV status of the mother), 4 6) suggest any additional stratifiers that were missing, and 5 7) provide their opinion on additional aspects of the presentation of the model, e.g. the 6 visual presentation of low birth weight with its sub-categories (prematurity, intrauterine 7 growth restriction and small for gestational age). 8 9 Nominal and ordinal categorical response options as well as free-text questions were used in 10 both questionnaires with round two containing more of the latter (See appendix 2 and 3 for 11 questionnaires used in round 1 and 2). The responses to each round were analysed by one 12 researcher (SF) and incorporated into the next model draft and questionnaire. Categorical 13 questions were analysed using simple descriptive statistics. Free text responses were explored 14 using a simple thematic analysis, coding them manually into themes (22). After each round 15 panellists received a summary report of the analysis (See appendix 4 and 5), ensuring 16 anonymity was maintained. 17 18 Stage three: Consensus meeting 19 In the final stage of the study, two online consensus meetings for experts in different time 20 zones were held to present the findings of the second Delphi round and to discuss and vote on 21 any remaining aspects of the model where consensus had not been reached during stage two. 22 The conceptual model was finalized by the first author following the consensus meeting. 23 24 Ethics 25 Ethics approval for this study was received on 12 July 2022 by the Research Ethics Committee of 26 the London School of Hygiene and Tropical Medicine (Reference number 27361). Informed 27 written consent was received from all Delphi panel members. 28 29 Role of the funding source 30 This study forms part of the PhD of SF supervised by KH, with an aim to be used in the cost- 31 effectiveness analysis of the IMPROVE (TRIA.2015-1076) and IMPROVE-2 (TRIA.2015-1076b) 32 trials. SF was funded on both IMPROVE trials to conduct the cost-effectiveness analysis into 33 which this conceptual model will feed. 34 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 7 1 The study received financial support from the EDCTP2 programme under Horizon 2020 2 (TRIA.2015-1076, TRIA.2015-1076b); the UK Department of Health and Social Care, the UK 3 Foreign Commonwealth and Development Office, the UK Medical Research Council, and 4 Wellcome Trust, through the Joint Global Health Trials scheme (MR/P006922/1); and the 5 Swedish International Development Cooperation Agency. 6 7 Results 8 Key results of the different stages of this study are summarized here (see supplementary 9 materials for additional detail). 10 11 Stage one: Preparation 12 The first draft of the conceptual model developed by the steering group included outcomes for 13 the mother and the child and relationships among them (Figure 2). Gravidity, timing of 14 exposure to P.falciparum (i.e. first, second or third trimester) and HIV status were selected as 15 the most relevant stratifiers for subpopulation analysis. The steering group identified 17 16 experts to be approached to participate in the study, of whom twelve agreed (71%). Amongst 17 eleven experts the average years of experience working in malaria in pregnancy was 17.9 years 18 (range 8-34) and the twelfth expert had over 15 years of experience in the economics of 19 malaria. 20 21 Stage two: Delphi consultation 22 Two rounds of consultation were required before sufficient consensus was reached. Changes 23 to the model made after each consultation round and the consultation meetings are illustrated 24 in table 1 with only the most significant changes highlighted here. 25 26 After the first consultation round all outcomes included in figure 2 remained in the model. On 27 recommendation of panel members "severe disease” and “serious complications” were 28 combined into a single outcome of “severe malaria” as experts pointed out the difficulty in 29 differentiating between these two outcomes. All experts agreed that “low birth weight” should 30 be separated into “intrauterine growth restriction” and “preterm birth”, with five experts 31 suggesting the addition of “small for gestational age”. Additional outcomes - all maternal - to 32 be incorporated into the next draft of the model were “asymptomatic parasitaemia”, “placental 33 malaria” and “hypertension disorders of pregnancy”. Responses on rating stratifiers (gravidity, . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 8 1 HIV status, timing of exposure) needed further exploration with transmission intensity 2 suggested as an additional stratifier by seven experts. 3 4 In round two experts reached consensus regarding the inclusion of asymptomatic parasitaemia 5 (100%, 12/12 agreed) and placental malaria (75%, 9/12 agreed) and their associated 6 relationships. Whether to include or exclude “hypertension disorder of pregnancy” was unclear 7 and had to be explored further during the consensus meeting. While all relationships associated 8 with “hypertension disorder of pregnancy” were judged to be correct, it appeared that there 9 was a difference in opinion regarding its importance and relevance amongst experts working in 10 low- versus high endemicity settings. 11 12 Experts were asked to vote for the two most important stratifiers for subpopulation analysis 13 leading to the following ranking from most to least important with the number of votes in 14 brackets: gravidity (10), transmission intensity (8), timing of exposure of P.falciparum (3) and 15 HIV status (2). Summary reports of both Delphi consultation round analyses can be found in 16 appendix 4 (round 1) and 5 (round 2) and intermediate model drafts after round 1 and 2 are 17 depicted in figure S1 and S2 in appendix 1. 18 19 Stage three: consensus meeting 20 All twelve experts completed both rounds of questionnaires and nine (75%) attended one of 21 the consensus meetings, held on 31st August and 8th of September 2023. The most relevant 22 topic discussed was “hypertension disorders of pregnancy” and its potential sequelae. All 23 attending experts agreed to keep “hypertension disorder of pregnancy” in the model without 24 splitting it further into “hypertension”, “pre-eclampsia” and “eclampsia”. However, they voted 25 to add “long-term effects of hypertension disorders of pregnancy” as a further outcome to 26 include long-term sequelae such as stroke or mental health disorders. Other less contentious 27 issues such as the relationship between “clinical malaria” and “anaemia” or relationships and 28 labelling of child morbidities were also agreed during the consensus meeting. 29 30 Experts expressed the importance of adapting economic models to context and allowing the 31 flexibility for them to evolve over time as more granular data become available. They also felt 32 that in addition to developing a conceptual model of malaria during pregnancy to be used in 33 future cost-effectiveness analysis, the work had helped to identify a number of areas where 34 data are limited and that it will be important to share these with the research community. The . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 9 1 final model is shown in figure 3, in which both child and maternal figures are combined, a 2 suggestion made during the consensus meeting. 3 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 10 1 Discussion 2 Summary 3 This article presents a consensus-building study using the Delphi methodology with the goal of 4 co-developing a conceptual model of malaria during pregnancy with a diverse group of expert 5 stakeholders. The result is a comprehensive disease policy model including ten maternal and 6 ten child outcomes with four stratifiers. To our knowledge, it is the first formal attempt to co- 7 develop a disease model of this kind either in the field of malaria or in a disease area 8 predominantly prevalent in low and middle income countries. 9 10 The study has highlighted the complexity of the model required to depict appropriately the 11 consequences of malaria during pregnancy to mothers and their offspring. Key contributors to 12 the success of the study were the selection of the expert panel, thorough preparation of each 13 stage as well as well as careful analysis and weighing-up of all responses. It was essential to be 14 accurate with language, which sometimes had to evolve over various stages, while remaining 15 accessible to a wide range of readers. 16 17 The process not only helped to develop the model to include relevant outcomes and 18 relationships, but also improved the visual presentation and accessibility of the model, for 19 example by adding symbols for the different timings of outcomes or appearance of arrows. 20 21 Strength and limitations 22 This study has a number of strengths and limitations. The literature search conducted by the 23 first author during the preparation stage was not a systematic review. Therefore some potential 24 outcomes and relationships may have been missed out of the first draft of the model, however 25 this was mitigated by the experts’ responses during the consultation rounds and consensus 26 meetings. Experts for the Delphi panel were purposively selected to balance the experience, 27 origin and focus area of work of panel members, however, the study may suffer from bias by 28 omitting other experts with differing views. 29 30 The acceptance rate of experts was high (71%) with a 100% retention during the two 31 consultation rounds; and 75% of panellists attended one of the two consensus meetings. The 32 use of the Delphi methodology preserved the anonymity of experts and allowed panellists to 33 respond freely without being influenced by other opinions or dominant personalities. The final 34 stage of the study using online consultation meetings was more susceptible to the effects of . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 11 1 group dynamics, however, this did not appear to be a problem with all experts engaging equally 2 and respectfully with each other in both meetings. 3 4 This study focused on P. falciparum malaria. To apply the model to other plasmodium species 5 such as P.vivax, malariae or ovale it would be necessary to review the model and consider 6 inclusion of additional outcomes, relationships and stratifiers, informed by a literature search 7 and expert consultation. 8 9 It may be a challenge to populate this comprehensive model for a cost-effectiveness study 10 because of the range of outcomes and complexity of relationships. Nevertheless, this study has 11 brought together experts from different fields and contexts to develop a model all could agree 12 to. 13 14 Areas for future research 15 During the study a number of areas requiring further research or development have emerged. 16 The most commonly used outcome in cost-effectiveness analysis of global health interventions, 17 the DALY, is a composite outcome combining mortality and morbidity, and in the case of 18 malaria during pregnancy can combine both maternal and child outcomes into one measure. 19 However, not all outcomes lend themselves equally well to calculating reliable DALY estimates 20 and all of them rely heavily on assumptions made in the Global Burden of Disease studies (23). 21 For example, estimating DALYs arising from “Long-term neurological and other sequelae” could 22 potentially be difficult as long-term follow up data are lacking from malaria trials, requiring 23 assumptions. Likewise, not all manifestations of severe malaria are equally associated with 24 mortality or long-term morbidities, ultimately affecting the DALY. Placental malaria and 25 asymptomatic parasitaemia were included in the model after the first consultation round, 26 because for the experts, in particular epidemiologists, it is important to have these 27 intermediate and often reported outcomes represented in the model for completeness and to 28 depict important pathways. 29 30 Experts expressed very differing views regarding the inclusion of hypertension disorders of 31 pregnancy, mostly shaped by different levels of awareness. The votes as well as the comments 32 provided in the Delphi consultation indicated that experts working in lower endemicity settings 33 were more aware of the link between malaria during pregnancy and hypertension disorder 34 during pregnancy. The consensus meetings provided a useful platform to discuss these . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 12 1 differences and the supporting evidence. Evidence from both Asia (24) and Africa points to an 2 association, with a meta-analysis including four case-control studies from Africa suggesting that 3 women experiencing malaria during pregnancy had 2.7 times higher odds of developing 4 gestational hypertension disorder compared with those who did not (25). Ideally, the model 5 should also differentiate hypertension disorder of pregnancy further into pre-eclampsia and 6 eclampsia, but experts agreed that this level of detail will be difficult to find in the currently 7 available data, but should be sought in the future. Experts commented that more effort should 8 be made to collect data on hypertension disorder in pregnancy as blood pressure is always 9 measured but often omitted from clinical trial databases. As above for “long-term neurological 10 and other sequalae” the estimation of DALYs arising from hypertension disorder of pregnancy 11 will require some assumptions on incidence and disability weights. 12 13 Some of the morbidities and outcomes can have lifelong consequences and be progressive. For 14 example “neurocognitive and physical development impairment in <5” will likely impact the 15 child for its entire life and influence its educational achievement and productivity. Likewise, for 16 a women with severe malaria who develops severe anaemia and requires a blood transfusion 17 there is a risk that the blood supply is contaminated, which then increases the risk of a wide 18 range of other morbidities associated with other infectious diseases. While it will not be 19 possible to quantify these future consequences for a cost-effectiveness analysis with currently 20 available data, it is certainly important to create awareness of the potential long term health 21 problems. 22 23 This study identified four important stratifiers: gravidity, transmission intensity, timing of 24 exposure and HIV status. However, this does not preclude other variables from being important 25 in certain analyses. Examples could be the sickle cell trait or the gender of the baby. At this 26 point in time insufficient data are available to differentiate the consequences of the timing of 27 exposure (e.g. first versus second, third trimester). HIV status also requires further 28 disaggregation of the data such as the CD4 count or whether the woman is receiving 29 antiretrovirals. Currently, cost-effectiveness models of chemoprevention for malaria during 30 pregnancy will naturally stratify by HIV status as different prevention interventions are given to 31 HIV negative and positive women. 32 33 Currently, the model does not include potential treatment or prevention interventions to 34 ensure it is widely applicable for different purposes. Depending on the type of intervention . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 13 1 study and context, further outcomes, such as side-effects, or effects on other diseases, such as 2 HIV transmission from mother to child in HIV positive women, might need to be incorporated 3 into the model. Studies using QALYs might also need to include patient’s health perception, 4 perceived quality of life and future outlook into the model. 5 6 Finally, an important outcome of this study is to identify areas where data are scarce and share 7 these with the research community, to raise awareness of the need for comparable outcome 8 measures reported by trials. 9 10 A number of these points highlight the urgent need for more granular data, and experts felt 11 that despite the complexity of the model it was important to create awareness of the wide 12 range of outcomes that can be prevented by preventing pregnant women from being exposed 13 to P. falciparum. More detailed data in the future should allow a move away from one size fits 14 all models to models that are more adaptable and fluid. 15 16 Conclusions 17 This study has demonstrated a more inclusive approach to developing disease policy models 18 that are capable of assisting in the design of clinical trials (and other policy evaluations) and 19 their associated health economic analysis. In so doing, we believe that this integrated 20 approach should become the gold-standard for disease modelling designed to inform health 21 policy in different countries and contexts. Co-development ensures wider perspectives are 22 incorporated into the model than is usually possible for a single academic team, which should 23 ensure the resulting model is more robust and fit for purpose. A robust conceptual modelling 24 co-design approach will also help identify data gaps, ensuring these are not overlooked as the 25 modelling proceeds to the implementation phase. 26 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 14 1 Contributors 2 All authors conceived the idea for the study. SF designed the study with input from KH and AB. 3 SF conducted the Delphi consultations & meetings and subsequent data analysis. KH verified the 4 data analysis. SF wrote the first draft of the manuscript. KH and AB critically reviewed and edited 5 several drafts of the manuscript. All authors had full access to all the data in the study and had 6 final responsibility for the decision to submit for publication. 7 Declaration of interests 8 We declare no competing interests. 9 Acknowledgement: 10 11 The authors would like to take this opportunity to acknowledge and thank the members of the 12 Delphi Panel and Steering group for giving up their precious time so selflessly to share their 13 invaluable wisdom with the study team. 14 15 Delphi panel members (in alphabetical order): Prof. Grant Dorsey (University of California San 16 Francisco, United States of America), Prof. Kevin Kain (University of Toronto, UHN-Toronto 17 General Hospital, Canada), Dr. Kassoum Kayentao (Malaria Research and Training Center- 18 University of Sciences, Techniques, and Technologies of Bamako, Mali), Prof. Feiko ter Kuile 19 (Liverpool School of Tropical Medicine, United Kingdom), Prof. Rose McGready (Shoklo Malaria 20 Research Unit, Thailand), Dr. George Mtove (National Institute for Medical Research, Tanzania), 21 Dr. Catherine Pitt (London School of Hygiene and Tropical Medicine, United Kingdom), Prof. 22 Stephen Rogerson (University of Melbourne, Australia), Dr. Makoto Saito (WorldWide 23 Antimalarial Resistance Network), Dr. Steve Taylor (Duke University, United States of America), 24 Prof. Halidou Tinto (Institut de Recherche en Sciences de la Santé, Clinical Research Unit of 25 Nanoro, Burkina Faso) and Dr. Holger Unger (Menzies School of Health Research, Darwin, 26 Australia). 27 28 Steering group members: Prof. Kara Hanson (London School of Hygiene and Tropical Medicine), 29 Prof. Andy Briggs (London School of Hygiene and Tropical Medicine), Prof. Feiko ter Kuile 30 (Liverpool School of Tropical Medicine), Silke Fernandes (London School of Hygiene and 31 Tropical Medicine) 32 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 15 1 Overview of figures, tables and appendices 2 Figures in main paper: 3  Figure 1: Methodology used in the study 4  Figure 2: Conceptual model: Draft 1 – steering group 5  Figure 3: Final model after Delphi consultation meetings 6 Figures in appendix 1: 7  Figure S1: Conceptual model: Draft 2- Delphi consultation post round 1 8  Figure S2: Conceptual model: Draft 3- Delphi consultation post round 2 9 Table: 10  Table 1: Changes to the model made after each consultation round and consultation 11 meetings 12 Appendices: 13  Appendix 1: Detailed methods and results 14  Appendix 2: Questionnaire Delphi consultation round 1 15  Appendix 3: Questionnaire Delphi consultation round 2 16  Appendix 4: Summary report Delphi consultation round 1 17  Appendix 5: Summary report Delphi consultation round 2 18 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 16 1 Table 1: Changes to the model made after each consultation round and consultation meetings Changes made after Delphi consultation round 1 Changes made after Delphi consultation round 2 Changes made after Delphi consultation meetings (final model) "MOTHER Exposure to P.falciparum while pregnant" changed to "MOTHER presence of P.falciparum while pregnant" Addition of "asymptomatic parasitaemia" "With and without Placental malaria" added to "clinical malaria" and "asymptomatic parasitaemia" "Long term neurological sequelae" changed to "long term sequelae" to include other long-term disabilities resulting from manifestations of severe malaria "Long term sequelae" changed to long term neurological and other sequelae" green colour (for long-term consequence to the mother) added to "long-term neurological and other sequelae" "Death in utero after maternal death" added to "miscarriage/stillbirth" "miscarriage/stillbirth/death in utero after maternal death" changed to "miscarriage/stillbirth/death in utero" "Hypertensions disorders of pregnancy" added Outcome "hypertension disorders of pregnancy" relabelled to "hypertension disorders of pregnancy and post-partum", footnote added: "includes pre-eclampsia, eclampsia and gestational hypertension" Additional outcome "Long-term effects of hypertension disorders of pregnancy" added as a consequene of "Hypertension disorders or preganncy and post-partum" Outcomes "Severe disease" and "serious complications" combined into "severe malaria" and WHO definition added Relationships to and from "hypertension disorders of pregnancy" added Relationship from "Hypertension disorders of pregancy and post-partum" to "Long-term effects of hypertension disorders of pregnancy" added Mother Relationships Relationship from maternal death to "death in utero after maternal death" added . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 17 Relationship to and from "asymptomatic parasitaemia" added Relationship from maternal anaemia to "severe malaria" added Relationship between maternal anaemia and clinical malaria made bi-directional Bidirectional relationship between "clinical malaria" and maternal anaemia" reversed to unidirectional (malaria to anaemia) and an arrow from "maternal anaemia" to "clinical malaria" indicating "contributes to progression" Arrow from "maternal anaemia" to "clinical malaria" indicating "contributes to progression" removed "Small for gestational age" added and visualization of "low birth weight" changed "short-, mid- and long-term morbidities" changed to "neonatal, infant, <5 and older child/adult morbidities" "Neonatal, infant, <5 and older child/adult morbidities" changed to "Other morbidities in neonates, infants, <5, older children and adults" Shape of "Other morbidities in neonates, infants, <5, older children and adults" changed from large arrow to an oval shape as other outcomes "CHILD with in utero exposure to P.falciparum" changed to "CHILD Presence of P.falciparum in utero" "CHILD Presence of P.falciparum in utero" relabelled to "CHILD Presence of/ exposure to P.falciparum in utero" "Neonatal, infant and <5 mortality" combined into 1 large outcome box and relabelled as "Neonatal, infant, <5 mortality & mortality in older children & adults" green colour added to box "Neonatal, infant, <5 mortality & mortality in older children & adults" "Modified incidence of malaria in <5" changed to "Increased incidence of malaria in <5" Outcomes "Neurocognitive development impairment in <5" relabelled to "Neurocognitive &physical development impairment in <5" Child Relationship Arrow from "neonatal, infant, <5 and older child/adult morbidities" to "neonatal, infant and <5 mortality" added Starting position of arrow from "neonatal, infant, <5 and older child/adult morbidities" to "neonatal, infant and <5 mortality" changed from back of the box to the middle . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 18 Arrows from 1) "CHILD Presence of/exposure to P.falciparum in utero", 2)"Fetal anaemia", 3) "Congenital malaria" to "Other morbidities in neonates, infants, <5, older children and adults" added Symbols in addition of colour code for the timing of health effects added Red on timeline changed from "neonate/ 28 days postpartum" to "neonate/mother 28/42 days postpartum" to reflect the postpartum period in which maternal death are counted description box moved below model Design Design of certain arrows and lines changed to help with distinguising them Design of arrows and lines changed again as previous change was confusing to expert Other Stratifiers "Transmission intensity" added as a stratifier to the next round in addition to "HIV status","Gravidity" and "Timing of exposure" Out of the four stratifiers "Transmission intensity", "HIV status","Gravidity" and "Timing of exposure", Transmission intensity and gravidity were considered most important by experts. Description box listing potential other stratifiers added below the model . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 19 1 Figures 2 Figure 1: Methodology used in the study 3 Figure 1 illustrates the methodology used in this Delphi consultation study, which can be split 4 into three stages: preparation, Delphi consultation and consensus meeting. 5 . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 20 1 Figure 2: Conceptual model: Draft 1- steering group 2 2a) Maternal outcomes 3 4 2b) Child outcomes . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 21 1 2 Figure 2 shows the maternal and child outcomes included in the first draft of the conceptual 3 model developed by the four members of the steering group to be used as a starting point for 4 the first round of the Delphi consultation. Outcomes were divided into maternal and child 5 outcomes. Outcomes affecting morbidity are shown at the top, while mortality outcomes at the 6 bottom. The colour coding of the outcomes represents the different timings of the health 7 effects. 8 Abbreviations: IUGR=Intrauterine growth restriction . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 22 Figure 3- Final model after Delphi consultation meetings Figure 3: The final agreed conceptual model of malaria during pregnancy following two consensus meetings attended by nine experts. The model combines both maternal and child outcomes. Outcomes were divided into maternal and child outcomes. Outcomes affecting morbidity are shown at . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 23 the top, while mortality outcomes at the bottom. The colour and shape coding of the outcomes represents the different timings of the health effects. Abbreviations: w/=with; w/o=without; WHO=World Health Organization . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 24 References 1. Roberts M, Russell LB, Paltiel AD, Chambers M, McEwan P, Krahn M, Force I- SMGRPT. Conceptualizing a model: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--2. Value Health. 2012;15(6):804-11. 2. McMeekin NJM. Conceptual models in health economic evaluation: a new role.: University of Glasgow; 2021. 3. Cruz Rivera S, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: A systematic review of methodological frameworks. PLoS Med. 2017;14(8):e1002370. 4. Becker-Dreps SI, Biddle AK, Pettifor A, Musuamba G, Imbie DN, Meshnick S, Behets F. Cost-effectiveness of adding bed net distribution for malaria prevention to antenatal services in Kinshasa, Democratic Republic of the Congo. Am J Trop Med Hyg. 2009;81(3):496-502. 5. Fernandes S, Sicuri E, Halimatou D, Akazili J, Boiang K, Chandramohan D, et al. Cost effectiveness of intermittent screening followed by treatment versus intermittent preventive treatment during pregnancy in West Africa: analysis and modelling of results from a non-inferiority trial. Malaria journal. 2016;15(1):493. 6. Fernandes S, Sicuri E, Kayentao K, van Eijk AM, Hill J, Webster J, et al. Cost- effectiveness of two versus three or more doses of intermittent preventive treatment for malaria during pregnancy in sub-Saharan Africa: a modelling study of meta-analysis and cost data. The Lancet Global health. 2015;3(3):e143-53. 7. Sicuri E, Bardaji A, Nhampossa T, Maixenchs M, Nhacolo A, Nhalungo D, et al. Cost-effectiveness of intermittent preventive treatment of malaria in pregnancy in southern Mozambique. PloS one. 2010;5(10):e13407. 8. Sicuri E, Fernandes S, Macete E, Gonzalez R, Mombo-Ngoma G, Massougbodgi A, et al. Economic evaluation of an alternative drug to sulfadoxine-pyrimethamine as intermittent preventive treatment of malaria in pregnancy. PloS one. 2015;10(4):e0125072. 9. Choi SE, Brandeau ML, Bendavid E. Cost-effectiveness of malaria preventive treatment for HIV-infected pregnant women in sub-Saharan Africa. Malaria journal. 2017;16(1):403. 10. Hansen KS, Ndyomugyenyi R, Magnussen P, Clarke SE. Cost-effectiveness analysis of three health interventions to prevent malaria in pregnancy in an area of low transmission in Uganda. Int Health. 2012;4(1):38-46. 11. Fernandes S, Were V, Gutman J, Dorsey G, Kakuru A, Desai M, et al. Cost- effectiveness of intermittent preventive treatment with dihydroartemisinin-piperaquine for malaria during pregnancy: an analysis using efficacy results from Uganda and Kenya, and pooled data. The Lancet Global health. 2020;8(12):e1512-e23. 12. Paintain L, Hill J, Ahmed R, Umbu Reku Landuwulang C, Ansariadi A, Rini Poespoprodjo J, et al. Cost-effectiveness of intermittent preventive treatment with dihydroartemisinin-piperaquine versus single screening and treatment for the control of malaria in pregnancy in Papua, Indonesia: a provider perspective analysis from a cluster-randomised trial. The Lancet Global health. 2020;8(12):e1524-e33. 13. Earp JA, Ennett ST. Conceptual models for health education research and practice. Health Educ Res. 1991;6(2):163-71. 14. Meyrick J. The Delphi method and health research. Health Education. 2003. 15. Dalkey NC. The Delphi Method: An Experimental Study of Group Opinion. Santa Monica, CA: RAND Corporation; 1969. . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint 25 16. Jones J, Hunter D. Qualitative Research: Consensus methods for medical and health services research. BMJ. 1995;311(7001):376. 17. Squires H, Chilcott J, Akehurst R, Burr J, Kelly MP. A Framework for Developing the Structure of Public Health Economic Models. Value Health. 2016;19(5):588-601. 18. Tabberer M, Gonzalez-McQuire S, Muellerova H, Briggs AH, Rutten-van Molken M, Chambers M, Lomas DA. Development of a Conceptual Model of Disease Progression for Use in Economic Modeling of Chronic Obstructive Pulmonary Disease. Medical decision making : an international journal of the Society for Medical Decision Making. 2017;37(4):440-52. 19. Tappenden P. Conceptual Modelling For Health Economic Model Development. 2012. 20. Linstone HA, Turoff M. Introduction. In The Delphi method: Techniques and applications. 2002. 21. Woodcock T, Adeleke Y, Goeschel C, Pronovost P, Dixon-Woods M. A modified Delphi study to identify the features of high quality measurement plans for healthcare improvement projects. BMC Med Res Methodol. 2020;20(1):8. 22. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3:77-101. 23. Das P, Samarasekera U. The story of GBD 2010: a "super-human" effort. Lancet. 2012;380(9859):2067-70. 24. Harrington WE, Moore KA, Min AM, Gilder ME, Tun NW, Paw MK, et al. Falciparum but not vivax malaria increases the risk of hypertensive disorders of pregnancy in women followed prospectively from the first trimester. BMC Med. 2021;19(1):98. 25. Mruma HA, McQuillan R, Norrie J. The association of malaria infection and gestational hypertension in Africa: Systematic review and meta-analysis. J Glob Health. 2020;10(2):020417. . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint . CC-BY 4.0 International licenseIt is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint The copyright holder for thisthis version posted September 12, 2024. ; https://doi.org/10.1101/2024.09.10.24313438doi: medRxiv preprint