Identifying and Selecting Priority Setting Criteria for an Exploratory Multi-Criteria Decision Analysis Study for Health Benefit Package Design in Kenya

Identifying and Selecting Priority Setting Criteria for an Exploratory Multi-Criteria Decision Analysis Study for Health Benefit Package Design in Kenya | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Identifying and Selecting Priority Setting Criteria for an Exploratory Multi-Criteria Decision Analysis Study for Health Benefit Package Design in Kenya Melvin Obadha, Edwine Barasa This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6630410/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background The paper focuses on the process of identifying and selecting priority setting criteria for use in an exploratory quantitative multi-criteria decision analysis study for health benefit package design in Kenya. Methods To identify and select criteria, a four-stage approach was adapted i.e., raw data collection, data reduction, removing inappropriate criteria, and wording. Results In stages one and two (raw data collection and reduction), 10 criteria that had been identified by the health benefit package advisory panel, when defining the universal health coverage essential benefit package, were used as a starting point. Stage three involved reducing the long list of 10 criteria to a shorter one using researchers’ judgement while taking into account multiple factors. Three researchers commented on the 10 criteria and removed inappropriate ones e.g., splitting cost-effectiveness criterion into two “cost of intervention” and “effectiveness of intervention”. In stage four, the resulting criteria and levels were further refined using semi-structured interviews with 10 stakeholders, and a pilot discrete choice modelling survey with 24 stakeholders. Results and feedback from the semi-structured interviews and the pilot discrete choice modelling survey were used to develop a final list of six criteria and levels i.e., burden of disease, congruence with existing priorities, cost of intervention, effectiveness of intervention, equity, and health systems capacity. Conclusion The study, which is part of a larger exploratory multi-criteria decision analysis exercise, provided insights into the priority setting criteria Kenyan stakeholders felt were important in health benefit package design. Africa Discrete choice Kenya MCDA Multiple Criteria Decision Analysis Priority setting Figures Figure 1 Figure 2 Figure 3 BACKGROUND In expanding the range of services in the pursuance of universal health coverage (UHC), countries need to use explicit evidence-based priority setting criteria. The World Health Organisation (WHO) consultative group on equity and UHC recommends using at least these three criteria when prioritising health interventions i.e., cost-effectiveness, financial risk protection, and priority for the worse off (equity) [ 1 , 2 ]. A scoping review on criteria used in defining health benefit packages determined cost-effectiveness as frequently used, followed by effectiveness, budget impact, equity, and burden of disease [ 3 ]. These were the top five criteria among many others revealed by the review such as necessity, safety, and feasibility among others [ 3 ]. In low-and middle income countries (LMICs), a systematic review determined cost-effectiveness criterion as frequently used in priority setting, followed by health benefits, and equity [ 4 ]. Cost-effectiveness has been used by a number of countries defining UHC essential packages for health services such as Ethiopia, Pakistan, Somalia, Sudan, and Zanzibar [ 5 ]. Financial risk protection has been used by Afghanistan, Ethiopia, Pakistan, and Zanzibar. Equity has also been used by Afghanistan, Ethiopia, Pakistan, and Zanzibar [ 5 ]. Nonetheless, other criteria that have been used by countries defining UHC essential packages for health services include budget impact, disease burden, effectiveness, feasibility, integrated service delivery, public and political acceptability, quality of evidence, and socio-economic impact [ 5 ]. In Kenya, priority setting practices have been identified as ad hoc, fragmented, and lacked use of explicit criteria [ 6 ]. However, there were a few initiatives that did incorporate explicit criteria use. In 2018, the Cabinet secretary for health appointed a panel of experts called the health benefits package advisory panel (HBPAP) [ 7 , 8 ]. The panel had the mandate of defining a UHC essential benefit package (UHC-EBP) in 60 days [ 8 , 9 ]. The package was to be funded through public sources, piloted in four counties, and scaled up to the rest of the counties [ 9 ]. The UHC-EBP process incorporated the use of four criteria i.e., affordability, cost-effectiveness, effectiveness, and feasibility [ 9 ]. The criteria were used individually (separately) rather than simultaneously to allow trade-offs between criteria [ 9 ]. However, this package was never adopted [ 8 ]. The current Kenyan government is still committed to achieving UHC and has facilitated the enactment of three relevant pieces of legislation into law i.e., the facility improvement financing act 2023, the primary health care act 2023, and the social health insurance act 2023 [ 10 – 13 ]. The primary health care act 2023 and the facility improvement financing act 2023 aim to improve primary health care and health facility financing [ 10 , 11 , 13 ]. The social health insurance act 2023 converted Kenya’s national health insurance fund (NHIF) into a social health insurance fund (SHIF) that is mandatory for everyone, including those in the informal sector, and managed by a social health authority (SHA) [ 12 – 15 ]. SHA regulations have created a new health benefit advisory panel and legislated a range of priority setting criteria. The new Panel has been selected and is in the process of being operationalised. This calls for the need for systematic priority setting methods that incorporate the use of multiple criteria simultaneously while allowing for trade-offs [ 16 ]. There are many techniques used in health intervention priority setting such as accountability for reasonableness (A4R) [ 17 , 18 ], burden of disease analysis [ 19 ], cost-effectiveness analysis (CEA) [ 20 ], health technology assessment (HTA) [ 21 ], multi-criteria decision analysis (MCDA) [ 16 ], and programme budgeting and marginal analysis (PBMA) [ 22 , 23 ] among others [ 24 ]. MCDA stands out as being capable of incorporating multiple criteria simultaneously in setting health priorities and capturing trade-offs being made [ 16 , 25 ]. Furthermore, MCDA allows involvement of stakeholders which makes the priority setting process transparent and validates it [ 16 , 25 ]. MCDA has been used to set priorities for health interventions in countries such as Colombia [ 26 ], Ghana [ 27 ], Kazakhstan [ 28 ], Italy [ 29 ], Norway [ 30 ], and Thailand [ 31 ] among others. Against this backdrop, we explored the use of quantitative MCDA in Kenya using the example of the country’s UHC benefit package. The aim was to prioritise health interventions into the package. Quantitative MCDA has several steps that entail stating the decision goal or objective (e.g., prioritise vaccines for coverage), mapping out stakeholders (e.g., policymakers, academics etc), identifying and selecting alternatives to be considered (e.g., vaccines, drugs etc), identifying and selecting decision making criteria, measuring performance, scoring alternatives on criteria, weighting criteria, aggregation, uncertainty analysis, and results reporting and interpretation [ 32 – 36 ]. This paper focuses on the process used to identify and select priority setting criteria for use in the exploratory quantitative MCDA study in Kenya using a four-step process i.e., raw data collection, data reduction, removing inappropriate criteria, and wording [ 37 ]. Every step of the MCDA process must be done well and reported to ensure quality decisions are made, bias reduced, and uncertainty is addressed or described, according to recommendations of ISPOR taskforce guidelines [ 34 , 35 ]. Furthermore, ensuring stakeholders participate (especially patients and the public) in the different stages of the MCDA exercise, makes the process transparent and validates it as earlier stated [ 16 , 25 ]. These aspects are sometimes not well reported in literature especially identification and selection of health interventions and priority setting criteria [ 34 , 35 ]. Moreover, stakeholders like patients and the public are left out of the exercise [ 38 ]. This study addresses these gaps and contributes to the body of knowledge by reporting how priority setting criteria for an exploratory quantitative MCDA study in Kenya were identified and selected and includes the participation of stakeholders (e.g., patients) in the exercise. METHODS Study setting and population . The study setting was at national level in Kenya and used the country’s UHC benefit package as an example. The aim was to prioritise health interventions into the package, but this paper focuses on the process of identifying and selecting criteria for prioritisation. The study population were stakeholders involved in health intervention priority setting identified using the “Ps” framework by Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, Conway PH, Daudelin D, Morrato EH and Leslie LK [ 39 ]. These stakeholder groups were purchasers, policymakers, providers, principal investigators (academics and researchers), patients, and the public [ 39 ]. Stakeholders at national and subnational levels were targeted. Framework for identifying and selecting priority setting criteria. To identify and select criteria, a four-stage approach by Helter and Boehler (see Fig. 1 ) was adapted i.e., raw data collection, data reduction, removing inappropriate criteria, and wording [ 37 , 40 ]. The framework was selected as it provides a systematic method of identifying and selecting criteria and allows multiple approaches to be incorporated, while involving stakeholders. Stakeholder involvement at different stages of the MCDA process is important as it provides transparency and validity to the process [ 38 ]. We had also previously used the framework in the Kenyan setting to develop a discrete choice modelling survey that explored preferences for capitation payments in the country [ 40 ]. Therefore, the framework was fit for the local context. Raw data collection consists of collecting data using qualitative and quantitative methods [ 37 , 40 ]. Data reduction involves reducing collected data to a long list of criteria by analysing it qualitatively and/or quantitatively [ 37 , 40 ]. Removing inappropriate criteria involves reducing the long list of criteria to a smaller list through considering factors such as capability of being traded, completeness, non-overlap, non-redundancy, plausibility, preference independence, and salience [ 34 , 35 , 37 , 40 ]. Wording refers to refining criteria using techniques such as piloting and researcher’s judgement among others [ 37 , 40 ]. Stages one and two: raw data collection and data reduction. Criteria that had been identified by HBPAP when defining the UHC-EBP were used as a starting point. The 10 criteria were affordability, burden of disease, catastrophic health expenditure, congruence with existing priorities, cost effectiveness, effectiveness and safety, equity, feasibility (health workforce requirements), feasibility (service and health products & medical technology requirements), and severity of disease (see Additional file 1) [ 8 , 9 ]. In identifying and selecting criteria for defining the UHC-EBP, HBPAP obtained the criteria and definitions from a study by Tromp N and Baltussen R [ 41 ]. The panel then conducted a modified nominal group technique (NGT) with stakeholders to refine the long criteria list and reach consensus on 10 [ 8 , 9 ]. This study used HBPAP’s 10 criteria as the starting point as the aim was to build on existing and past priority setting processes. This would enable the study to explore the feasibility of using MCDA for health intervention priority setting in Kenya and provide lessons that would easily be acceptable to and adopted by local stakeholders. Starting with an existing set of criteria meant that the raw data collection and data reduction stages had been conducted, as the criteria were obtained from HBPAP’s list who had in turn obtained them from Tromp N and Baltussen R [ 41 ]. Stage three: removing inappropriate criteria. Stage three involved reducing the long list of 10 criteria to a shorter one. Researchers’ judgement was used where three researchers provided comments based on their expertise in HTA, MCDA, priority-setting, and discrete choice modelling [ 37 , 40 ]. The target was to refine and reduce the criteria to approximately five or six and a maximum of four levels per criteria (minimum two levels) [ 40 , 42 ]. This was because discrete choice modelling was one of the methods that would be used in scoring and weighting in the quantitative MCDA process. A systematic review had found that discrete choice modelling studies that elicited preferences for health interventions used a mean of 5.74 attributes and 3.26 levels per attribute [ 43 ]. Increasing the number of criteria complicates the choice tasks and leads to a greater cognitive burden on respondents [ 40 , 42 , 44 – 50 ]. Several factors are usually considered when removing inappropriate criteria. In this stage, the researchers considered factors like availability of data, capability of being traded, completeness, correlation between criteria, decision context, non-overlap, non-redundancy, plausibility, preference independence, relevance to study objective, and salience [ 34 , 35 , 37 , 40 , 50 ]. The researchers provided comments on the criteria, domains, lay, and technical definitions. They also defined probable levels for the criteria to be used in the discrete choice modelling survey. The resulting criteria, definitions, and probable levels were to be refined by stakeholders using semi-structured interviews and pilot tested using discrete choice modelling in the next stage. Stage four: wording In this stage, the resulting criteria and levels were further refined using semi-structured interviews and a pilot discrete choice modelling survey with stakeholders. Key informant semi-structured Interviews Semi-structured interviews were selected as they offered flexibility in exploring the themes under consideration therefore enabling the stakeholders to offer detailed information [ 51 , 52 ]. Furthermore, it would make getting busy stakeholders easier as it meant they could be interviewed at their place of work and on different days. Other methods such as focus group discussions (FGDs), meetings, or nominal group technique (NGT) would have required all stakeholders to be present at a specified date and time. The informants for the semi-structured interviews were Kenyan-based stakeholders with knowledge of UHC, HTA, and priority setting. Therefore, stakeholders who had participated in HTA and priority setting initiatives in Kenya were mainly targeted. Additionally, stakeholders conducting health research, those providing healthcare, and those involved in health policymaking were also targeted. The targeted stakeholders were policymakers (national and county-level), principal investigators (academics and researchers), providers (faith-based/non-governmental organisations NGO, private, and public sectors), and purchasers (ministry of health, county department of health, and health insurance organisations) [ 39 ]. They were contacted through emails and phone calls or approached-in person at their places of work. Additionally, snow balling was used where busy stakeholders identified others who were subsequently approached using emails, phone calls, or in-person at their place of work. Overall, 42 stakeholders were approached, with 11 of them agreeing to participate. A total of 10 stakeholders were interviewed at their places of work or convenient venues, after signing informed consent forms (ICFs). The interviews were conducted in person between June 2022 and July 2022. The interviews lasted between 45 minutes and 90 minutes. The interview guide was developed using the priority setting criteria list (see additional file 2) and other aspects being considered in the broader quantitative MCDA study such as health interventions [ 53 ]. The guide was piloted with four researchers who had experience in priority-setting. The guide had three sections. The first section covered socio-demographic characteristics, respondents’ involvement in HTA and priority setting (e.g., benefit package design or essential medicines lists development), and questions about UHC pilot in four Kenyan countries. In the second section, questions focused on a list of health interventions under consideration in the study. In the third section, stakeholders were provided with a list of six criteria, their levels, and lay and technical definitions to comment on. Finally, they were asked general questions about the barriers and enablers of institutionalisation of HTA in Kenya, and what other countries could learn from the Kenyan experience. The focus, in this publication, will be the third section of the interview guide where stakeholders provided feedback on the six criteria, definitions, and levels. Analysis of the data employed a framework approach because it is systematic, reproducible, and transparent [ 54 , 55 ]. Interviews were transcribed verbatim by a research company. One researcher (MO) reviewed the transcripts and made corrections by listening to the recordings. MO then familiarised with the data and developed an initial framework using the priority-setting criteria, health interventions, and interview guide. The resulting framework was reviewed by another researcher. MO applied the framework to the data (indexing and sorting). The rest of the steps involved reviewing data extracts, data summary and display, and abstraction and interpretation [ 55 , 56 ]. NVIVO Release 1 was used to manage the data [ 57 ]. Finally, stakeholder sociodemographic characteristics were analysed quantitatively and presented using measures of central tendency (means and medians), measures of dispersion (interquartile range and standard deviation), and proportions. R version 4.2.2 was used in the analysis [ 58 ]. Discrete choice modelling pilot The pilot survey was conducted for two purposes. First, discrete choice modelling had been selected as a method to be used in the scoring and weighting phase of the quantitative MCDA in Kenya. Therefore, the pilot would generate priors for the main discrete choice modelling survey’s experimental design [ 40 , 59 ]. The second reason was to experiment with the criteria and levels. Discrete choice modelling was selected for the exploratory quantitative MCDA study as it was grounded in established theories in fields such as economics and psychology e.g., decision field theory (DFT) [ 60 – 64 ], random regret minimisation (RRM) [ 65 – 67 ], and random utility maximisation (RUM) [ 68 – 70 ]. Models that follow RUM were used in this study as they are commonly used in quantitative MCDA studies that employ discrete choice modelling [ 27 , 30 , 71 – 76 ]. Six criteria and levels were used for the pilot study (see additional file 3). Three criteria had three levels while the other three had two levels each. An unlabelled discrete choice experiment that did not have an opt out was adopted. The experiment had two hypothetical interventions i.e., Intervention A and B. A forced choice scenario was adopted because the assumption was that in real life, stakeholders involved in priority setting for benefit package design do not opt out from appraising interventions before them. Similar studies that used discrete choice modelling in MCDA for health intervention priority setting in LMICs adopted unlabelled designs that had two hypothetical alternatives without an opt-out. These studies were conducted in Côte d'Ivoire [ 77 ], Ghana [ 27 , 71 ], Nepal [ 72 ], and Thailand [ 73 ]. A D-efficient experimental design was used, optimising for a main effects multinomial logit model (MNL) using Ngene software version 1.3 [ 78 ]. Priors used were educated best guesses, with values closer to zero (see Additional file 4) [ 59 ]. All criteria were categorical incorporating dummy coding [ 79 ]. A total of 12 choice tasks were generated, which seemed optimal, based on previous studies in Kenya and Uganda among community health workers and health facility managers [ 40 , 80 – 82 ]. Five versions of the questionnaire (see Additional file 5) were generated where the order of the 12 choice tasks were randomised. The first version comprised of the original order generated by the experimental design. The remaining four versions each had the order of the 12 choice tasks randomised. The questionnaires had two practice choice tasks where the second choice task consisted of a dominant alternative (see Fig. 2 ) meant to gauge stakeholders’ understanding of answering the choice tasks [ 80 ]. Stakeholder sampling was purposive and targeted five stakeholder groups “Ps” at national and county level. These stakeholders were patient advocacy groups, principal investigators (academics and researchers), providers (faith-based/NGO, private, and public), public, and purchasers and policy makers [ 39 ]. Snowballing sampling technique was additionally used to reach eligible stakeholders. Eligible stakeholders were approached directly at their place of work or contacted in advance through emails or phone calls, and a convenient date and time set. Other stakeholders were called to a central venue to complete the survey questionnaire. They were invited through emails, phone calls, and snowballing. Computer assisted personal interviewing (CAPI) was employed for the pilot survey. In CAPI, the researcher or study participant uses a device such as a tablet, mobile phone, or laptop to complete the survey questionnaire in person [ 83 ]. The discrete choice modelling survey was self-administered (in person) using a tablet or smart phone in the presence of a researcher. Data were collected and managed using REDCap tool [ 84 , 85 ]. The researcher explained the aims of the study and provided an ICF to the stakeholder to complete (on a tablet/smart phone and a paper version). The paper version of the ICF was provided to test whether leaving a paper copy with the stakeholder was better for reference. Stakeholders could also enter their email to receive an electronic version of the ICF. Once stakeholders signed both the paper ICF and the tablet/smartphone version, they completed the first section of the questionnaire on REDcap using a tablet or smartphone. Then, the researcher explained the choice scenario in the second section. Here, stakeholders were prompted to assume that the government wanted to define a UHC benefit package, and they needed to select interventions to be included in the package. The criteria and levels were explained, and stakeholders were taken through two practice choice tasks. The second-choice task included a dominant alternative to check whether the stakeholder had understood the task [ 80 ]. Pseudo randomisation was used to allocate one of the five versions of the 12 choice tasks to stakeholders. Stakeholders were able to verbalise their thought process or justify their answers during the think aloud exercise [ 37 , 80 , 86 , 87 ]. Overall, 84 participants were approached either directly or through snowballing. Only 25 signed the ICF and started the survey, with 24 completing the choice tasks. The response rate was 29.76%. One respondent did not complete the choice tasks as they found the exercise cognitively burdensome. The survey took between 30 and 45 minutes to complete. The survey was administered at the stakeholder’s place of work. For those that were administered at places away from the workstation, stakeholders were paid out of duty station allowance of Kenya shillings (KES) 1000 (US $ 7.50) and transport costs were reimbursed depending on where they travelled from. In data analysis, a main effects MNL model was used to model the choice probabilities. In the utility function, categorical attributes were dummy coded to estimate non-linear effects [ 42 ]. Alternative specific constant was used and set at alternative A to check for the presence of left to right bias [ 42 , 88 ]. Analysis was conducted using Apollo choice modelling package version 0.2.9 on R version 4.3.3 [ 58 , 89 ]. Relative importance estimates were also computed (see additional file 6) on Apollo and the delta method was used to generate corresponding robust standard errors and 95% confidence intervals [ 40 , 80 , 89 – 91 ]. The dataset used in the analysis including the Rscript for the MNL analysis are available open access [ 92 ]. Final list of criteria and levels Results and feedback from the semi-structured interviews and the pilot discrete choice modelling survey were used to develop the final list of six criteria and levels. This section was done by one researcher (MO). RESULTS Stage three: removing inappropriate criteria. Cost-effectiveness criterion was split into two criteria, cost of intervention, and effectiveness of intervention. The resulting extra effectiveness criterion was dropped as it was redundant. The cost of intervention criterion was omitted from the list of criteria to be considered for inclusion in the scoring and weighting stages of the MCDA. Baltussen R, Marsh K, Thokala P, Diaby V, Castro H, Cleemput I, Garau M, Iskrov G, Olyaeemanesh A, Mirelman A, Mobinizadeh M, Morton A, Tringali M, van Til J, Valentim J, Wagner M, Youngkong S, Zah V, Toll A, Jansen M, Bijlmakers L, Oortwijn W and Broekhuizen H [ 25 ] argue that costs or cost-effectiveness criterion should not be included in the value function as stakeholders would not be able to know financial restrictions in priority setting of health interventions. Therefore, with lack of this knowledge, stakeholders would not be able to make adequate decisions on the cost or cost effectiveness criterion [ 25 ]. Nonetheless, data on cost of intervention would still be obtained and combined with the MCDA results at the end, to enable ranking of health interventions [ 25 ]. Other criteria were also considered. Affordability criterion was dropped as it was considered redundant in the presence of the cost of intervention criterion. feasibility (service, health products and technology requirements) and feasibility (health workforce requirements) were merged, and the new criterion was named health systems capacity requirements. Burden of disease and severity of disease criteria were considered as overlapping. Therefore, severity of disease was selected as it gave priority to the worse off according to the WHO consultative group on equity and UHC’s recommendation of taking into account the worse off when expanding service coverage [ 1 , 2 ]. Those with severe forms of illness are considered as having greater health needs [ 1 , 2 ]. The levels for the criteria were obtained from multiple sources such as the evidence synthesis done by HBPAP when defining the UHC-EBP [ 9 ] and literature on MCDA applications in priority setting in LMICs that used discrete choice modelling [ 72 , 77 ]. Stage four: wording Key informant semi-structured Interviews Stakeholders had diverse characteristics. Stakeholders had a median age of 38.50 years (interquartile range IQR 8.00 years), a majority came from medical backgrounds, had master’s degrees, and had a median overall work experience of 12.50 years (IQR 6.75 years) (see Table 1 ). Most importantly, 70.00% had been part of a committee or group tasked with the development or revision of a benefit package. Table 1 Characteristics of semi structured interview respondents Characteristic Value N Gender Male 60.00% 6 Female 40.00% 4 10 Age (Years) Mean (standard deviation) 41.90 (7.84) 10 Median (IQR) 38.50 (8.00) 10 Education Completed Doctorate degree 10.00% 1 Completed master’s degree 80.00% 8 Completed bachelor’s degree 10.00% 1 10 Profession Medical doctors 60.00% 6 Public Health Officers 10.00% 1 Nurses 10.00% 1 Epidemiologists 10.00% 1 Community Health professionals 10.00% 1 10 Stakeholder group Policymakers & purchasers 40.00% 4 Principal investigators 20.00% 2 Providers 20.00% 2 Others 20.00% 2 10 Overall work experience (Years) Mean (standard deviation) 16.60 (8.11) 10 Median (IQR) 12.50 (6.75) 10 Stakeholder's current or previous involvement in health benefits package development before No 30.00% 3 Yes 70.00% 7 10 Stakeholders gave feedback on the criteria, levels, and definitions (see additional file 7). Effectiveness and safety were viewed as two criteria. Some stakeholders noted that interventions could be effective but not safe. Therefore, they suggested if the criterion could be split into two “effectiveness” and “safety”. The levels of the “health systems capacity requirements” criterion were not easily understood without the researcher expounding. For example, the “Above/Below average” naming of the levels were viewed as ambiguous as stakeholders noted they needed a reference value/average value. Therefore, stakeholders suggested if the levels could be renamed to “has capacity/does not have capacity” or “country has capacity/country does not have capacity”. Stakeholders were also prompted to choose one criterion between burden and severity of disease. Almost all of them preferred burden of disease over severity criterion. The main reason was that they prioritised conditions/diseases that affected many people arguing that interventions targeting a higher disease burden would have greater impact at population level. Discrete choice modelling pilot The stakeholders who completed the survey included six patient advocates, five principal investigators (academics and researchers), and 12 providers (see Table 2 ). The median age was 36.00 years (IQR 7 years), three-quarters were female, and 37.50% had completed a bachelor’s degree. Overall, the stakeholders had a median work experience of 9.00 years (IQR 8.25 years) and 20.83% had participated in HTA processes such as the development of essential packages for health, health benefit packages, or essential medicines lists. Table 2 Sociodemographic characteristics of DCE pilot stakeholders Characteristic Value N Gender Female 75.00% 18 Male 25.00% 6 24 Age (Years) Mean (SD) 37.00 (7.17) 24 Median (IQR) 36.00 (7) 24 Stakeholder group Patients 25.00% 6 Principal investigators 20.83% 5 Providers 54.17% 13 24 Education Completed Doctorate degree 4.17% 1 Completed master’s degree 20.83% 5 Completed bachelor’s degree 37.50% 9 Completed Diploma/Higher Diploma 33.33% 8 Completed Certificate 4.17% 1 24 Profession Medical doctor 16.67% 4 Nurse 20.83% 5 Clinical officer 20.83% 5 Health economist 8.33% 2 Epidemiologist 4.17% 1 Other 29.17% 7 24 Type of organisation the stakeholder works for County department of health 4.35% 1 Faith-based/NGO health facility 8.70% 2 Private health facility 8.70% 2 Public health facility 34.78% 8 Research institution 21.74% 5 Patient advocacy group 21.74% 5 23 Overall work experience (Years) Mean (SD) 10.49 (7.19) 24 Median (IQR) 9.00 (8.25) 24 Stakeholder’s current or previous involvement in HTA e.g., health benefits package development before No 79.17% 19 Yes 20.83% 5 24 IQR: Interquartile range. N: number of observations. SD: Standard deviation. All but one of the coefficients of the criteria had expected signs (see Table 3 ). Stakeholders significantly preferred interventions that were of greater effectiveness and were safer, reduced the financial burden of paying out of pocket, addressed diseases that mainly affected the poor, targeted severe diseases, and were more congruent with existing priorities. Table 3 Main effects MNL model Criteria Coefficient 95% CI P-value Effectiveness and safety of intervention Slightly effective and safe Ref. (0) Moderately effective and safe 0.60 0.33, 1.13 0.027 Highly effective and safe 0.86 0.61, 1.34 0.001 Catastrophic health expenditure Does not reduce the financial burden of paying out of pocket Ref. (0) Reduces the financial burden of paying out of pocket 1.22 1.01, 1.64 < 0.001 Health systems capacity requirements Below national average Ref. (0) Above national average -0.21 -0.49, 0.34 0.457 Equity Disease mainly affects the well off Ref. (0) Disease mainly affects the poor 0.83 0.60, 1.28 < 0.001 Severity of disease Mild Ref. (0) Moderate 0.19 -0.08, 0.72 0.474 Severe 0.63 0.40, 1.08 0.006 Congruence with existing priorities Low Priority Ref. (0) Medium Priority 0.68 0.35, 1.33 0.039 High Priority 0.86 0.62, 1.34 < 0.001 Alternative specific constant (Alternative A) -0.05 -0.22, 0.28 0.755 Model fit statistics Log likelihood (final) -136.57 Rho-square (C) 0.32 Adj. Rho-square (C) 0.27 Akaike Information Criterion 293.14 Bayesian Information Criterion 329.77 Number of modelled outcomes 288.00 Number of decision makers (n) 24.00 Adj: Adjusted. CI: Confidence Interval. P-value are two sided. Interestingly, the sign of the health systems capacity requirements criterion was negative, indicating that even though the capacity of implementing the intervention was below national average, stakeholders would still prefer the intervention over one that had capacity above national average. However, this was not statistically significant. The alternative specific constant was set at alternative A. The sign was negative and not statistically significant which signified absence of left to right bias. The relative importance estimates were computed. Catastrophic health expenditure was the most important criterion followed by congruence with existing priorities (see Fig. 3 and additional file 6). Health systems capacity requirements was the least important criteria. There were some interesting comments in the think-aloud exercise. Stakeholders noted that the catastrophic health expenditure levels “reduces/does not reduce the financial burden of paying out of pocket” seemed similar or related to the equity criterion levels “disease mainly affects well-off/poor”. Furthermore, another respondent felt that the levels of the catastrophic health expenditure criterion went against the main tenets of UHC of ensuring financial risk protection. They felt that all interventions being considered for inclusion should reduce out-of-pocket expenses for the user. Therefore, the levels of that criterion did not make sense. Stakeholders also gave comments on the effectiveness and safety, health systems capacity requirements, congruency with existing priorities, and equity criteria. Some noted that effectiveness and safety were bundled together, and the safety part of the level was not changing, as an intervention could be highly effective but not safe. Stakeholders also found it difficult to understand the levels of the health systems capacity requirements criterion “Above/below national average”. They would have liked to know the value of the national average. Therefore, they preferred using “above capacity” or “has capacity” or “capacity to provide service” categorisation rather than “above/below national average”. Furthermore, the word "requirements" in the criterion was confusing and could be interpreted as capacity required in future rather than capacity available at present for the intervention to be successfully rolled out. Nonetheless, congruence with existing priorities criterion was well understood and the levels “high priority”, “medium priority”, and “low priority” could easily be interpreted. Lastly, the equity criterion was well understood according to feedback from stakeholders. However, others noted that conditions or diseases could affect both the poor and the well-off. Therefore, the levels would need rephrasing. Some stakeholders suggested if the researchers could try and measure equity in access for the intervention using other metrics, rather than dichotomising using socio-economic status (poor/rich) and prevalence. Stakeholders gave feedback on the general survey design. They found 12 choice tasks easily manageable. One respondent noted if they could have reference to the criteria definitions while completing the choice tasks, and if this could be provided in form of a paper sheet. The definitions of the criteria were on the previous page. Therefore, they could not go back due to the randomisation algorithm. This was evident as some had to seek clarification on the definitions of some of the criteria from the researcher while answering the choice tasks. Final list of criteria and levels The final list of six criteria levels (see Table 4 ) were developed using results and feedback from the semi-structured interviews and the pilot discrete choice modelling survey. Severity of disease criterion was dropped, and burden of disease included in the final list according to the feedback from the qualitative study. The two criteria could not all be included as they overlapped. The levels of burden of disease were named “high burden”, “medium burden”, and “low burden”. Congruence with existing priorities was retained in the final list as stakeholders noted it was salient. It was the second most important criterion among the stakeholders who participated in the pilot discrete choice modelling survey (see Fig. 3 ). Furthermore, safety was split from the effectiveness and safety criterion according to feedback from the semi-structured interviews and pilot discrete choice modelling survey. The new criteria were called “effectiveness of intervention” and “safety of intervention”. To limit the number of criteria to five or six as earlier explained, safety of intervention criterion was dropped from the study. Though cost of intervention criterion was included in the final list, it will not be used in scoring and weighting stages of the quantitative MCDA as explained earlier. It will be used later to calculate cost per value metric to aid in ranking health interventions in a league table [ 25 ]. Table 4 Final attributes, levels, and lay definitions. Criteria Levels Lay definition Burden of disease 0. Low burden 1. Medium burden 2. High burden “Whether the service addresses a condition/disease that affects many Kenyans.” [ 9 , p.13]. Congruence with existing priorities 0. Low priority 1. Medium priority 2. High priority “Whether the service is in line with constitution, prevailing laws and prevailing health sector policies and priorities as further investments and policies are made.” [ 9 , p.13] Effectiveness of intervention 0. Slightly effective 1. Moderately effective 2. Highly effective “Whether the service delivers an improvement in health status, reduction in mortality or improvement in quality of life.” [ 9 , p.13] Equity 0. Condition/disease mainly affects the well-off 1. Condition/disease mainly affects the poor “Whether the service addresses the disparities in access and utilisation of needed health services and health status of Kenyans.” [ 9 , p.13] Health systems capacity 0. Country has inadequate capacity 1. Country has adequate capacity “Whether the service can be provided to Kenyans based on existing health system capacity in terms of human resources, medicines, supplies, and other service provision requirements.” [ 9 , p.13] Cost of intervention Criterion will not be included in the value function (scoring and weighting stage) of the quantitative MCDA. It will be considered at the end of the MCDA where a cost per value metric will be calculated. Unit cost of health intervention Criteria and definitions were derived from HBPAP’s report [ 9 ]. HBPAP had obtained the criteria and definitions from Tromp N and Baltussen R [ 41 ]. Adjustments were made to other criteria as well. Though catastrophic health expenditure criterion was the most important according to the pilot discrete choice modelling results (see Fig. 3 ), it was dropped from the final list because respondents viewed it as a cost attribute. Furthermore, cost is used in the calculation of catastrophic health expenses. There was already a cost of intervention criterion in the list, and it would overlap with the catastrophic health expenditure criterion. Moreover, including catastrophic health expenses as a criterion to be used in scoring and weighting of health interventions, would be like including cost in the value function [ 25 , 36 ]. Additionally, the word “requirements” was dropped from the health systems capacity requirements criterion according to feedback from the qualitative study and pilot discrete choice modelling survey. The levels of this criterion were renamed to “country has adequate capacity” and “country has inadequate capacity”. Nonetheless, equity criterion was maintained in the final list as it represented a key aspect of UHC that called for priority for the worse-off [ 1 ]. Finally, all criteria were categorical, and levels were made qualitative. The advantage was that they could be mapped to any set of health interventions with different performance cut-offs. This meant that the future quantitative MCDA results could be used in contexts beyond Kenya, or with any set of health interventions. The disadvantage of qualitative levels was they could be open to misinterpretation if not well understood [ 93 ]. Therefore, to minimise bias in the future steps of the study, making sure that stakeholders clearly understand the criteria and level would be crucial. DISCUSSION The study set out to identify and select criteria to be used in an exploratory quantitative MCDA study for health intervention priority setting in Kenya employing the country’s UHC benefit package as an example. A four-stage process was used to accomplish the objective that involved raw data collection, data reduction, removing inappropriate criteria, and wording [ 37 , 40 ]. A list of 10 criteria were selected from the criteria identified by HBPAP and used as the starting point for this study [ 9 , 41 ]. Researchers (MO, and two other academics) commented on the criteria to remove inappropriate ones and refined them, while taking into consideration factors such as availability of data, capability of being traded, completeness, non-overlap, non-redundancy, plausibility, preference independence, and salience [ 34 , 35 , 37 , 40 , 50 ]. Semi structured interviews and a pilot discrete choice modelling survey with stakeholders were used to further refine the criteria, levels, and definitions. Finally, six criteria and levels were agreed upon i.e., burden of disease, congruence with existing priorities, cost of intervention, effectiveness of intervention, equity, and health systems capacity [ 94 ]. However, cost of intervention will not be used in the scoring and weighting stage of the quantitative MCDA [ 25 ]. It will be incorporated in the final stages to create a cost per value metric to aid in ranking health interventions [ 25 ]. The final criteria selected (burden of disease, congruence with existing priorities, cost of intervention, effectiveness of intervention, equity, and health systems capacity) were similar to those used in other studies and settings globally [ 94 ]. Burden of disease, effectiveness of intervention, equity, and cost of intervention are some of the criteria advocated for in prioritising national health plans, policies, and strategies by a WHO report [ 95 ]. Baltussen R, Mwalim O, Blanchet K, Carballo M, Eregata GT, Hailu A, Huda M, Jama M, Johansson KA, Reynolds T, Raza W, Mallender J and Majdzadeh R [ 5 ] described the experience of six countries in defining or refining UHC essential packages for health services found that similar criteria were used. Burden of disease was used in Ethiopia, Somalia, Sudan, and Zanzibar. Effectiveness was used in Afghanistan, Pakistan, and Sudan. Equity was used in Afghanistan, Ethiopia, Pakistan, and Zanzibar. Feasibility (health systems capacity) was used in Afghanistan, Pakistan, Somalia, and Sudan [ 5 ]. Additionally, a scoping review [ 3 ] on criteria used in health benefit package design globally found that, effectiveness was the second most used criteria after cost-effectiveness, equity was fourth while burden of disease was the fifth most used. Cost of intervention and feasibility (equivalent to health systems capacity) were also some of the criteria found by the study [ 3 ]. MCDA applications in health intervention priority setting have also incorporated similar criteria to those derived in this study. A study exploring the feasibility of MCDA for reimbursement decisions in Colombia [ 26 ] applied the EVIDEM (Evidence and Value: Impact on DEcision-Making) framework and used criteria such as size of population affected by disease (burden of disease), improvement of efficacy/effectiveness, and attention to vulnerable groups of population (equity). Effectiveness was used in an MCDA study prioritising interventions for HIV/AIDS in Thailand [ 73 ]. In Pakistan, effectiveness of alternative and manpower required for implementation (health systems capacity) were used in an MCDA for prioritising interventions for preventing dengue [ 96 ]. Burden of disease related criteria (attributable burden and the number of people to be potentially affected by intervention) and an equity (prevalence differences between income levels) criterion were used in an MCDA in Iran that prioritised interventions targeting NCDs [ 97 ]. Another Iranian MCDA study on prioritisation of rehabilitative interventions into the Iranian benefit package used four similar criteria i.e., burden of disease, cost, effectiveness, and feasibility [ 98 ]. Therefore, criteria in this study have been widely used. Burden of disease ended up in the final list of criteria over severity of disease as Kenyan stakeholders preferred it. Both criteria could not be included in the final list of criteria as they overlapped. Interestingly, severity of disease is still a common criterion in MCDA studies in priority setting. It is one of the criteria that captures priority for the worse off (together with equity) and is advocated for use in expanding service coverage for UHC by the WHO consultative group on equity and UHC [ 1 , 2 ]. Severity has been used in MCDA studies for health intervention priority setting in Bulgaria [ 99 ], China [ 75 ], Côte d'Ivoire [ 77 ], South Africa [ 100 ], and Uganda [ 101 ] among others. In an MCDA priority setting study in Kazakhstan, burden of disease had been selected as one of the decision making criteria [ 28 ]. However, the research team advised the committee on the cons of using burden of disease as it would have led to interventions that addressed more common conditions being prioritised over those targeting rare conditions [ 3 , 28 ]. The committee were in agreement with this [ 28 ]. In our MCDA study, the wishes of the stakeholders prevailed. In future studies, using the approach adopted in Kazakhstan could be an option. Equity is another important criterion in priority setting for UHC, as it advances priority for the worse off agenda [ 1 ]. The WHO consultative group on equity and UHC recommends considering the worse off when expanding service coverage, especially the poor and those who live in rural settings [ 1 ]. The equity criterion in our study focused on the socioeconomic aspect of equity i.e., consideration for the poor versus the well-off (rich). This poor/rich classification has been used in other MCDA studies that employed discrete choice modelling such as in Côte d'Ivoire [ 77 ], Ghana [ 27 ], and Nepal [ 72 ]. Nonetheless, some Kenyan stakeholders had suggested whether other aspects of equity could be pursued such as gender or geographic (rural/urban). The socioeconomic aspect of equity, specifically the poor/well-off categorisation, was adopted for our study as it is one of the aspects advocated for by the WHO consultative group on equity and UHC [ 1 ]. Strengths The study had several strengths. First, a systematic process was used to identify and select priority setting criteria. Following a four-step process ensured a structured and transparent process while keeping in line with ISPOR task force MCDA recommendations [ 34 , 35 ] and discrete choice modelling guidance [ 50 ]. Secondly, stakeholders were involved in the semi structured interviews and pilot discrete choice modelling survey where they provided feedback that helped in refining and wording of criteria and levels. Involving stakeholders is critical and helps to ensure transparency and legitimacy of the process [ 38 , 102 ]. Third, criteria that had been identified by HBPAP were used as the starting point [ 9 , 41 ]. The main advantage of this was that our study built on local existing processes rather than formulating things that would be viewed as foreign. Limitations This study had several limitations. First, in analysis of the semi-structured interviews, one researcher (MO) applied the codes to the data. It would have been better to have two independent coders for the qualitative data. Furthermore, MO compiled the final list of criteria and levels using feedback from stakeholders from the semi-structured interviews and pilot discrete choice modelling survey. A better approach would have been to involve a larger panel of stakeholders or researchers in this final step. Nonetheless, MO just analysed the data from semi-structured interviews and pilot discrete choice modelling survey and incorporated the feedback to shape the criteria and levels. Second, the sample size for the pilot discrete choice modelling survey was small (24 stakeholders). However, the results were valuable in shaping the final criteria and levels and in generating priors to be used for the main discrete choice modelling survey in the scoring and weighting phase of the quantitative MCDA study. Third, the levels of the final criteria were qualitative (i.e., high, medium, low). The disadvantage was that they could be prone to misinterpretation by stakeholders if not well understood [ 93 ]. Conversely, the advantage is that they can be mapped to any set of health interventions using the performance matrix and new cut off points could always be defined. CONCLUSION In conclusion, the paper focussed on identifying and selecting criteria to be used in scoring and weighting stages of an exploratory quantitative MCDA process in Kenya. A systematic and transparent four stage process was used, and six criteria were selected. Cost of intervention was added but would not be used in the scoring and weighting stage. It is important for priority setting criteria to be identified and selected using systematic and transparent processes while involving stakeholders [ 38 ]. This enhances transparency, legitimacy, and fairness [ 38 ]. LIST OF ABBREVIATIONS A4R Accountability for reasonableness CAPI Computer assisted personal interviewing CEA Cost Effectiveness Analysis DFT Decision Field Theory EVIDEM Evidence and Value: Impact on DEcision-Making FGDs Focus Group Discussions HBPAP Health Benefits Package Advisory Panel HTA Health Technology Assessment ICFs Informed Consent Forms IQR Interquartile range KEMRI Kenya Medical Research Institute LMICs Low- and Middle-Income Countries MCDA Multi-Criteria Decision Analysis MNL Multinomial Logit Model NGOs Non-governmental organisations NGT Nominal Group Technique NHIF National Hospital Insurance Fund OxTREC Oxford Tropical Research Ethics Committee PBMA Programme Budgeting and Marginal Analysis RRM Random Regret Minimisation RUM Random Utility Maximisation SD Standard Deviation SHA Social Health Authority SHIF Social Health Insurance Fund UHC Universal Health Coverage UHC-EBP Universal Health Coverage Essential benefit package WHO World Health Organisation Declarations Ethics approval and consent to participate Approval to conduct the exploratory quantitative MCDA study in Kenya, which this publication forms part of, was granted by the Oxford Tropical Research Ethics Committee (OxTREC) (Reference: 564-20) and the Kenya Medical Research Institute / Scientific and Ethics Review Unit (no. KEMRI/CGMR-C/210/4095). Furthermore, stakeholders who participated in the semi-structured interviews and discrete choice modelling surveys signed informed consent forms (ICFs) in paper format and/or on Redcap software before participating. Consent for publication Consent to publish findings of the study was obtained from the participants. Availability of data and materials Choice data generated and used in the discrete choice analysis is available open access from the KEMRI-Wellcome Trust Research Programme Harvard Dataverse https://doi.org/10.7910/DVN/CUTGVF [92]. Data from the semi-structured interviews could not be anonymised and are therefore not available open access. Competing interests The authors declare that they have no competing interests. Funding The study received funding from the University of Oxford (Clarendon Fund and Oxford Population Health), University College Oxford (Oxford Radcliffe Graduate Scholarship), International Decision Support Initiative (Bill and Melinda Gates Foundation), and KEMRI-Wellcome Trust Research Programme. This study was also funded in part, by the Wellcome Trust [DEL-15-003] and the UK Foreign, Commonwealth & Development Office, with support from the Developing Excellence in Leadership, Training and Science in Africa (DELTAS Africa) programme. For open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. The funders had no role in conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript. Authors' contributions Conceptualisation: MO & EB. Data Curation: MO. Formal Analysis: MO. Funding Acquisition: MO & EB. Investigation: MO. Methodology: MO & EB. Project Administration: MO. Resources: MO. Supervision: EB. Software: MO. Validation: MO & EB. Visualization: MO. Writing – Original Draft Preparation: MO. Writing – Review & Editing: MO & EB. Acknowledgements Evelyn Kabia (EK) of the KEMRI-Wellcome Trust Research Programme is acknowledged for reviewing the coding framework. James Bukosia (JB) of KEMRI-Wellcome Trust Research Programme developed the discrete choice modelling survey questionnaire on RedCap and curated the dataset on Harvard Dataverse. Finally, MO thanks his D. Phil thesis examiners for their valuable comments during his viva voce and everyone who contributed to the study. References World Health Organization: Making fair choices on the path to universal health coverage: final report of the WHO consultative group on equity and universal health coverage. Geneva: World Health Organization; 2014. Ottersen T, Norheim OF: Setting Priorities in the Pursuit of Universal Health Coverage. In Global Health Priority-Setting: Beyond Cost-Effectiveness. Edited by Norheim OF, Emanuel EJ, Millum J: Oxford University Press; 2019: 0 Hayati R, Bastani P, Kabir MJ, Kavosi Z, Sobhani G: Scoping literature review on the basic health benefit package and its determinant criteria. Globalization and Health 2018, 14: 26. Kaur G, Prinja S, Lakshmi PVM, Downey L, Sharma D, Teerawattananon Y: Criteria Used for Priority-Setting for Public Health Resource Allocation in Low- and Middle-Income Countries: A Systematic Review. International Journal of Technology Assessment in Health Care 2019, 35: 474-483. Baltussen R, Mwalim O, Blanchet K, Carballo M, Eregata GT, Hailu A, Huda M, Jama M, Johansson KA, Reynolds T, et al: Decision-making processes for essential packages of health services: experience from six countries. BMJ Global Health 2023, 8: e010704. Barasa EW, Cleary S, Molyneux S, English M: Setting healthcare priorities: a description and evaluation of the budgeting and planning process in county hospitals in Kenya. Health Policy and Planning 2017, 32: 329-337. Republic of Kenya: Advisory Panel for the Design and Assessment of the Kenya UHC Essential Benefit Package (UHC-EBP). In The Kenya Gazette , vol. CXX —No. 69. pp. 1780-1781. Nairobi: The Government Printer; 2018:1780-1781. Mbau R, Oliver K, Vassall A, Gilson L, Barasa E: A qualitative evaluation of priority-setting by the Health Benefits Package Advisory Panel in Kenya. Health Policy and Planning 2023, 38: 49-60. Ministry of Health Kenya: Report on the UHC Essential Benefits Package. Nairobi, Kenya: Universal Health Coverage Health Benefits Package Advisory Panel; 2018. Facility Improvement Financing Act: Facility Improvement Financing. In 14 ; 2023. Primary Health Care Act: Primary Health Care Act. In 13 ; 2023. The Social Health Insurance Act: Social Health Insurance. In 16 of 2023 ; 2023. Barasa E: The good, bad of Social Health Insurance Bill. In Business Daily ; 2023. The Social Health Insurance (General) Regulations: The Social Health Insurance (General) Regulations. 2023. Kenya health insurance fund: Boost for President William Ruto as court lifts ban [https://www.bbc.co.uk/news/world-africa-68027893] Baltussen R, Niessen L: Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness and Resource Allocation 2006, 4: 14. Daniels N, Sabin JE, Daniels N, Sabin JE: 43Accountability for Reasonableness. In Setting Limits Fairly: Can we learn to share medical resources? : Oxford University Press; 2002: 0 Daniels N, Sabin JE: Setting Limits Fairly: Learning to share resources for health. 2nd edn. New York: Oxford University Press; 2008. Ottersen T, Norheim OF: 293The Uses of Burden-of-Disease Data for Priority Setting. In Measuring the Global Burden of Disease: Philosophical Dimensions. Edited by Eyal N, Hurst SA, Murray CJL, Schroeder SA, Wikler D, Eyal N, Hurst SA, Murray CJL, Schroeder SA, Wikler D: Oxford University Press; 2020: 0 Musgrove P, Fox-Rushby J: Cost-Effectiveness Analysis for Priority Setting. In Disease Control Priorities in Developing Countries. Edited by Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB, Jha P, Mills A, Musgrove P. Washington, DC: World Bank and Oxford University Press; 2006 O'Rourke B, Oortwijn W, Schuller T: The new definition of health technology assessment: A milestone in international collaboration. International Journal of Technology Assessment in Health Care 2020, 36: 187-190. Mitton CR, Donaldson C: Setting priorities and allocating resources in health regions: lessons from a project evaluating program budgeting and marginal analysis (PBMA). Health Policy 2003, 64: 335-348. Mitton C, Donaldson C: Health care priority setting: principles, practice and challenges. Cost Effectiveness and Resource Allocation 2004, 2: 3. Kapiriri L, Razavi D: How have systematic priority setting approaches influenced policy making? A synthesis of the current literature. Health Policy 2017, 121: 937-946. Baltussen R, Marsh K, Thokala P, Diaby V, Castro H, Cleemput I, Garau M, Iskrov G, Olyaeemanesh A, Mirelman A, et al: Multicriteria Decision Analysis to Support Health Technology Assessment Agencies: Benefits, Limitations, and the Way Forward. Value in Health 2019, 22: 1283-1288. Castro Jaramillo HE, Goetghebeur M, Moreno-Mattar O: TESTING MULTI-CRITERIA DECISION ANALYSIS FOR MORE TRANSPARENT RESOURCE-ALLOCATION DECISION MAKING IN COLOMBIA. International Journal of Technology Assessment in Health Care 2016, 32: 307-314. Jehu-Appiah C, Baltussen R, Acquah C, Aikins M, Amah d'Almeida S, Bosu WK, Koolman X, Lauer J, Osei D, Adjei S: Balancing Equity and Efficiency in Health Priorities in Ghana: The Use of Multicriteria Decision Analysis. Value in Health 2008, 11: 1081-1087. Oortwijn W, Surgey G, Novakovic T, Baltussen R, Kosherbayeva L: The Use of Evidence-Informed Deliberative Processes for Health Benefit Package Design in Kazakhstan. International Journal of Environmental Research and Public Health 2022, 19: 11412. Radaelli G, Lettieri E, Masella C, Merlino L, Strada A, Tringali M: IMPLEMENTATION OF EUNETHTA CORE MODEL® IN LOMBARDIA: THE VTS FRAMEWORK. International Journal of Technology Assessment in Health Care 2014, 30: 105-112. Defechereux T, Paolucci F, Mirelman A, Youngkong S, Botten G, Hagen TP, Niessen LW: Health care priority setting in Norway a multicriteria decision analysis. BMC Health Services Research 2012, 12: 39. Youngkong S, Baltussen R, Tantivess S, Mohara A, Teerawattananon Y: Multicriteria Decision Analysis for Including Health Interventions in the Universal Health Coverage Benefit Package in Thailand. Value in Health 2012, 15: 961-970. Belton V, Stewart T: Multiple Criteria Decision Analysis: An Integrated Approach. Norwell, Massachusetts: Kluwer Academic Publishers; 2002. Regier DA, Peacock S: Theoretical Foundations of MCDA. In Multi-Criteria Decision Analysis to Support Healthcare Decisions. Edited by Marsh K, Goetghebeur M, Thokala P, Baltussen R. Cham: Springer International Publishing; 2017: 9-28 Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, Longrenn T, Mussen F, Peacock S, Watkins J, Ijzerman M: Multiple Criteria Decision Analysis for Health Care Decision Making; An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health 2016, 19: 1-13. Marsh K, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, Lönngren T, Mussen F, Peacock S, Watkins J, Devlin N: Multiple Criteria Decision Analysis for Health Care Decision Making—Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health 2016, 19: 125-137. Hansen P, Devlin N: Multi-Criteria Decision Analysis (MCDA) in Healthcare Decision-Making. Oxford University Press; 2019. Helter TM, Boehler CEH: Developing attributes for discrete choice experiments in health: a systematic literature review and case study of alcohol misuse interventions. Journal of Substance Use 2016, 21: 662-668. SteelFisher GK: Including the public perspective in health-related MCDA: ideas from the field of public opinion research and polling. Cost Effectiveness and Resource Allocation 2018, 16: 39. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, Conway PH, Daudelin D, Morrato EH, Leslie LK: A New Taxonomy for Stakeholder Engagement in Patient-Centered Outcomes Research. Journal of General Internal Medicine 2012, 27: 985-991. Obadha M, Barasa E, Kazungu J, Abiiro GA, Chuma J: Attribute development and level selection for a discrete choice experiment to elicit the preferences of health care providers for capitation payment mechanism in Kenya. Health Economics Review 2019, 9: 30. Tromp N, Baltussen R: Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers. BMC Health Services Research 2012, 12: 454. Bliemer MCJ, Rose JM: Designing and Conducting Stated Choice Experiments. In Handbook of Choice Modelling. Second edition. Edited by Hess S, Daly A: Forthcoming; 2022 Trapero-Bertran M, Rodríguez-Martín B, López-Bastida J: What attributes should be included in a discrete choice experiment related to health technologies? A systematic literature review. PLOS ONE 2019, 14: e0219905. Green PE, Srinivasan V: Conjoint Analysis in Marketing: New Developments with Implications for Research and Practice. Journal of Marketing 1990, 54: 3-19. DeShazo JR, Fermo G: Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency. Journal of Environmental Economics and Management 2002, 44: 123-143. Arentze T, Borgers A, Timmermans H, DelMistro R: Transport stated choice responses: effects of task complexity, presentation format and literacy. Transportation Research Part E: Logistics and Transportation Review 2003, 39: 229-244. Caussade S, Ortúzar JdD, Rizzi LI, Hensher DA: Assessing the influence of design dimensions on stated choice experiment estimates. Transportation Research Part B: Methodological 2005, 39: 621-640. Hensher DA: How do respondents process stated choice experiments? Attribute consideration under varying information load. Journal of Applied Econometrics 2006, 21: 861-878. Rose JM, Hensher DA, Caussade S, Ortúzar JdD, Jou R-C: Identifying differences in willingness to pay due to dimensionality in stated choice experiments: a cross country analysis. Journal of Transport Geography 2009, 17: 21-29. Bridges JFP, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, Johnson FR, Mauskopf J: Conjoint Analysis Applications in Health—a Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value in Health 2011, 14: 403-413. Hinton L, Ryan S: Interviews. In Qualitative Research in Health Care. 2020: 43-55 Clark T, Foster L, Sloan L, Bryman A: Bryman's Social Research Methods. Sixth edn. Oxford, United Kingdom: Oxford University Press; 2021. Universal health coverage (UHC) [https://www.who.int/en/news-room/fact-sheets/detail/universal-health-coverage-(uhc)] Ritchie J, Spencer L: Qualitative data analysis for applied policy research. In Analyzing qualitative data. Edited by Burgess RG, Bryman A. London: Routledge; 1994: 173-194 Pope C, Ziebland S, Mays N: Analysis. In Qualitative Research in Health Care. 4th edition. Edited by Pope C, Mays N. Hoboken: Wiley-Blackwell; 2020: 111-133 Spencer L, Ritchie J, Ormston R, O'Connor W, Barnard M: Analysis: Principles and processes. In Qualitative Research Practice: A Guide for Social Science Students and Researchers. 2nd edition. Edited by Ritchie J, Lewis J, Nicholls CM, Ormston R. London: SAGE; 2014 QSR International Pty Ltd: NVivo (released in June 2021). 2021. R Core Team: R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2024. Bliemer MCJ, Collins AT: On determining priors for the generation of efficient stated choice experimental designs. Journal of Choice Modelling 2016, 21: 10-14. Busemeyer JR, Townsend JT: Fundamental derivations from decision field theory. Mathematical Social Sciences 1992, 23: 255-282. Busemeyer JR, Townsend JT: Decision Field Theory: A Dynamic–Cognitive Approach to Decision Making in an Uncertain Environment. Psychological Review 1993, 100: p 432–459. Diederich A: Dynamic Stochastic Models for Decision Making under Time Constraints. Journal of Mathematical Psychology 1997, 41: 260-274. Roe RM, Busemeyer JR, Townsend JT: Multialternative decision field theory: A dynamic connectionst model of decision making. Psychological Review 2001, 108: 370-392. Hancock TO, Hess S, Choudhury CF: Decision field theory: Improvements to current methodology and comparisons with standard choice modelling techniques. Transportation Research Part B: Methodological 2018, 107: 18-40. Chorus CG, Arentze TA, Timmermans HJP: A Random Regret-Minimization model of travel choice. Transportation Research Part B: Methodological 2008, 42: 1-18. Chorus CG: A New Model of Random Regret Minimization. European Journal of Transport and Infrastructure Research 2010, 10 . Chorus C: Random Regret Minimization: An Overview of Model Properties and Empirical Evidence. Transport Reviews 2012, 32: 75-92. McFadden D: Conditional logit analysis of qualitative choice behavior. In Frontiers in Econometrics. Edited by Zarembka P. New York: Academic Press; 1974: 105-142 Train KE: Discrete Choice Methods with Simulation. 2 edn. Cambridge: Cambridge University Press; 2009. Hensher DA, Rose JM, Greene WH: Applied Choice Analysis. 2nd edn. Cambridge: Cambridge University Press; 2015. Baltussen R, Stolk E, Chisholm D, Aikins M: Towards a multi-criteria approach for priority setting: an application to Ghana. Health Economics 2006, 15: 689-696. Baltussen R, ten Asbroek AHA, Koolman X, Shrestha N, Bhattarai P, Niessen LW: Priority setting using multiple criteria: should a lung health programme be implemented in Nepal? Health Policy and Planning 2007, 22: 178-185. Youngkong S, Teerawattananon Y, Tantivess S, Baltussen R: Multi-criteria decision analysis for setting priorities on HIV/AIDS interventions in Thailand. Health Research Policy and Systems 2012, 10: 6. Marsh K, Dolan P, Kempster J, Lugon M: Prioritizing investments in public health: a multi-criteria decision analysis. Journal of Public Health 2013, 35: 460-466. Paolucci F, Mentzakis E, Defechereux T, Niessen LW: Equity and efficiency preferences of health policy makers in China—a stated preference analysis. Health Policy and Planning 2015, 30: 1059-1066. Meusel V, Mentzakis E, Baji P, Fiorentini G, Paolucci F: Priority setting in the German healthcare system: results from a discrete choice experiment. International Journal of Health Economics and Management 2023, 23: 411-431. Diaby V, Lachaine J: An application of a proposed framework for formulary listing in low-income countries. Applied Health Economics and Health Policy 2011, 9: 389-402. ChoiceMetrics: Ngene 1.3 User Manual & Reference Guide. Sydney, Australia; 2022. Daly A, Dekker T, Hess S: Dummy coding vs effects coding for categorical variables: Clarifications and extensions. Journal of Choice Modelling 2016, 21: 36-41. Obadha M, Chuma J, Kazungu J, Abiiro GA, Beck MJ, Barasa E: Preferences of healthcare providers for capitation payment in Kenya: a discrete choice experiment. Health Policy and Planning 2020, 35: 842-854. Agarwal S, Abuya T, Kintu R, Mwanga D, Obadha M, Pandya S, Warren CE: Understanding community health worker incentive preferences in Uganda using a discrete choice experiment. J Glob Health 2021, 11: 07005. Abuya T, Mwanga D, Obadha M, Ndwiga C, Odwe G, Kavoo D, Wanyugu J, Warren C, Agarwal S: Incentive preferences for community health volunteers in Kenya: findings from a discrete choice experiment. BMJ Open 2021, 11: e048059. Computer-Assisted Personal Interviewing (CAPI) Surveys: A Guide [https://www.geopoll.com/blog/capi-surveys-guide/] Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics 2009, 42: 377-381. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, McLeod L, Delacqua G, Delacqua F, Kirby J, Duda SN: The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics 2019, 95: 103208. Willis GB: Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, California: SAGE Publications Inc; 2004. De Brún A, Flynn D, Ternent L, Price CI, Rodgers H, Ford GA, Rudd M, Lancsar E, Simpson S, Teah J, Thomson RG: A novel design process for selection of attributes for inclusion in discrete choice experiments: case study exploring variation in clinical decision-making about thrombolysis in the treatment of acute ischaemic stroke. BMC Health Services Research 2018, 18: 483. Ryan M, Krucien N, Hermens F: The eyes have it: Using eye tracking to inform information processing strategies in multi-attributes choices. Health Economics 2018, 27: 709-721. Hess S, Palma D: Apollo: A flexible, powerful and customisable freeware package for choice model estimation and application. Journal of Choice Modelling 2019, 32: 100170. Vermunt JK, Magidson J: Technical guide for Latent GOLD Choice 4.0: Basic and advanced. Belmont, Massachusetts: Statistical Innovations Inc; 2005. Maaya L, Meulders M, Surmont N, Vandebroek M: Effect of Environmental and Altruistic Attitudes on Willingness-to-Pay for Organic and Fair Trade Coffee in Flanders. In Sustainability , vol. 10; 2018. Obadha M, Barasa E: Pilot discrete choice modelling survey for an MCDA study in Kenya. V1 edition: Harvard Dataverse; 2024. Marsh K, Byun J-H, Youngkong S, Willke R: Using Multi-Criteria Decision Analysis in Healthcare Decision Making: Approaches and Applications. In ISPOR Virtual Short Course Program 12 - 15 October 2020 ; 2020. Obadha M, Mumbi A, Njuguna RG, Orangi S, Nguhiu P, Ngaiza G, Omollo H, Njeru N, Barasa E: HTA16 Eliciting Preferences for Health Technology Assessment Criteria Using Analytic Hierarchy Process and Discrete Choice Experiment in Kenya. Value in Health 2023, 26: S261. Terwindt F, Rajan D, Soucat A: Priority-setting for national health policies, strategies and plans. In Strategizing national health in the 21st century: a handbook. Volume 71. Edited by Schmets G, Rajan D, Kadandale S. Geneva: World Health Organization; 2016 Sabir M, Ali Y, Muhammad N: Forecasting incidence of dengue and selecting best method for prevention. J Pak Med Assoc 2018, 68: 1383-1386. Bakhtiari A, Takian A, Majdzadeh R, Haghdoost AA: Assessment and prioritization of the WHO “best buys” and other recommended interventions for the prevention and control of non-communicable diseases in Iran. BMC Public Health 2020, 20: 333. Shahabi S, Pardhan S, Ahmadi Teymourlouy A, Skempes D, Shahali S, Mojgani P, Jalali M, Lankarani KB: Prioritizing solutions to incorporate Prosthetics and Orthotics services into Iranian health benefits package: Using an analytic hierarchy process. PLOS ONE 2021, 16: e0253001. Iskrov G, Miteva-Katrandzhieva T, Stefanov R: Multi-Criteria Decision Analysis for Assessment and Appraisal of Orphan Drugs. Frontiers in Public Health 2016, 4 . Miot J, Wagner M, Khoury H, Rindress D, Goetghebeur MM: Field testing of a multicriteria decision analysis (MCDA) framework for coverage of a screening test for cervical cancer in South Africa. Cost Effectiveness and Resource Allocation 2012, 10: 2. McCormick BJJ, Waiswa P, Nalwadda C, Sewankambo NK, Knobler SL: SMART Vaccines 2.0 decision-support platform: a tool to facilitate and promote priority setting for sustainable vaccination in resource-limited settings. BMJ Global Health 2020, 5: e003587. Thokala P, Madhavan G: Stakeholder involvement in Multi-Criteria Decision Analysis. Cost Effectiveness and Resource Allocation 2018, 16: 0. Additional Declarations No competing interests reported. Supplementary Files Additionalfile1.docx Additionalfile2.docx Additionalfile3.docx Additionalfile4.docx Additionalfile5.docx Additionalfile6.docx Additionalfile7.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6630410","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":456722315,"identity":"f7e44a7b-e996-4d26-8212-512bc780ccf6","order_by":0,"name":"Melvin Obadha","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABQklEQVRIie2RP0vDQBTAX8jQJZr1xdR+hpPAtcGCX8Wj4NSu2qHUlMJNha4Vwc8gCKXjhUCzBFwjDjZLp3yACgpeakLbGMHR4X5wd48HP96fA1Ao/iFE6B5cZgEAlQ/qJsiMDOTZBt/hHq7QDpRWzfK0QtEqlQuR53Olb5Ii85tCwxHHZAGNZi2YrzaAthP6/ORl0QIS+h5sBsDuvEMl8jmyCBx3cnVzNgF0aMS43YsQSMQ8bbIEdl+qErMxYRzYo+hSNAA7VGQKl0osGzvygD2UNvaW5MpzSq0PwNunabJTtM8KJdZGq60Sd6ktq+gE96roWZVSY7LhTEHHna2v7TpBHeNkfC4Vw5KzBPUlOqXxSRgI/523G02zM7fS/lA3px3/tceHjeMw8JN00D6dCaigWD3ZpYzsEj9+RaFQKBR/4Qtd13YeJv+83AAAAABJRU5ErkJggg==","orcid":"","institution":"KEMRI-Wellcome Trust Research Programme","correspondingAuthor":true,"prefix":"","firstName":"Melvin","middleName":"","lastName":"Obadha","suffix":""},{"id":456722316,"identity":"7332f472-65e8-4818-9dff-231b49bad90e","order_by":1,"name":"Edwine Barasa","email":"","orcid":"","institution":"KEMRI-Wellcome Trust Research Programme","correspondingAuthor":false,"prefix":"","firstName":"Edwine","middleName":"","lastName":"Barasa","suffix":""}],"badges":[],"createdAt":"2025-05-09 16:38:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6630410/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6630410/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":82872117,"identity":"49ffbbbd-2fbf-4a61-b374-8f297c4e5ea4","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":78039,"visible":true,"origin":"","legend":"\u003cp\u003eFramework for identifying and selecting criteria as adapted from Helter TM and Boehler CEH [37]: Source [40] Creative Commons Attribution 4.0 International License (CC BY 4.0)\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/48c918f9caed6c5d143e8343.jpg"},{"id":82873351,"identity":"2a4669d0-3fb3-4c91-8673-a6f44052031a","added_by":"auto","created_at":"2025-05-16 09:15:59","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":248162,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eA practice choice task used as a dominance test.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/22a9d80a47bbb0f316f3cba9.jpg"},{"id":82872123,"identity":"6636debd-b922-4466-bc8a-0c255c07414f","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":42148,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRelative importance of the criteria expressed in percentages.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/094465b6b345161a7c7aa7b9.jpg"},{"id":100726441,"identity":"99ba09b5-9e2a-49e3-bcc8-03d1d68dabc3","added_by":"auto","created_at":"2026-01-20 20:24:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5035737,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/6f19a719-6b0f-4674-ba17-e3db618655e9.pdf"},{"id":82872119,"identity":"38b51a37-42fb-41a6-be54-33f7e8fc90f8","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":29319,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/20bcb2828075fc002db1a98e.docx"},{"id":82873352,"identity":"2ad0161c-dbdc-4c19-a898-3b9c0ad23fee","added_by":"auto","created_at":"2025-05-16 09:15:59","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":36308,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile2.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/4cde55112b81452482b1caba.docx"},{"id":82873358,"identity":"d0c36e21-e87f-44df-9ad6-dc93f1345992","added_by":"auto","created_at":"2025-05-16 09:15:59","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":28181,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile3.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/801b5b21eac69d5d275a73ac.docx"},{"id":82872130,"identity":"361da7ca-721d-4f4e-b6dc-703872ed6a82","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":29674,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile4.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/b46e1f3b2a7301bada67daa2.docx"},{"id":82872124,"identity":"09a5dccc-d9bd-412f-b550-44a5ea84c5a8","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":47311,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile5.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/904128858ab7d6f3c115f3ad.docx"},{"id":82872128,"identity":"36a48395-999a-4ffd-a17a-09f3f1e3c4f0","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"docx","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":27087,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile6.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/a3115811994a1f2cc1aea912.docx"},{"id":82872137,"identity":"b5e0aa7c-444d-4227-b70b-222afd713c2a","added_by":"auto","created_at":"2025-05-16 09:07:59","extension":"docx","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":31346,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile7.docx","url":"https://assets-eu.researchsquare.com/files/rs-6630410/v1/c66f94f4f0b3206d86e7f855.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003eIdentifying and Selecting Priority Setting Criteria for an Exploratory Multi-Criteria Decision Analysis Study for Health Benefit Package Design in Kenya\u003c/p\u003e","fulltext":[{"header":"BACKGROUND","content":"\u003cp\u003eIn expanding the range of services in the pursuance of universal health coverage (UHC), countries need to use explicit evidence-based priority setting criteria. The World Health Organisation (WHO) consultative group on equity and UHC recommends using at least these three criteria when prioritising health interventions i.e., cost-effectiveness, financial risk protection, and priority for the worse off (equity) [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. A scoping review on criteria used in defining health benefit packages determined cost-effectiveness as frequently used, followed by effectiveness, budget impact, equity, and burden of disease [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. These were the top five criteria among many others revealed by the review such as necessity, safety, and feasibility among others [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. In low-and middle income countries (LMICs), a systematic review determined cost-effectiveness criterion as frequently used in priority setting, followed by health benefits, and equity [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. Cost-effectiveness has been used by a number of countries defining UHC essential packages for health services such as Ethiopia, Pakistan, Somalia, Sudan, and Zanzibar [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Financial risk protection has been used by Afghanistan, Ethiopia, Pakistan, and Zanzibar. Equity has also been used by Afghanistan, Ethiopia, Pakistan, and Zanzibar [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Nonetheless, other criteria that have been used by countries defining UHC essential packages for health services include budget impact, disease burden, effectiveness, feasibility, integrated service delivery, public and political acceptability, quality of evidence, and socio-economic impact [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn Kenya, priority setting practices have been identified as ad hoc, fragmented, and lacked use of explicit criteria [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. However, there were a few initiatives that did incorporate explicit criteria use. In 2018, the Cabinet secretary for health appointed a panel of experts called the health benefits package advisory panel (HBPAP) [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. The panel had the mandate of defining a UHC essential benefit package (UHC-EBP) in 60 days [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. The package was to be funded through public sources, piloted in four counties, and scaled up to the rest of the counties [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. The UHC-EBP process incorporated the use of four criteria i.e., affordability, cost-effectiveness, effectiveness, and feasibility [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. The criteria were used individually (separately) rather than simultaneously to allow trade-offs between criteria [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. However, this package was never adopted [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. The current Kenyan government is still committed to achieving UHC and has facilitated the enactment of three relevant pieces of legislation into law i.e., the facility improvement financing act 2023, the primary health care act 2023, and the social health insurance act 2023 [\u003cspan additionalcitationids=\"CR11 CR12\" citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. The primary health care act 2023 and the facility improvement financing act 2023 aim to improve primary health care and health facility financing [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. The social health insurance act 2023 converted Kenya\u0026rsquo;s national health insurance fund (NHIF) into a social health insurance fund (SHIF) that is mandatory for everyone, including those in the informal sector, and managed by a social health authority (SHA) [\u003cspan additionalcitationids=\"CR13 CR14\" citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. SHA regulations have created a new health benefit advisory panel and legislated a range of priority setting criteria. The new Panel has been selected and is in the process of being operationalised. This calls for the need for systematic priority setting methods that incorporate the use of multiple criteria simultaneously while allowing for trade-offs [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThere are many techniques used in health intervention priority setting such as accountability for reasonableness (A4R) [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e], burden of disease analysis [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], cost-effectiveness analysis (CEA) [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e], health technology assessment (HTA) [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e], multi-criteria decision analysis (MCDA) [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e], and programme budgeting and marginal analysis (PBMA) [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] among others [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. MCDA stands out as being capable of incorporating multiple criteria simultaneously in setting health priorities and capturing trade-offs being made [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. Furthermore, MCDA allows involvement of stakeholders which makes the priority setting process transparent and validates it [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. MCDA has been used to set priorities for health interventions in countries such as Colombia [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e], Ghana [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e], Kazakhstan [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e], Italy [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e], Norway [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e], and Thailand [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] among others.\u003c/p\u003e \u003cp\u003eAgainst this backdrop, we explored the use of quantitative MCDA in Kenya using the example of the country\u0026rsquo;s UHC benefit package. The aim was to prioritise health interventions into the package. Quantitative MCDA has several steps that entail stating the decision goal or objective (e.g., prioritise vaccines for coverage), mapping out stakeholders (e.g., policymakers, academics etc), identifying and selecting alternatives to be considered (e.g., vaccines, drugs etc), identifying and selecting decision making criteria, measuring performance, scoring alternatives on criteria, weighting criteria, aggregation, uncertainty analysis, and results reporting and interpretation [\u003cspan additionalcitationids=\"CR33 CR34 CR35\" citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. This paper focuses on the process used to identify and select priority setting criteria for use in the exploratory quantitative MCDA study in Kenya using a four-step process i.e., raw data collection, data reduction, removing inappropriate criteria, and wording [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eEvery step of the MCDA process must be done well and reported to ensure quality decisions are made, bias reduced, and uncertainty is addressed or described, according to recommendations of ISPOR taskforce guidelines [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]. Furthermore, ensuring stakeholders participate (especially patients and the public) in the different stages of the MCDA exercise, makes the process transparent and validates it as earlier stated [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. These aspects are sometimes not well reported in literature especially identification and selection of health interventions and priority setting criteria [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]. Moreover, stakeholders like patients and the public are left out of the exercise [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. This study addresses these gaps and contributes to the body of knowledge by reporting how priority setting criteria for an exploratory quantitative MCDA study in Kenya were identified and selected and includes the participation of stakeholders (e.g., patients) in the exercise.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cp\u003e \u003cb\u003eStudy setting and population\u003c/b\u003e.\u003c/p\u003e \u003cp\u003eThe study setting was at national level in Kenya and used the country\u0026rsquo;s UHC benefit package as an example. The aim was to prioritise health interventions into the package, but this paper focuses on the process of identifying and selecting criteria for prioritisation. The study population were stakeholders involved in health intervention priority setting identified using the \u0026ldquo;Ps\u0026rdquo; framework by Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, Conway PH, Daudelin D, Morrato EH and Leslie LK [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. These stakeholder groups were purchasers, policymakers, providers, principal investigators (academics and researchers), patients, and the public [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. Stakeholders at national and subnational levels were targeted.\u003c/p\u003e \u003cp\u003e \u003cb\u003eFramework for identifying and selecting priority setting criteria.\u003c/b\u003e \u003c/p\u003e \u003cp\u003eTo identify and select criteria, a four-stage approach by Helter and Boehler (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) was adapted i.e., raw data collection, data reduction, removing inappropriate criteria, and wording [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. The framework was selected as it provides a systematic method of identifying and selecting criteria and allows multiple approaches to be incorporated, while involving stakeholders. Stakeholder involvement at different stages of the MCDA process is important as it provides transparency and validity to the process [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. We had also previously used the framework in the Kenyan setting to develop a discrete choice modelling survey that explored preferences for capitation payments in the country [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Therefore, the framework was fit for the local context. Raw data collection consists of collecting data using qualitative and quantitative methods [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Data reduction involves reducing collected data to a long list of criteria by analysing it qualitatively and/or quantitatively [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Removing inappropriate criteria involves reducing the long list of criteria to a smaller list through considering factors such as capability of being traded, completeness, non-overlap, non-redundancy, plausibility, preference independence, and salience [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Wording refers to refining criteria using techniques such as piloting and researcher\u0026rsquo;s judgement among others [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eStages one and two: raw data collection and data reduction.\u003c/b\u003e \u003c/p\u003e \u003cp\u003eCriteria that had been identified by HBPAP when defining the UHC-EBP were used as a starting point. The 10 criteria were affordability, burden of disease, catastrophic health expenditure, congruence with existing priorities, cost effectiveness, effectiveness and safety, equity, feasibility (health workforce requirements), feasibility (service and health products \u0026amp; medical technology requirements), and severity of disease (see Additional file 1) [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. In identifying and selecting criteria for defining the UHC-EBP, HBPAP obtained the criteria and definitions from a study by Tromp N and Baltussen R [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. The panel then conducted a modified nominal group technique (NGT) with stakeholders to refine the long criteria list and reach consensus on 10 [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThis study used HBPAP\u0026rsquo;s 10 criteria as the starting point as the aim was to build on existing and past priority setting processes. This would enable the study to explore the feasibility of using MCDA for health intervention priority setting in Kenya and provide lessons that would easily be acceptable to and adopted by local stakeholders. Starting with an existing set of criteria meant that the raw data collection and data reduction stages had been conducted, as the criteria were obtained from HBPAP\u0026rsquo;s list who had in turn obtained them from Tromp N and Baltussen R [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cb\u003eStage three: removing inappropriate criteria.\u003c/b\u003e \u003c/p\u003e \u003cp\u003eStage three involved reducing the long list of 10 criteria to a shorter one. Researchers\u0026rsquo; judgement was used where three researchers provided comments based on their expertise in HTA, MCDA, priority-setting, and discrete choice modelling [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. The target was to refine and reduce the criteria to approximately five or six and a maximum of four levels per criteria (minimum two levels) [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. This was because discrete choice modelling was one of the methods that would be used in scoring and weighting in the quantitative MCDA process. A systematic review had found that discrete choice modelling studies that elicited preferences for health interventions used a mean of 5.74 attributes and 3.26 levels per attribute [\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e]. Increasing the number of criteria complicates the choice tasks and leads to a greater cognitive burden on respondents [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e, \u003cspan additionalcitationids=\"CR45 CR46 CR47 CR48 CR49\" citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eSeveral factors are usually considered when removing inappropriate criteria. In this stage, the researchers considered factors like availability of data, capability of being traded, completeness, correlation between criteria, decision context, non-overlap, non-redundancy, plausibility, preference independence, relevance to study objective, and salience [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e]. The researchers provided comments on the criteria, domains, lay, and technical definitions. They also defined probable levels for the criteria to be used in the discrete choice modelling survey. The resulting criteria, definitions, and probable levels were to be refined by stakeholders using semi-structured interviews and pilot tested using discrete choice modelling in the next stage.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStage four: wording\u003c/h2\u003e \u003cp\u003eIn this stage, the resulting criteria and levels were further refined using semi-structured interviews and a pilot discrete choice modelling survey with stakeholders.\u003c/p\u003e \u003cp\u003eKey informant semi-structured Interviews\u003c/p\u003e \u003cp\u003eSemi-structured interviews were selected as they offered flexibility in exploring the themes under consideration therefore enabling the stakeholders to offer detailed information [\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e, \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e]. Furthermore, it would make getting busy stakeholders easier as it meant they could be interviewed at their place of work and on different days. Other methods such as focus group discussions (FGDs), meetings, or nominal group technique (NGT) would have required all stakeholders to be present at a specified date and time. The informants for the semi-structured interviews were Kenyan-based stakeholders with knowledge of UHC, HTA, and priority setting. Therefore, stakeholders who had participated in HTA and priority setting initiatives in Kenya were mainly targeted. Additionally, stakeholders conducting health research, those providing healthcare, and those involved in health policymaking were also targeted.\u003c/p\u003e \u003cp\u003eThe targeted stakeholders were policymakers (national and county-level), principal investigators (academics and researchers), providers (faith-based/non-governmental organisations NGO, private, and public sectors), and purchasers (ministry of health, county department of health, and health insurance organisations) [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. They were contacted through emails and phone calls or approached-in person at their places of work. Additionally, snow balling was used where busy stakeholders identified others who were subsequently approached using emails, phone calls, or in-person at their place of work. Overall, 42 stakeholders were approached, with 11 of them agreeing to participate. A total of 10 stakeholders were interviewed at their places of work or convenient venues, after signing informed consent forms (ICFs). The interviews were conducted in person between June 2022 and July 2022. The interviews lasted between 45 minutes and 90 minutes.\u003c/p\u003e \u003cp\u003eThe interview guide was developed using the priority setting criteria list (see additional file 2) and other aspects being considered in the broader quantitative MCDA study such as health interventions [\u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e]. The guide was piloted with four researchers who had experience in priority-setting. The guide had three sections. The first section covered socio-demographic characteristics, respondents\u0026rsquo; involvement in HTA and priority setting (e.g., benefit package design or essential medicines lists development), and questions about UHC pilot in four Kenyan countries. In the second section, questions focused on a list of health interventions under consideration in the study. In the third section, stakeholders were provided with a list of six criteria, their levels, and lay and technical definitions to comment on. Finally, they were asked general questions about the barriers and enablers of institutionalisation of HTA in Kenya, and what other countries could learn from the Kenyan experience. The focus, in this publication, will be the third section of the interview guide where stakeholders provided feedback on the six criteria, definitions, and levels.\u003c/p\u003e \u003cp\u003eAnalysis of the data employed a framework approach because it is systematic, reproducible, and transparent [\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e, \u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e]. Interviews were transcribed verbatim by a research company. One researcher (MO) reviewed the transcripts and made corrections by listening to the recordings. MO then familiarised with the data and developed an initial framework using the priority-setting criteria, health interventions, and interview guide. The resulting framework was reviewed by another researcher. MO applied the framework to the data (indexing and sorting). The rest of the steps involved reviewing data extracts, data summary and display, and abstraction and interpretation [\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e, \u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e]. NVIVO Release 1 was used to manage the data [\u003cspan citationid=\"CR57\" class=\"CitationRef\"\u003e57\u003c/span\u003e]. Finally, stakeholder sociodemographic characteristics were analysed quantitatively and presented using measures of central tendency (means and medians), measures of dispersion (interquartile range and standard deviation), and proportions. R version 4.2.2 was used in the analysis [\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eDiscrete choice modelling pilot\u003c/p\u003e \u003cp\u003eThe pilot survey was conducted for two purposes. First, discrete choice modelling had been selected as a method to be used in the scoring and weighting phase of the quantitative MCDA in Kenya. Therefore, the pilot would generate priors for the main discrete choice modelling survey\u0026rsquo;s experimental design [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e]. The second reason was to experiment with the criteria and levels. Discrete choice modelling was selected for the exploratory quantitative MCDA study as it was grounded in established theories in fields such as economics and psychology e.g., decision field theory (DFT) [\u003cspan additionalcitationids=\"CR61 CR62 CR63\" citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e], random regret minimisation (RRM) [\u003cspan additionalcitationids=\"CR66\" citationid=\"CR65\" class=\"CitationRef\"\u003e65\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR67\" class=\"CitationRef\"\u003e67\u003c/span\u003e], and random utility maximisation (RUM) [\u003cspan additionalcitationids=\"CR69\" citationid=\"CR68\" class=\"CitationRef\"\u003e68\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR70\" class=\"CitationRef\"\u003e70\u003c/span\u003e]. Models that follow RUM were used in this study as they are commonly used in quantitative MCDA studies that employ discrete choice modelling [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan additionalcitationids=\"CR72 CR73 CR74 CR75\" citationid=\"CR71\" class=\"CitationRef\"\u003e71\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR76\" class=\"CitationRef\"\u003e76\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eSix criteria and levels were used for the pilot study (see additional file 3). Three criteria had three levels while the other three had two levels each. An unlabelled discrete choice experiment that did not have an opt out was adopted. The experiment had two hypothetical interventions i.e., Intervention A and B. A forced choice scenario was adopted because the assumption was that in real life, stakeholders involved in priority setting for benefit package design do not opt out from appraising interventions before them. Similar studies that used discrete choice modelling in MCDA for health intervention priority setting in LMICs adopted unlabelled designs that had two hypothetical alternatives without an opt-out. These studies were conducted in C\u0026ocirc;te d'Ivoire [\u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e], Ghana [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR71\" class=\"CitationRef\"\u003e71\u003c/span\u003e], Nepal [\u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e72\u003c/span\u003e], and Thailand [\u003cspan citationid=\"CR73\" class=\"CitationRef\"\u003e73\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eA D-efficient experimental design was used, optimising for a main effects multinomial logit model (MNL) using Ngene software version 1.3 [\u003cspan citationid=\"CR78\" class=\"CitationRef\"\u003e78\u003c/span\u003e]. Priors used were educated best guesses, with values closer to zero (see Additional file 4) [\u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e]. All criteria were categorical incorporating dummy coding [\u003cspan citationid=\"CR79\" class=\"CitationRef\"\u003e79\u003c/span\u003e]. A total of 12 choice tasks were generated, which seemed optimal, based on previous studies in Kenya and Uganda among community health workers and health facility managers [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan additionalcitationids=\"CR81\" citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR82\" class=\"CitationRef\"\u003e82\u003c/span\u003e]. Five versions of the questionnaire (see Additional file 5) were generated where the order of the 12 choice tasks were randomised. The first version comprised of the original order generated by the experimental design. The remaining four versions each had the order of the 12 choice tasks randomised. The questionnaires had two practice choice tasks where the second choice task consisted of a dominant alternative (see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e) meant to gauge stakeholders\u0026rsquo; understanding of answering the choice tasks [\u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eStakeholder sampling was purposive and targeted five stakeholder groups \u0026ldquo;Ps\u0026rdquo; at national and county level. These stakeholders were patient advocacy groups, principal investigators (academics and researchers), providers (faith-based/NGO, private, and public), public, and purchasers and policy makers [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. Snowballing sampling technique was additionally used to reach eligible stakeholders. Eligible stakeholders were approached directly at their place of work or contacted in advance through emails or phone calls, and a convenient date and time set. Other stakeholders were called to a central venue to complete the survey questionnaire. They were invited through emails, phone calls, and snowballing.\u003c/p\u003e \u003cp\u003eComputer assisted personal interviewing (CAPI) was employed for the pilot survey. In CAPI, the researcher or study participant uses a device such as a tablet, mobile phone, or laptop to complete the survey questionnaire in person [\u003cspan citationid=\"CR83\" class=\"CitationRef\"\u003e83\u003c/span\u003e]. The discrete choice modelling survey was self-administered (in person) using a tablet or smart phone in the presence of a researcher. Data were collected and managed using REDCap tool [\u003cspan citationid=\"CR84\" class=\"CitationRef\"\u003e84\u003c/span\u003e, \u003cspan citationid=\"CR85\" class=\"CitationRef\"\u003e85\u003c/span\u003e]. The researcher explained the aims of the study and provided an ICF to the stakeholder to complete (on a tablet/smart phone and a paper version). The paper version of the ICF was provided to test whether leaving a paper copy with the stakeholder was better for reference. Stakeholders could also enter their email to receive an electronic version of the ICF. Once stakeholders signed both the paper ICF and the tablet/smartphone version, they completed the first section of the questionnaire on REDcap using a tablet or smartphone. Then, the researcher explained the choice scenario in the second section. Here, stakeholders were prompted to assume that the government wanted to define a UHC benefit package, and they needed to select interventions to be included in the package. The criteria and levels were explained, and stakeholders were taken through two practice choice tasks. The second-choice task included a dominant alternative to check whether the stakeholder had understood the task [\u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e]. Pseudo randomisation was used to allocate one of the five versions of the 12 choice tasks to stakeholders. Stakeholders were able to verbalise their thought process or justify their answers during the think aloud exercise [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e, \u003cspan citationid=\"CR86\" class=\"CitationRef\"\u003e86\u003c/span\u003e, \u003cspan citationid=\"CR87\" class=\"CitationRef\"\u003e87\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e Overall, 84 participants were approached either directly or through snowballing. Only 25 signed the ICF and started the survey, with 24 completing the choice tasks. The response rate was 29.76%. One respondent did not complete the choice tasks as they found the exercise cognitively burdensome. The survey took between 30 and 45 minutes to complete. The survey was administered at the stakeholder\u0026rsquo;s place of work. For those that were administered at places away from the workstation, stakeholders were paid out of duty station allowance of Kenya shillings (KES) 1000 (US \u003cspan\u003e$\u003c/span\u003e7.50) and transport costs were reimbursed depending on where they travelled from.\u003c/p\u003e \u003cp\u003eIn data analysis, a main effects MNL model was used to model the choice probabilities. In the utility function, categorical attributes were dummy coded to estimate non-linear effects [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. Alternative specific constant was used and set at alternative A to check for the presence of left to right bias [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e, \u003cspan citationid=\"CR88\" class=\"CitationRef\"\u003e88\u003c/span\u003e]. Analysis was conducted using Apollo choice modelling package version 0.2.9 on R version 4.3.3 [\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e, \u003cspan citationid=\"CR89\" class=\"CitationRef\"\u003e89\u003c/span\u003e]. Relative importance estimates were also computed (see additional file 6) on Apollo and the delta method was used to generate corresponding robust standard errors and 95% confidence intervals [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e, \u003cspan additionalcitationids=\"CR90\" citationid=\"CR89\" class=\"CitationRef\"\u003e89\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR91\" class=\"CitationRef\"\u003e91\u003c/span\u003e]. The dataset used in the analysis including the Rscript for the MNL analysis are available open access [\u003cspan citationid=\"CR92\" class=\"CitationRef\"\u003e92\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFinal list of criteria and levels\u003c/p\u003e \u003cp\u003eResults and feedback from the semi-structured interviews and the pilot discrete choice modelling survey were used to develop the final list of six criteria and levels. This section was done by one researcher (MO).\u003c/p\u003e \u003c/div\u003e"},{"header":"RESULTS","content":"\u003cp\u003e \u003cb\u003eStage three: removing inappropriate criteria.\u003c/b\u003e \u003c/p\u003e \u003cp\u003eCost-effectiveness criterion was split into two criteria, cost of intervention, and effectiveness of intervention. The resulting extra effectiveness criterion was dropped as it was redundant. The cost of intervention criterion was omitted from the list of criteria to be considered for inclusion in the scoring and weighting stages of the MCDA. Baltussen R, Marsh K, Thokala P, Diaby V, Castro H, Cleemput I, Garau M, Iskrov G, Olyaeemanesh A, Mirelman A, Mobinizadeh M, Morton A, Tringali M, van Til J, Valentim J, Wagner M, Youngkong S, Zah V, Toll A, Jansen M, Bijlmakers L, Oortwijn W and Broekhuizen H [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] argue that costs or cost-effectiveness criterion should not be included in the value function as stakeholders would not be able to know financial restrictions in priority setting of health interventions. Therefore, with lack of this knowledge, stakeholders would not be able to make adequate decisions on the cost or cost effectiveness criterion [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. Nonetheless, data on cost of intervention would still be obtained and combined with the MCDA results at the end, to enable ranking of health interventions [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eOther criteria were also considered. Affordability criterion was dropped as it was considered redundant in the presence of the cost of intervention criterion. feasibility (service, health products and technology requirements) and feasibility (health workforce requirements) were merged, and the new criterion was named health systems capacity requirements. Burden of disease and severity of disease criteria were considered as overlapping. Therefore, severity of disease was selected as it gave priority to the worse off according to the WHO consultative group on equity and UHC\u0026rsquo;s recommendation of taking into account the worse off when expanding service coverage [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Those with severe forms of illness are considered as having greater health needs [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. The levels for the criteria were obtained from multiple sources such as the evidence synthesis done by HBPAP when defining the UHC-EBP [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] and literature on MCDA applications in priority setting in LMICs that used discrete choice modelling [\u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e72\u003c/span\u003e, \u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e].\u003c/p\u003e\n\u003ch3\u003eStage four: wording\u003c/h3\u003e\n\u003cp\u003eKey informant semi-structured Interviews\u003c/p\u003e \u003cp\u003eStakeholders had diverse characteristics. Stakeholders had a median age of 38.50 years (interquartile range IQR 8.00 years), a majority came from medical backgrounds, had master\u0026rsquo;s degrees, and had a median overall work experience of 12.50 years (IQR 6.75 years) (see Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Most importantly, 70.00% had been part of a committee or group tasked with the development or revision of a benefit package.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eCharacteristics of semi structured interview respondents\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCharacteristic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValue\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGender\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e60.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFemale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e40.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAge (Years)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMean (standard deviation)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e41.90 (7.84)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedian (IQR)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e38.50 (8.00)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eEducation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted Doctorate degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted master\u0026rsquo;s degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e80.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted bachelor\u0026rsquo;s degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eProfession\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedical doctors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e60.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePublic Health Officers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNurses\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEpidemiologists\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCommunity Health professionals\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eStakeholder group\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePolicymakers \u0026amp; purchasers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e40.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrincipal investigators\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eProviders\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOthers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eOverall work experience (Years)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMean (standard deviation)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e16.60 (8.11)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedian (IQR)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e12.50 (6.75)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eStakeholder's current or previous involvement in health benefits package development before\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e30.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e70.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eStakeholders gave feedback on the criteria, levels, and definitions (see additional file 7). Effectiveness and safety were viewed as two criteria. Some stakeholders noted that interventions could be effective but not safe. Therefore, they suggested if the criterion could be split into two \u0026ldquo;effectiveness\u0026rdquo; and \u0026ldquo;safety\u0026rdquo;. The levels of the \u0026ldquo;health systems capacity requirements\u0026rdquo; criterion were not easily understood without the researcher expounding. For example, the \u0026ldquo;Above/Below average\u0026rdquo; naming of the levels were viewed as ambiguous as stakeholders noted they needed a reference value/average value. Therefore, stakeholders suggested if the levels could be renamed to \u0026ldquo;has capacity/does not have capacity\u0026rdquo; or \u0026ldquo;country has capacity/country does not have capacity\u0026rdquo;. Stakeholders were also prompted to choose one criterion between burden and severity of disease. Almost all of them preferred burden of disease over severity criterion. The main reason was that they prioritised conditions/diseases that affected many people arguing that interventions targeting a higher disease burden would have greater impact at population level.\u003c/p\u003e \u003cp\u003eDiscrete choice modelling pilot\u003c/p\u003e \u003cp\u003eThe stakeholders who completed the survey included six patient advocates, five principal investigators (academics and researchers), and 12 providers (see Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). The median age was 36.00 years (IQR 7 years), three-quarters were female, and 37.50% had completed a bachelor\u0026rsquo;s degree. Overall, the stakeholders had a median work experience of 9.00 years (IQR 8.25 years) and 20.83% had participated in HTA processes such as the development of essential packages for health, health benefit packages, or essential medicines lists.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eSociodemographic characteristics of DCE pilot stakeholders\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCharacteristic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValue\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGender\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFemale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e75.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e25.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAge (Years)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMean (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e37.00 (7.17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedian (IQR)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e36.00 (7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eStakeholder group\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePatients\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e25.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrincipal investigators\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.83%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eProviders\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e54.17%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e13\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eEducation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted Doctorate degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.17%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted master\u0026rsquo;s degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.83%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted bachelor\u0026rsquo;s degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e37.50%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted Diploma/Higher Diploma\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e33.33%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCompleted Certificate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.17%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eProfession\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedical doctor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e16.67%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNurse\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.83%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eClinical officer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.83%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHealth economist\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e8.33%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEpidemiologist\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.17%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOther\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e29.17%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eType of organisation the stakeholder works for\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCounty department of health\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.35%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFaith-based/NGO health facility\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e8.70%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrivate health facility\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e8.70%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePublic health facility\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e34.78%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResearch institution\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e21.74%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePatient advocacy group\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e21.74%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e23\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eOverall work experience (Years)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMean (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.49 (7.19)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedian (IQR)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e9.00 (8.25)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStakeholder\u0026rsquo;s current or previous involvement in HTA e.g., health benefits package development before\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e79.17%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e20.83%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIQR: Interquartile range. N: number of observations. SD: Standard deviation.\u003c/p\u003e \u003cp\u003eAll but one of the coefficients of the criteria had expected signs (see Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e ). Stakeholders significantly preferred interventions that were of greater effectiveness and were safer, reduced the financial burden of paying out of pocket, addressed diseases that mainly affected the poor, targeted severe diseases, and were more congruent with existing priorities.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eMain effects MNL model\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCriteria\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCoefficient\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e95% CI\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eP-value\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEffectiveness and safety of intervention\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSlightly effective and safe\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRef. (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModerately effective and safe\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.60\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.33, 1.13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.027\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHighly effective and safe\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.86\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.61, 1.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCatastrophic health expenditure\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDoes not reduce the financial burden of paying out of pocket\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRef. (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eReduces the financial burden of paying out of pocket\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.22\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.01, 1.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eHealth systems capacity requirements\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBelow national average\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRef. (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAbove national average\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-0.21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e-0.49, 0.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.457\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eEquity\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDisease mainly affects the well off\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRef. (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDisease mainly affects the poor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.83\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.60, 1.28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSeverity of disease\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMild\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRef. (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModerate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e-0.08, 0.72\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.474\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSevere\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.63\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.40, 1.08\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.006\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCongruence with existing priorities\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLow Priority\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRef. (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedium Priority\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.68\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.35, 1.33\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.039\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHigh Priority\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.86\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.62, 1.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAlternative specific constant (Alternative A)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-0.05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e-0.22, 0.28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.755\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eModel fit statistics\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLog likelihood (final)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-136.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRho-square (C)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdj. Rho-square (C)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAkaike Information Criterion\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e293.14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBayesian Information Criterion\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e329.77\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNumber of modelled outcomes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e288.00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNumber of decision makers (n)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e24.00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAdj: Adjusted. CI: Confidence Interval. P-value are two sided.\u003c/p\u003e \u003cp\u003eInterestingly, the sign of the health systems capacity requirements criterion was negative, indicating that even though the capacity of implementing the intervention was below national average, stakeholders would still prefer the intervention over one that had capacity above national average. However, this was not statistically significant. The alternative specific constant was set at alternative A. The sign was negative and not statistically significant which signified absence of left to right bias.\u003c/p\u003e \u003cp\u003eThe relative importance estimates were computed. Catastrophic health expenditure was the most important criterion followed by congruence with existing priorities (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e and additional file 6). Health systems capacity requirements was the least important criteria.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThere were some interesting comments in the think-aloud exercise. Stakeholders noted that the catastrophic health expenditure levels \u0026ldquo;reduces/does not reduce the financial burden of paying out of pocket\u0026rdquo; seemed similar or related to the equity criterion levels \u0026ldquo;disease mainly affects well-off/poor\u0026rdquo;. Furthermore, another respondent felt that the levels of the catastrophic health expenditure criterion went against the main tenets of UHC of ensuring financial risk protection. They felt that all interventions being considered for inclusion should reduce out-of-pocket expenses for the user. Therefore, the levels of that criterion did not make sense.\u003c/p\u003e \u003cp\u003eStakeholders also gave comments on the effectiveness and safety, health systems capacity requirements, congruency with existing priorities, and equity criteria. Some noted that effectiveness and safety were bundled together, and the safety part of the level was not changing, as an intervention could be highly effective but not safe. Stakeholders also found it difficult to understand the levels of the health systems capacity requirements criterion \u0026ldquo;Above/below national average\u0026rdquo;. They would have liked to know the value of the national average. Therefore, they preferred using \u0026ldquo;above capacity\u0026rdquo; or \u0026ldquo;has capacity\u0026rdquo; or \u0026ldquo;capacity to provide service\u0026rdquo; categorisation rather than \u0026ldquo;above/below national average\u0026rdquo;. Furthermore, the word \"requirements\" in the criterion was confusing and could be interpreted as capacity required in future rather than capacity available at present for the intervention to be successfully rolled out. Nonetheless, congruence with existing priorities criterion was well understood and the levels \u0026ldquo;high priority\u0026rdquo;, \u0026ldquo;medium priority\u0026rdquo;, and \u0026ldquo;low priority\u0026rdquo; could easily be interpreted. Lastly, the equity criterion was well understood according to feedback from stakeholders. However, others noted that conditions or diseases could affect both the poor and the well-off. Therefore, the levels would need rephrasing. Some stakeholders suggested if the researchers could try and measure equity in access for the intervention using other metrics, rather than dichotomising using socio-economic status (poor/rich) and prevalence.\u003c/p\u003e \u003cp\u003eStakeholders gave feedback on the general survey design. They found 12 choice tasks easily manageable. One respondent noted if they could have reference to the criteria definitions while completing the choice tasks, and if this could be provided in form of a paper sheet. The definitions of the criteria were on the previous page. Therefore, they could not go back due to the randomisation algorithm. This was evident as some had to seek clarification on the definitions of some of the criteria from the researcher while answering the choice tasks.\u003c/p\u003e \u003cp\u003eFinal list of criteria and levels\u003c/p\u003e \u003cp\u003eThe final list of six criteria levels (see Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e) were developed using results and feedback from the semi-structured interviews and the pilot discrete choice modelling survey. Severity of disease criterion was dropped, and burden of disease included in the final list according to the feedback from the qualitative study. The two criteria could not all be included as they overlapped. The levels of burden of disease were named \u0026ldquo;high burden\u0026rdquo;, \u0026ldquo;medium burden\u0026rdquo;, and \u0026ldquo;low burden\u0026rdquo;. Congruence with existing priorities was retained in the final list as stakeholders noted it was salient. It was the second most important criterion among the stakeholders who participated in the pilot discrete choice modelling survey (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Furthermore, safety was split from the effectiveness and safety criterion according to feedback from the semi-structured interviews and pilot discrete choice modelling survey. The new criteria were called \u0026ldquo;effectiveness of intervention\u0026rdquo; and \u0026ldquo;safety of intervention\u0026rdquo;. To limit the number of criteria to five or six as earlier explained, safety of intervention criterion was dropped from the study. Though cost of intervention criterion was included in the final list, it will not be used in scoring and weighting stages of the quantitative MCDA as explained earlier. It will be used later to calculate cost per value metric to aid in ranking health interventions in a league table [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eFinal attributes, levels, and lay definitions.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCriteria\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLevels\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLay definition\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBurden of disease\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0. Low burden\u003c/p\u003e \u003cp\u003e1. Medium burden\u003c/p\u003e \u003cp\u003e2. High burden\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ldquo;Whether the service addresses a condition/disease that affects many Kenyans.\u0026rdquo; [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, p.13].\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCongruence with existing priorities\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0. Low priority\u003c/p\u003e \u003cp\u003e1. Medium priority\u003c/p\u003e \u003cp\u003e2. High priority\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ldquo;Whether the service is in line with constitution, prevailing laws and prevailing health sector policies and priorities as further investments and policies are made.\u0026rdquo; [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, p.13]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEffectiveness of intervention\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0. Slightly effective\u003c/p\u003e \u003cp\u003e1. Moderately effective\u003c/p\u003e \u003cp\u003e2. Highly effective\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ldquo;Whether the service delivers an improvement in health status, reduction in mortality or improvement in quality of life.\u0026rdquo; [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, p.13]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEquity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0. Condition/disease mainly affects the well-off\u003c/p\u003e \u003cp\u003e1. Condition/disease mainly affects the poor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ldquo;Whether the service addresses the disparities in access and utilisation of needed health services and health status of Kenyans.\u0026rdquo; [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, p.13]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHealth systems capacity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0. Country has inadequate capacity\u003c/p\u003e \u003cp\u003e1. Country has adequate capacity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ldquo;Whether the service can be provided to Kenyans based on existing health system capacity in terms of human resources, medicines, supplies, and other service provision requirements.\u0026rdquo; [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, p.13]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCost of intervention\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCriterion will not be included in the value function (scoring and weighting stage) of the quantitative MCDA. It will be considered at the end of the MCDA where a cost per value metric will be calculated.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eUnit cost of health intervention\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eCriteria and definitions were derived from HBPAP\u0026rsquo;s report [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. HBPAP had obtained the criteria and definitions from Tromp N and Baltussen R [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eAdjustments were made to other criteria as well. Though catastrophic health expenditure criterion was the most important according to the pilot discrete choice modelling results (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e), it was dropped from the final list because respondents viewed it as a cost attribute. Furthermore, cost is used in the calculation of catastrophic health expenses. There was already a cost of intervention criterion in the list, and it would overlap with the catastrophic health expenditure criterion. Moreover, including catastrophic health expenses as a criterion to be used in scoring and weighting of health interventions, would be like including cost in the value function [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. Additionally, the word \u0026ldquo;requirements\u0026rdquo; was dropped from the health systems capacity requirements criterion according to feedback from the qualitative study and pilot discrete choice modelling survey. The levels of this criterion were renamed to \u0026ldquo;country has adequate capacity\u0026rdquo; and \u0026ldquo;country has inadequate capacity\u0026rdquo;. Nonetheless, equity criterion was maintained in the final list as it represented a key aspect of UHC that called for priority for the worse-off [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFinally, all criteria were categorical, and levels were made qualitative. The advantage was that they could be mapped to any set of health interventions with different performance cut-offs. This meant that the future quantitative MCDA results could be used in contexts beyond Kenya, or with any set of health interventions. The disadvantage of qualitative levels was they could be open to misinterpretation if not well understood [\u003cspan citationid=\"CR93\" class=\"CitationRef\"\u003e93\u003c/span\u003e]. Therefore, to minimise bias in the future steps of the study, making sure that stakeholders clearly understand the criteria and level would be crucial.\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eThe study set out to identify and select criteria to be used in an exploratory quantitative MCDA study for health intervention priority setting in Kenya employing the country\u0026rsquo;s UHC benefit package as an example. A four-stage process was used to accomplish the objective that involved raw data collection, data reduction, removing inappropriate criteria, and wording [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. A list of 10 criteria were selected from the criteria identified by HBPAP and used as the starting point for this study [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. Researchers (MO, and two other academics) commented on the criteria to remove inappropriate ones and refined them, while taking into consideration factors such as availability of data, capability of being traded, completeness, non-overlap, non-redundancy, plausibility, preference independence, and salience [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e]. Semi structured interviews and a pilot discrete choice modelling survey with stakeholders were used to further refine the criteria, levels, and definitions. Finally, six criteria and levels were agreed upon i.e., burden of disease, congruence with existing priorities, cost of intervention, effectiveness of intervention, equity, and health systems capacity [\u003cspan citationid=\"CR94\" class=\"CitationRef\"\u003e94\u003c/span\u003e]. However, cost of intervention will not be used in the scoring and weighting stage of the quantitative MCDA [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. It will be incorporated in the final stages to create a cost per value metric to aid in ranking health interventions [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe final criteria selected (burden of disease, congruence with existing priorities, cost of intervention, effectiveness of intervention, equity, and health systems capacity) were similar to those used in other studies and settings globally [\u003cspan citationid=\"CR94\" class=\"CitationRef\"\u003e94\u003c/span\u003e]. Burden of disease, effectiveness of intervention, equity, and cost of intervention are some of the criteria advocated for in prioritising national health plans, policies, and strategies by a WHO report [\u003cspan citationid=\"CR95\" class=\"CitationRef\"\u003e95\u003c/span\u003e]. Baltussen R, Mwalim O, Blanchet K, Carballo M, Eregata GT, Hailu A, Huda M, Jama M, Johansson KA, Reynolds T, Raza W, Mallender J and Majdzadeh R [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] described the experience of six countries in defining or refining UHC essential packages for health services found that similar criteria were used. Burden of disease was used in Ethiopia, Somalia, Sudan, and Zanzibar. Effectiveness was used in Afghanistan, Pakistan, and Sudan. Equity was used in Afghanistan, Ethiopia, Pakistan, and Zanzibar. Feasibility (health systems capacity) was used in Afghanistan, Pakistan, Somalia, and Sudan [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Additionally, a scoping review [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] on criteria used in health benefit package design globally found that, effectiveness was the second most used criteria after cost-effectiveness, equity was fourth while burden of disease was the fifth most used. Cost of intervention and feasibility (equivalent to health systems capacity) were also some of the criteria found by the study [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eMCDA applications in health intervention priority setting have also incorporated similar criteria to those derived in this study. A study exploring the feasibility of MCDA for reimbursement decisions in Colombia [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] applied the EVIDEM (Evidence and Value: Impact on DEcision-Making) framework and used criteria such as size of population affected by disease (burden of disease), improvement of efficacy/effectiveness, and attention to vulnerable groups of population (equity). Effectiveness was used in an MCDA study prioritising interventions for HIV/AIDS in Thailand [\u003cspan citationid=\"CR73\" class=\"CitationRef\"\u003e73\u003c/span\u003e]. In Pakistan, effectiveness of alternative and manpower required for implementation (health systems capacity) were used in an MCDA for prioritising interventions for preventing dengue [\u003cspan citationid=\"CR96\" class=\"CitationRef\"\u003e96\u003c/span\u003e]. Burden of disease related criteria (attributable burden and the number of people to be potentially affected by intervention) and an equity (prevalence differences between income levels) criterion were used in an MCDA in Iran that prioritised interventions targeting NCDs [\u003cspan citationid=\"CR97\" class=\"CitationRef\"\u003e97\u003c/span\u003e]. Another Iranian MCDA study on prioritisation of rehabilitative interventions into the Iranian benefit package used four similar criteria i.e., burden of disease, cost, effectiveness, and feasibility [\u003cspan citationid=\"CR98\" class=\"CitationRef\"\u003e98\u003c/span\u003e]. Therefore, criteria in this study have been widely used.\u003c/p\u003e \u003cp\u003eBurden of disease ended up in the final list of criteria over severity of disease as Kenyan stakeholders preferred it. Both criteria could not be included in the final list of criteria as they overlapped. Interestingly, severity of disease is still a common criterion in MCDA studies in priority setting. It is one of the criteria that captures priority for the worse off (together with equity) and is advocated for use in expanding service coverage for UHC by the WHO consultative group on equity and UHC [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Severity has been used in MCDA studies for health intervention priority setting in Bulgaria [\u003cspan citationid=\"CR99\" class=\"CitationRef\"\u003e99\u003c/span\u003e], China [\u003cspan citationid=\"CR75\" class=\"CitationRef\"\u003e75\u003c/span\u003e], C\u0026ocirc;te d'Ivoire [\u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e], South Africa [\u003cspan citationid=\"CR100\" class=\"CitationRef\"\u003e100\u003c/span\u003e], and Uganda [\u003cspan citationid=\"CR101\" class=\"CitationRef\"\u003e101\u003c/span\u003e] among others. In an MCDA priority setting study in Kazakhstan, burden of disease had been selected as one of the decision making criteria [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. However, the research team advised the committee on the cons of using burden of disease as it would have led to interventions that addressed more common conditions being prioritised over those targeting rare conditions [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. The committee were in agreement with this [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. In our MCDA study, the wishes of the stakeholders prevailed. In future studies, using the approach adopted in Kazakhstan could be an option.\u003c/p\u003e \u003cp\u003eEquity is another important criterion in priority setting for UHC, as it advances priority for the worse off agenda [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. The WHO consultative group on equity and UHC recommends considering the worse off when expanding service coverage, especially the poor and those who live in rural settings [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. The equity criterion in our study focused on the socioeconomic aspect of equity i.e., consideration for the poor versus the well-off (rich). This poor/rich classification has been used in other MCDA studies that employed discrete choice modelling such as in C\u0026ocirc;te d'Ivoire [\u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e], Ghana [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e], and Nepal [\u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e72\u003c/span\u003e]. Nonetheless, some Kenyan stakeholders had suggested whether other aspects of equity could be pursued such as gender or geographic (rural/urban). The socioeconomic aspect of equity, specifically the poor/well-off categorisation, was adopted for our study as it is one of the aspects advocated for by the WHO consultative group on equity and UHC [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e\n\u003ch3\u003eStrengths\u003c/h3\u003e\n\u003cp\u003eThe study had several strengths. First, a systematic process was used to identify and select priority setting criteria. Following a four-step process ensured a structured and transparent process while keeping in line with ISPOR task force MCDA recommendations [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] and discrete choice modelling guidance [\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e]. Secondly, stakeholders were involved in the semi structured interviews and pilot discrete choice modelling survey where they provided feedback that helped in refining and wording of criteria and levels. Involving stakeholders is critical and helps to ensure transparency and legitimacy of the process [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR102\" class=\"CitationRef\"\u003e102\u003c/span\u003e]. Third, criteria that had been identified by HBPAP were used as the starting point [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. The main advantage of this was that our study built on local existing processes rather than formulating things that would be viewed as foreign.\u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eLimitations\u003c/h2\u003e \u003cp\u003eThis study had several limitations. First, in analysis of the semi-structured interviews, one researcher (MO) applied the codes to the data. It would have been better to have two independent coders for the qualitative data. Furthermore, MO compiled the final list of criteria and levels using feedback from stakeholders from the semi-structured interviews and pilot discrete choice modelling survey. A better approach would have been to involve a larger panel of stakeholders or researchers in this final step. Nonetheless, MO just analysed the data from semi-structured interviews and pilot discrete choice modelling survey and incorporated the feedback to shape the criteria and levels. Second, the sample size for the pilot discrete choice modelling survey was small (24 stakeholders). However, the results were valuable in shaping the final criteria and levels and in generating priors to be used for the main discrete choice modelling survey in the scoring and weighting phase of the quantitative MCDA study. Third, the levels of the final criteria were qualitative (i.e., high, medium, low). The disadvantage was that they could be prone to misinterpretation by stakeholders if not well understood [\u003cspan citationid=\"CR93\" class=\"CitationRef\"\u003e93\u003c/span\u003e]. Conversely, the advantage is that they can be mapped to any set of health interventions using the performance matrix and new cut off points could always be defined.\u003c/p\u003e \u003c/div\u003e"},{"header":"CONCLUSION","content":"\u003cp\u003eIn conclusion, the paper focussed on identifying and selecting criteria to be used in scoring and weighting stages of an exploratory quantitative MCDA process in Kenya. A systematic and transparent four stage process was used, and six criteria were selected. Cost of intervention was added but would not be used in the scoring and weighting stage. It is important for priority setting criteria to be identified and selected using systematic and transparent processes while involving stakeholders [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. This enhances transparency, legitimacy, and fairness [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e].\u003c/p\u003e"},{"header":"LIST OF ABBREVIATIONS","content":"\u003cp\u003eA4R\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Accountability for reasonableness\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCAPI\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Computer assisted personal interviewing\u003c/p\u003e\n\u003cp\u003eCEA\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Cost Effectiveness Analysis\u003c/p\u003e\n\u003cp\u003eDFT\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Decision Field Theory\u003c/p\u003e\n\u003cp\u003eEVIDEM\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Evidence and Value: Impact on DEcision-Making\u003c/p\u003e\n\u003cp\u003eFGDs\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Focus Group Discussions\u003c/p\u003e\n\u003cp\u003eHBPAP\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Health Benefits Package Advisory Panel\u003c/p\u003e\n\u003cp\u003eHTA\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Health Technology Assessment\u003c/p\u003e\n\u003cp\u003eICFs\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Informed Consent Forms\u003c/p\u003e\n\u003cp\u003eIQR\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Interquartile range\u003c/p\u003e\n\u003cp\u003eKEMRI\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Kenya Medical Research Institute\u003c/p\u003e\n\u003cp\u003eLMICs\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Low- and Middle-Income Countries\u003c/p\u003e\n\u003cp\u003eMCDA\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Multi-Criteria Decision Analysis\u003c/p\u003e\n\u003cp\u003eMNL\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Multinomial Logit Model\u003c/p\u003e\n\u003cp\u003eNGOs\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Non-governmental organisations\u003c/p\u003e\n\u003cp\u003eNGT\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Nominal Group Technique\u003c/p\u003e\n\u003cp\u003eNHIF\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;National Hospital Insurance Fund\u003c/p\u003e\n\u003cp\u003eOxTREC\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Oxford Tropical Research Ethics Committee\u003c/p\u003e\n\u003cp\u003ePBMA\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Programme Budgeting and Marginal Analysis\u003c/p\u003e\n\u003cp\u003eRRM\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Random Regret Minimisation\u003c/p\u003e\n\u003cp\u003eRUM\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Random Utility Maximisation\u003c/p\u003e\n\u003cp\u003eSD\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Standard Deviation\u003c/p\u003e\n\u003cp\u003eSHA\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Social Health Authority\u003c/p\u003e\n\u003cp\u003eSHIF\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Social Health Insurance Fund\u003c/p\u003e\n\u003cp\u003eUHC\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Universal Health Coverage\u003c/p\u003e\n\u003cp\u003eUHC-EBP \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Universal Health Coverage Essential benefit package\u003c/p\u003e\n\u003cp\u003eWHO World Health Organisation\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eApproval to conduct the exploratory quantitative MCDA study in Kenya, which this publication forms part of, was granted by the Oxford Tropical Research Ethics Committee (OxTREC) (Reference: 564-20) and the Kenya Medical Research Institute / Scientific and Ethics Review Unit (no. KEMRI/CGMR-C/210/4095). Furthermore, stakeholders who participated in the semi-structured interviews and discrete choice modelling surveys signed informed consent forms (ICFs) in paper format and/or on Redcap software before participating.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConsent to publish findings of the study was obtained from the participants.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eChoice data generated and used in the discrete choice analysis is available open access from the KEMRI-Wellcome Trust Research Programme Harvard Dataverse\u0026nbsp;\u003ca href=\"https://doi.org/10.7910/DVN/CUTGVF\"\u003ehttps://doi.org/10.7910/DVN/CUTGVF\u003c/a\u003e [92]. Data from the semi-structured interviews could not be anonymised and are therefore not available open access.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study received funding from the University of Oxford (Clarendon Fund and Oxford Population Health), University College Oxford (Oxford Radcliffe Graduate Scholarship), International Decision Support Initiative (Bill and Melinda Gates Foundation), and KEMRI-Wellcome Trust Research Programme. This study was also funded in part, by the Wellcome Trust [DEL-15-003] and the UK Foreign, Commonwealth \u0026amp; Development Office, with support from the Developing Excellence in Leadership, Training and Science in Africa (DELTAS Africa) programme. For open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. The funders had no role in conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceptualisation: MO \u0026amp; EB. Data Curation: MO. Formal Analysis: MO. Funding Acquisition: MO \u0026amp; EB. Investigation: MO. Methodology: MO \u0026amp; EB. Project Administration: MO. Resources: MO. Supervision: EB. Software: MO. Validation: MO \u0026amp; EB. Visualization: MO. Writing \u0026ndash; Original Draft Preparation: MO. Writing \u0026ndash; Review \u0026amp; Editing: MO \u0026amp; EB.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEvelyn Kabia (EK) of the KEMRI-Wellcome Trust Research Programme is acknowledged for reviewing the coding framework. James Bukosia (JB) of KEMRI-Wellcome Trust Research Programme developed the discrete choice modelling survey questionnaire on RedCap and curated the dataset on Harvard Dataverse. Finally, MO thanks his D. Phil thesis examiners for their valuable comments during his viva voce and everyone who contributed to the study.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eWorld Health Organization: \u003cem\u003eMaking fair choices on the path to universal health coverage: final report of the WHO consultative group on equity and universal health coverage.\u003c/em\u003e Geneva: World Health Organization; 2014.\u003c/li\u003e\n\u003cli\u003eOttersen T, Norheim OF: \u003cstrong\u003eSetting Priorities in the Pursuit of Universal Health Coverage.\u003c/strong\u003e In \u003cem\u003eGlobal Health Priority-Setting: Beyond Cost-Effectiveness.\u003c/em\u003e Edited by Norheim OF, Emanuel EJ, Millum J: Oxford University Press; 2019: 0\u003c/li\u003e\n\u003cli\u003eHayati R, Bastani P, Kabir MJ, Kavosi Z, Sobhani G: \u003cstrong\u003eScoping literature review on the basic health benefit package and its determinant criteria.\u003c/strong\u003e \u003cem\u003eGlobalization and Health \u003c/em\u003e2018, \u003cstrong\u003e14:\u003c/strong\u003e26.\u003c/li\u003e\n\u003cli\u003eKaur G, Prinja S, Lakshmi PVM, Downey L, Sharma D, Teerawattananon Y: \u003cstrong\u003eCriteria Used for Priority-Setting for Public Health Resource Allocation in Low- and Middle-Income Countries: A Systematic Review.\u003c/strong\u003e \u003cem\u003eInternational Journal of Technology Assessment in Health Care \u003c/em\u003e2019, \u003cstrong\u003e35:\u003c/strong\u003e474-483.\u003c/li\u003e\n\u003cli\u003eBaltussen R, Mwalim O, Blanchet K, Carballo M, Eregata GT, Hailu A, Huda M, Jama M, Johansson KA, Reynolds T, et al: \u003cstrong\u003eDecision-making processes for essential packages of health services: experience from six countries.\u003c/strong\u003e \u003cem\u003eBMJ Global Health \u003c/em\u003e2023, \u003cstrong\u003e8:\u003c/strong\u003ee010704.\u003c/li\u003e\n\u003cli\u003eBarasa EW, Cleary S, Molyneux S, English M: \u003cstrong\u003eSetting healthcare priorities: a description and evaluation of the budgeting and planning process in county hospitals in Kenya.\u003c/strong\u003e \u003cem\u003eHealth Policy and Planning \u003c/em\u003e2017, \u003cstrong\u003e32:\u003c/strong\u003e329-337.\u003c/li\u003e\n\u003cli\u003eRepublic of Kenya: \u003cstrong\u003eAdvisory Panel for the Design and Assessment of the Kenya UHC Essential Benefit Package (UHC-EBP).\u003c/strong\u003e In \u003cem\u003eThe Kenya Gazette\u003c/em\u003e, vol. CXX \u0026mdash;No. 69. pp. 1780-1781. Nairobi: The Government Printer; 2018:1780-1781.\u003c/li\u003e\n\u003cli\u003eMbau R, Oliver K, Vassall A, Gilson L, Barasa E: \u003cstrong\u003eA qualitative evaluation of priority-setting by the Health Benefits Package Advisory Panel in Kenya.\u003c/strong\u003e \u003cem\u003eHealth Policy and Planning \u003c/em\u003e2023, \u003cstrong\u003e38:\u003c/strong\u003e49-60.\u003c/li\u003e\n\u003cli\u003eMinistry of Health Kenya: \u003cem\u003eReport on the UHC Essential Benefits Package.\u003c/em\u003e Nairobi, Kenya: Universal Health Coverage Health Benefits Package Advisory Panel; 2018.\u003c/li\u003e\n\u003cli\u003eFacility Improvement Financing Act: \u003cstrong\u003eFacility Improvement Financing.\u003c/strong\u003e In \u003cem\u003e14\u003c/em\u003e; 2023.\u003c/li\u003e\n\u003cli\u003ePrimary Health Care Act: \u003cstrong\u003ePrimary Health Care Act.\u003c/strong\u003e In \u003cem\u003e13\u003c/em\u003e; 2023.\u003c/li\u003e\n\u003cli\u003eThe Social Health Insurance Act: \u003cstrong\u003eSocial Health Insurance.\u003c/strong\u003e In \u003cem\u003e16 of 2023\u003c/em\u003e; 2023.\u003c/li\u003e\n\u003cli\u003eBarasa E: \u003cstrong\u003eThe good, bad of Social Health Insurance Bill.\u003c/strong\u003e In \u003cem\u003eBusiness Daily\u003c/em\u003e; 2023.\u003c/li\u003e\n\u003cli\u003eThe Social Health Insurance (General) Regulations: \u003cstrong\u003eThe Social Health Insurance (General) Regulations.\u003c/strong\u003e 2023.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eKenya health insurance fund: Boost for President William Ruto as court lifts ban \u003c/strong\u003e[https://www.bbc.co.uk/news/world-africa-68027893]\u003c/li\u003e\n\u003cli\u003eBaltussen R, Niessen L: \u003cstrong\u003ePriority setting of health interventions: the need for multi-criteria decision analysis.\u003c/strong\u003e \u003cem\u003eCost Effectiveness and Resource Allocation \u003c/em\u003e2006, \u003cstrong\u003e4:\u003c/strong\u003e14.\u003c/li\u003e\n\u003cli\u003eDaniels N, Sabin JE, Daniels N, Sabin JE: \u003cstrong\u003e43Accountability for Reasonableness.\u003c/strong\u003e In \u003cem\u003eSetting Limits Fairly: Can we learn to share medical resources?\u003c/em\u003e : Oxford University Press; 2002: 0\u003c/li\u003e\n\u003cli\u003eDaniels N, Sabin JE: \u003cem\u003eSetting Limits Fairly: Learning to share resources for health.\u003c/em\u003e 2nd edn. New York: Oxford University Press; 2008.\u003c/li\u003e\n\u003cli\u003eOttersen T, Norheim OF: \u003cstrong\u003e293The Uses of Burden-of-Disease Data for Priority Setting.\u003c/strong\u003e In \u003cem\u003eMeasuring the Global Burden of Disease: Philosophical Dimensions.\u003c/em\u003e Edited by Eyal N, Hurst SA, Murray CJL, Schroeder SA, Wikler D, Eyal N, Hurst SA, Murray CJL, Schroeder SA, Wikler D: Oxford University Press; 2020: 0\u003c/li\u003e\n\u003cli\u003eMusgrove P, Fox-Rushby J: \u003cstrong\u003eCost-Effectiveness Analysis for Priority Setting.\u003c/strong\u003e In \u003cem\u003eDisease Control Priorities in Developing Countries.\u003c/em\u003e Edited by Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB, Jha P, Mills A, Musgrove P. Washington, DC: World Bank and Oxford University Press; 2006\u003c/li\u003e\n\u003cli\u003eO\u0026apos;Rourke B, Oortwijn W, Schuller T: \u003cstrong\u003eThe new definition of health technology assessment: A milestone in international collaboration.\u003c/strong\u003e \u003cem\u003eInternational Journal of Technology Assessment in Health Care \u003c/em\u003e2020, \u003cstrong\u003e36:\u003c/strong\u003e187-190.\u003c/li\u003e\n\u003cli\u003eMitton CR, Donaldson C: \u003cstrong\u003eSetting priorities and allocating resources in health regions: lessons from a project evaluating program budgeting and marginal analysis (PBMA).\u003c/strong\u003e \u003cem\u003eHealth Policy \u003c/em\u003e2003, \u003cstrong\u003e64:\u003c/strong\u003e335-348.\u003c/li\u003e\n\u003cli\u003eMitton C, Donaldson C: \u003cstrong\u003eHealth care priority setting: principles, practice and challenges.\u003c/strong\u003e \u003cem\u003eCost Effectiveness and Resource Allocation \u003c/em\u003e2004, \u003cstrong\u003e2:\u003c/strong\u003e3.\u003c/li\u003e\n\u003cli\u003eKapiriri L, Razavi D: \u003cstrong\u003eHow have systematic priority setting approaches influenced policy making? A synthesis of the current literature.\u003c/strong\u003e \u003cem\u003eHealth Policy \u003c/em\u003e2017, \u003cstrong\u003e121:\u003c/strong\u003e937-946.\u003c/li\u003e\n\u003cli\u003eBaltussen R, Marsh K, Thokala P, Diaby V, Castro H, Cleemput I, Garau M, Iskrov G, Olyaeemanesh A, Mirelman A, et al: \u003cstrong\u003eMulticriteria Decision Analysis to Support Health Technology Assessment Agencies: Benefits, Limitations, and the Way Forward.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2019, \u003cstrong\u003e22:\u003c/strong\u003e1283-1288.\u003c/li\u003e\n\u003cli\u003eCastro Jaramillo HE, Goetghebeur M, Moreno-Mattar O: \u003cstrong\u003eTESTING MULTI-CRITERIA DECISION ANALYSIS FOR MORE TRANSPARENT RESOURCE-ALLOCATION DECISION MAKING IN COLOMBIA.\u003c/strong\u003e \u003cem\u003eInternational Journal of Technology Assessment in Health Care \u003c/em\u003e2016, \u003cstrong\u003e32:\u003c/strong\u003e307-314.\u003c/li\u003e\n\u003cli\u003eJehu-Appiah C, Baltussen R, Acquah C, Aikins M, Amah d\u0026apos;Almeida S, Bosu WK, Koolman X, Lauer J, Osei D, Adjei S: \u003cstrong\u003eBalancing Equity and Efficiency in Health Priorities in Ghana: The Use of Multicriteria Decision Analysis.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2008, \u003cstrong\u003e11:\u003c/strong\u003e1081-1087.\u003c/li\u003e\n\u003cli\u003eOortwijn W, Surgey G, Novakovic T, Baltussen R, Kosherbayeva L: \u003cstrong\u003eThe Use of Evidence-Informed Deliberative Processes for Health Benefit Package Design in Kazakhstan.\u003c/strong\u003e \u003cem\u003eInternational Journal of Environmental Research and Public Health \u003c/em\u003e2022, \u003cstrong\u003e19:\u003c/strong\u003e11412.\u003c/li\u003e\n\u003cli\u003eRadaelli G, Lettieri E, Masella C, Merlino L, Strada A, Tringali M: \u003cstrong\u003eIMPLEMENTATION OF EUNETHTA CORE MODEL\u0026reg; IN LOMBARDIA: THE VTS FRAMEWORK.\u003c/strong\u003e \u003cem\u003eInternational Journal of Technology Assessment in Health Care \u003c/em\u003e2014, \u003cstrong\u003e30:\u003c/strong\u003e105-112.\u003c/li\u003e\n\u003cli\u003eDefechereux T, Paolucci F, Mirelman A, Youngkong S, Botten G, Hagen TP, Niessen LW: \u003cstrong\u003eHealth care priority setting in Norway a multicriteria decision analysis.\u003c/strong\u003e \u003cem\u003eBMC Health Services Research \u003c/em\u003e2012, \u003cstrong\u003e12:\u003c/strong\u003e39.\u003c/li\u003e\n\u003cli\u003eYoungkong S, Baltussen R, Tantivess S, Mohara A, Teerawattananon Y: \u003cstrong\u003eMulticriteria Decision Analysis for Including Health Interventions in the Universal Health Coverage Benefit Package in Thailand.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2012, \u003cstrong\u003e15:\u003c/strong\u003e961-970.\u003c/li\u003e\n\u003cli\u003eBelton V, Stewart T: \u003cem\u003eMultiple Criteria Decision Analysis: An Integrated Approach.\u003c/em\u003e Norwell, Massachusetts: Kluwer Academic Publishers; 2002.\u003c/li\u003e\n\u003cli\u003eRegier DA, Peacock S: \u003cstrong\u003eTheoretical Foundations of MCDA.\u003c/strong\u003e In \u003cem\u003eMulti-Criteria Decision Analysis to Support Healthcare Decisions.\u003c/em\u003e Edited by Marsh K, Goetghebeur M, Thokala P, Baltussen R. Cham: Springer International Publishing; 2017: 9-28\u003c/li\u003e\n\u003cli\u003eThokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, Longrenn T, Mussen F, Peacock S, Watkins J, Ijzerman M: \u003cstrong\u003eMultiple Criteria Decision Analysis for Health Care Decision Making; An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2016, \u003cstrong\u003e19:\u003c/strong\u003e1-13.\u003c/li\u003e\n\u003cli\u003eMarsh K, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kal\u0026oacute; Z, L\u0026ouml;nngren T, Mussen F, Peacock S, Watkins J, Devlin N: \u003cstrong\u003eMultiple Criteria Decision Analysis for Health Care Decision Making\u0026mdash;Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2016, \u003cstrong\u003e19:\u003c/strong\u003e125-137.\u003c/li\u003e\n\u003cli\u003eHansen P, Devlin N: \u003cstrong\u003eMulti-Criteria Decision Analysis (MCDA) in Healthcare Decision-Making.\u003c/strong\u003e Oxford University Press; 2019.\u003c/li\u003e\n\u003cli\u003eHelter TM, Boehler CEH: \u003cstrong\u003eDeveloping attributes for discrete choice experiments in health: a systematic literature review and case study of alcohol misuse interventions.\u003c/strong\u003e \u003cem\u003eJournal of Substance Use \u003c/em\u003e2016, \u003cstrong\u003e21:\u003c/strong\u003e662-668.\u003c/li\u003e\n\u003cli\u003eSteelFisher GK: \u003cstrong\u003eIncluding the public perspective in health-related MCDA: ideas from the field of public opinion research and polling.\u003c/strong\u003e \u003cem\u003eCost Effectiveness and Resource Allocation \u003c/em\u003e2018, \u003cstrong\u003e16:\u003c/strong\u003e39.\u003c/li\u003e\n\u003cli\u003eConcannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, Conway PH, Daudelin D, Morrato EH, Leslie LK: \u003cstrong\u003eA New Taxonomy for Stakeholder Engagement in Patient-Centered Outcomes Research.\u003c/strong\u003e \u003cem\u003eJournal of General Internal Medicine \u003c/em\u003e2012, \u003cstrong\u003e27:\u003c/strong\u003e985-991.\u003c/li\u003e\n\u003cli\u003eObadha M, Barasa E, Kazungu J, Abiiro GA, Chuma J: \u003cstrong\u003eAttribute development and level selection for a discrete choice experiment to elicit the preferences of health care providers for capitation payment mechanism in Kenya.\u003c/strong\u003e \u003cem\u003eHealth Economics Review \u003c/em\u003e2019, \u003cstrong\u003e9:\u003c/strong\u003e30.\u003c/li\u003e\n\u003cli\u003eTromp N, Baltussen R: \u003cstrong\u003eMapping of multiple criteria for priority setting of health interventions: an aid for decision makers.\u003c/strong\u003e \u003cem\u003eBMC Health Services Research \u003c/em\u003e2012, \u003cstrong\u003e12:\u003c/strong\u003e454.\u003c/li\u003e\n\u003cli\u003eBliemer MCJ, Rose JM: \u003cstrong\u003eDesigning and Conducting Stated Choice Experiments.\u003c/strong\u003e In \u003cem\u003eHandbook of Choice Modelling.\u003c/em\u003e Second edition. Edited by Hess S, Daly A: Forthcoming; 2022\u003c/li\u003e\n\u003cli\u003eTrapero-Bertran M, Rodr\u0026iacute;guez-Mart\u0026iacute;n B, L\u0026oacute;pez-Bastida J: \u003cstrong\u003eWhat attributes should be included in a discrete choice experiment related to health technologies? A systematic literature review.\u003c/strong\u003e \u003cem\u003ePLOS ONE \u003c/em\u003e2019, \u003cstrong\u003e14:\u003c/strong\u003ee0219905.\u003c/li\u003e\n\u003cli\u003eGreen PE, Srinivasan V: \u003cstrong\u003eConjoint Analysis in Marketing: New Developments with Implications for Research and Practice.\u003c/strong\u003e \u003cem\u003eJournal of Marketing \u003c/em\u003e1990, \u003cstrong\u003e54:\u003c/strong\u003e3-19.\u003c/li\u003e\n\u003cli\u003eDeShazo JR, Fermo G: \u003cstrong\u003eDesigning Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency.\u003c/strong\u003e \u003cem\u003eJournal of Environmental Economics and Management \u003c/em\u003e2002, \u003cstrong\u003e44:\u003c/strong\u003e123-143.\u003c/li\u003e\n\u003cli\u003eArentze T, Borgers A, Timmermans H, DelMistro R: \u003cstrong\u003eTransport stated choice responses: effects of task complexity, presentation format and literacy.\u003c/strong\u003e \u003cem\u003eTransportation Research Part E: Logistics and Transportation Review \u003c/em\u003e2003, \u003cstrong\u003e39:\u003c/strong\u003e229-244.\u003c/li\u003e\n\u003cli\u003eCaussade S, Ort\u0026uacute;zar JdD, Rizzi LI, Hensher DA: \u003cstrong\u003eAssessing the influence of design dimensions on stated choice experiment estimates.\u003c/strong\u003e \u003cem\u003eTransportation Research Part B: Methodological \u003c/em\u003e2005, \u003cstrong\u003e39:\u003c/strong\u003e621-640.\u003c/li\u003e\n\u003cli\u003eHensher DA: \u003cstrong\u003eHow do respondents process stated choice experiments? Attribute consideration under varying information load.\u003c/strong\u003e \u003cem\u003eJournal of Applied Econometrics \u003c/em\u003e2006, \u003cstrong\u003e21:\u003c/strong\u003e861-878.\u003c/li\u003e\n\u003cli\u003eRose JM, Hensher DA, Caussade S, Ort\u0026uacute;zar JdD, Jou R-C: \u003cstrong\u003eIdentifying differences in willingness to pay due to dimensionality in stated choice experiments: a cross country analysis.\u003c/strong\u003e \u003cem\u003eJournal of Transport Geography \u003c/em\u003e2009, \u003cstrong\u003e17:\u003c/strong\u003e21-29.\u003c/li\u003e\n\u003cli\u003eBridges JFP, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, Johnson FR, Mauskopf J: \u003cstrong\u003eConjoint Analysis Applications in Health\u0026mdash;a Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2011, \u003cstrong\u003e14:\u003c/strong\u003e403-413.\u003c/li\u003e\n\u003cli\u003eHinton L, Ryan S: \u003cstrong\u003eInterviews.\u003c/strong\u003e In \u003cem\u003eQualitative Research in Health Care.\u003c/em\u003e 2020: 43-55\u003c/li\u003e\n\u003cli\u003eClark T, Foster L, Sloan L, Bryman A: \u003cem\u003eBryman\u0026apos;s Social Research Methods.\u003c/em\u003e Sixth edn. Oxford, United Kingdom: Oxford University Press; 2021.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eUniversal health coverage (UHC) \u003c/strong\u003e[https://www.who.int/en/news-room/fact-sheets/detail/universal-health-coverage-(uhc)]\u003c/li\u003e\n\u003cli\u003eRitchie J, Spencer L: \u003cstrong\u003eQualitative data analysis for applied policy research.\u003c/strong\u003e In \u003cem\u003eAnalyzing qualitative data.\u003c/em\u003e Edited by Burgess RG, Bryman A. London: Routledge; 1994: 173-194\u003c/li\u003e\n\u003cli\u003ePope C, Ziebland S, Mays N: \u003cstrong\u003eAnalysis.\u003c/strong\u003e In \u003cem\u003eQualitative Research in Health Care.\u003c/em\u003e 4th edition. Edited by Pope C, Mays N. Hoboken: Wiley-Blackwell; 2020: 111-133\u003c/li\u003e\n\u003cli\u003eSpencer L, Ritchie J, Ormston R, O\u0026apos;Connor W, Barnard M: \u003cstrong\u003eAnalysis: Principles and processes.\u003c/strong\u003e In \u003cem\u003eQualitative Research Practice: A Guide for Social Science Students and Researchers.\u003c/em\u003e 2nd edition. Edited by Ritchie J, Lewis J, Nicholls CM, Ormston R. London: SAGE; 2014\u003c/li\u003e\n\u003cli\u003eQSR International Pty Ltd: \u003cem\u003eNVivo (released in June 2021).\u003c/em\u003e 2021.\u003c/li\u003e\n\u003cli\u003eR Core Team: \u003cem\u003eR: A Language and Environment for Statistical Computing.\u003c/em\u003e Vienna, Austria: R Foundation for Statistical Computing; 2024.\u003c/li\u003e\n\u003cli\u003eBliemer MCJ, Collins AT: \u003cstrong\u003eOn determining priors for the generation of efficient stated choice experimental designs.\u003c/strong\u003e \u003cem\u003eJournal of Choice Modelling \u003c/em\u003e2016, \u003cstrong\u003e21:\u003c/strong\u003e10-14.\u003c/li\u003e\n\u003cli\u003eBusemeyer JR, Townsend JT: \u003cstrong\u003eFundamental derivations from decision field theory.\u003c/strong\u003e \u003cem\u003eMathematical Social Sciences \u003c/em\u003e1992, \u003cstrong\u003e23:\u003c/strong\u003e255-282.\u003c/li\u003e\n\u003cli\u003eBusemeyer JR, Townsend JT: \u003cstrong\u003eDecision Field Theory: A Dynamic\u0026ndash;Cognitive Approach to Decision Making in an Uncertain Environment.\u003c/strong\u003e \u003cem\u003ePsychological Review \u003c/em\u003e1993, \u003cstrong\u003e100:\u003c/strong\u003ep 432\u0026ndash;459.\u003c/li\u003e\n\u003cli\u003eDiederich A: \u003cstrong\u003eDynamic Stochastic Models for Decision Making under Time Constraints.\u003c/strong\u003e \u003cem\u003eJournal of Mathematical Psychology \u003c/em\u003e1997, \u003cstrong\u003e41:\u003c/strong\u003e260-274.\u003c/li\u003e\n\u003cli\u003eRoe RM, Busemeyer JR, Townsend JT: \u003cstrong\u003eMultialternative decision field theory: A dynamic connectionst model of decision making.\u003c/strong\u003e \u003cem\u003ePsychological Review \u003c/em\u003e2001, \u003cstrong\u003e108:\u003c/strong\u003e370-392.\u003c/li\u003e\n\u003cli\u003eHancock TO, Hess S, Choudhury CF: \u003cstrong\u003eDecision field theory: Improvements to current methodology and comparisons with standard choice modelling techniques.\u003c/strong\u003e \u003cem\u003eTransportation Research Part B: Methodological \u003c/em\u003e2018, \u003cstrong\u003e107:\u003c/strong\u003e18-40.\u003c/li\u003e\n\u003cli\u003eChorus CG, Arentze TA, Timmermans HJP: \u003cstrong\u003eA Random Regret-Minimization model of travel choice.\u003c/strong\u003e \u003cem\u003eTransportation Research Part B: Methodological \u003c/em\u003e2008, \u003cstrong\u003e42:\u003c/strong\u003e1-18.\u003c/li\u003e\n\u003cli\u003eChorus CG: \u003cstrong\u003eA New Model of Random Regret Minimization.\u003c/strong\u003e \u003cem\u003eEuropean Journal of Transport and Infrastructure Research \u003c/em\u003e2010, \u003cstrong\u003e10\u003c/strong\u003e.\u003c/li\u003e\n\u003cli\u003eChorus C: \u003cstrong\u003eRandom Regret Minimization: An Overview of Model Properties and Empirical Evidence.\u003c/strong\u003e \u003cem\u003eTransport Reviews \u003c/em\u003e2012, \u003cstrong\u003e32:\u003c/strong\u003e75-92.\u003c/li\u003e\n\u003cli\u003eMcFadden D: \u003cstrong\u003eConditional logit analysis of qualitative choice behavior.\u003c/strong\u003e In \u003cem\u003eFrontiers in Econometrics.\u003c/em\u003e Edited by Zarembka P. New York: Academic Press; 1974: 105-142\u003c/li\u003e\n\u003cli\u003eTrain KE: \u003cem\u003eDiscrete Choice Methods with Simulation.\u003c/em\u003e 2 edn. Cambridge: Cambridge University Press; 2009.\u003c/li\u003e\n\u003cli\u003eHensher DA, Rose JM, Greene WH: \u003cem\u003eApplied Choice Analysis.\u003c/em\u003e 2nd edn. Cambridge: Cambridge University Press; 2015.\u003c/li\u003e\n\u003cli\u003eBaltussen R, Stolk E, Chisholm D, Aikins M: \u003cstrong\u003eTowards a multi-criteria approach for priority setting: an application to Ghana.\u003c/strong\u003e \u003cem\u003eHealth Economics \u003c/em\u003e2006, \u003cstrong\u003e15:\u003c/strong\u003e689-696.\u003c/li\u003e\n\u003cli\u003eBaltussen R, ten Asbroek AHA, Koolman X, Shrestha N, Bhattarai P, Niessen LW: \u003cstrong\u003ePriority setting using multiple criteria: should a lung health programme be implemented in Nepal?\u003c/strong\u003e \u003cem\u003eHealth Policy and Planning \u003c/em\u003e2007, \u003cstrong\u003e22:\u003c/strong\u003e178-185.\u003c/li\u003e\n\u003cli\u003eYoungkong S, Teerawattananon Y, Tantivess S, Baltussen R: \u003cstrong\u003eMulti-criteria decision analysis for setting priorities on HIV/AIDS interventions in Thailand.\u003c/strong\u003e \u003cem\u003eHealth Research Policy and Systems \u003c/em\u003e2012, \u003cstrong\u003e10:\u003c/strong\u003e6.\u003c/li\u003e\n\u003cli\u003eMarsh K, Dolan P, Kempster J, Lugon M: \u003cstrong\u003ePrioritizing investments in public health: a multi-criteria decision analysis.\u003c/strong\u003e \u003cem\u003eJournal of Public Health \u003c/em\u003e2013, \u003cstrong\u003e35:\u003c/strong\u003e460-466.\u003c/li\u003e\n\u003cli\u003ePaolucci F, Mentzakis E, Defechereux T, Niessen LW: \u003cstrong\u003eEquity and efficiency preferences of health policy makers in China\u0026mdash;a stated preference analysis.\u003c/strong\u003e \u003cem\u003eHealth Policy and Planning \u003c/em\u003e2015, \u003cstrong\u003e30:\u003c/strong\u003e1059-1066.\u003c/li\u003e\n\u003cli\u003eMeusel V, Mentzakis E, Baji P, Fiorentini G, Paolucci F: \u003cstrong\u003ePriority setting in the German healthcare system: results from a discrete choice experiment.\u003c/strong\u003e \u003cem\u003eInternational Journal of Health Economics and Management \u003c/em\u003e2023, \u003cstrong\u003e23:\u003c/strong\u003e411-431.\u003c/li\u003e\n\u003cli\u003eDiaby V, Lachaine J: \u003cstrong\u003eAn application of a proposed framework for formulary listing in low-income countries.\u003c/strong\u003e \u003cem\u003eApplied Health Economics and Health Policy \u003c/em\u003e2011, \u003cstrong\u003e9:\u003c/strong\u003e389-402.\u003c/li\u003e\n\u003cli\u003eChoiceMetrics: \u003cem\u003eNgene 1.3 User Manual \u0026amp; Reference Guide.\u003c/em\u003e Sydney, Australia; 2022.\u003c/li\u003e\n\u003cli\u003eDaly A, Dekker T, Hess S: \u003cstrong\u003eDummy coding vs effects coding for categorical variables: Clarifications and extensions.\u003c/strong\u003e \u003cem\u003eJournal of Choice Modelling \u003c/em\u003e2016, \u003cstrong\u003e21:\u003c/strong\u003e36-41.\u003c/li\u003e\n\u003cli\u003eObadha M, Chuma J, Kazungu J, Abiiro GA, Beck MJ, Barasa E: \u003cstrong\u003ePreferences of healthcare providers for capitation payment in Kenya: a discrete choice experiment.\u003c/strong\u003e \u003cem\u003eHealth Policy and Planning \u003c/em\u003e2020, \u003cstrong\u003e35:\u003c/strong\u003e842-854.\u003c/li\u003e\n\u003cli\u003eAgarwal S, Abuya T, Kintu R, Mwanga D, Obadha M, Pandya S, Warren CE: \u003cstrong\u003eUnderstanding community health worker incentive preferences in Uganda using a discrete choice experiment.\u003c/strong\u003e \u003cem\u003eJ Glob Health \u003c/em\u003e2021, \u003cstrong\u003e11:\u003c/strong\u003e07005.\u003c/li\u003e\n\u003cli\u003eAbuya T, Mwanga D, Obadha M, Ndwiga C, Odwe G, Kavoo D, Wanyugu J, Warren C, Agarwal S: \u003cstrong\u003eIncentive preferences for community health volunteers in Kenya: findings from a discrete choice experiment.\u003c/strong\u003e \u003cem\u003eBMJ Open \u003c/em\u003e2021, \u003cstrong\u003e11:\u003c/strong\u003ee048059.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eComputer-Assisted Personal Interviewing (CAPI) Surveys: A Guide \u003c/strong\u003e[https://www.geopoll.com/blog/capi-surveys-guide/]\u003c/li\u003e\n\u003cli\u003eHarris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: \u003cstrong\u003eResearch electronic data capture (REDCap)\u0026mdash;A metadata-driven methodology and workflow process for providing translational research informatics support.\u003c/strong\u003e \u003cem\u003eJournal of Biomedical Informatics \u003c/em\u003e2009, \u003cstrong\u003e42:\u003c/strong\u003e377-381.\u003c/li\u003e\n\u003cli\u003eHarris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O\u0026apos;Neal L, McLeod L, Delacqua G, Delacqua F, Kirby J, Duda SN: \u003cstrong\u003eThe REDCap consortium: Building an international community of software platform partners.\u003c/strong\u003e \u003cem\u003eJournal of Biomedical Informatics \u003c/em\u003e2019, \u003cstrong\u003e95:\u003c/strong\u003e103208.\u003c/li\u003e\n\u003cli\u003eWillis GB: \u003cem\u003eCognitive Interviewing: A Tool for Improving Questionnaire Design.\u003c/em\u003e Thousand Oaks, California: SAGE Publications Inc; 2004.\u003c/li\u003e\n\u003cli\u003eDe Br\u0026uacute;n A, Flynn D, Ternent L, Price CI, Rodgers H, Ford GA, Rudd M, Lancsar E, Simpson S, Teah J, Thomson RG: \u003cstrong\u003eA novel design process for selection of attributes for inclusion in discrete choice experiments: case study exploring variation in clinical decision-making about thrombolysis in the treatment of acute ischaemic stroke.\u003c/strong\u003e \u003cem\u003eBMC Health Services Research \u003c/em\u003e2018, \u003cstrong\u003e18:\u003c/strong\u003e483.\u003c/li\u003e\n\u003cli\u003eRyan M, Krucien N, Hermens F: \u003cstrong\u003eThe eyes have it: Using eye tracking to inform information processing strategies in multi-attributes choices.\u003c/strong\u003e \u003cem\u003eHealth Economics \u003c/em\u003e2018, \u003cstrong\u003e27:\u003c/strong\u003e709-721.\u003c/li\u003e\n\u003cli\u003eHess S, Palma D: \u003cstrong\u003eApollo: A flexible, powerful and customisable freeware package for choice model estimation and application.\u003c/strong\u003e \u003cem\u003eJournal of Choice Modelling \u003c/em\u003e2019, \u003cstrong\u003e32:\u003c/strong\u003e100170.\u003c/li\u003e\n\u003cli\u003eVermunt JK, Magidson J: \u003cstrong\u003eTechnical guide for Latent GOLD Choice 4.0: Basic and advanced.\u003c/strong\u003e Belmont, Massachusetts: Statistical Innovations Inc; 2005.\u003c/li\u003e\n\u003cli\u003eMaaya L, Meulders M, Surmont N, Vandebroek M: \u003cstrong\u003eEffect of Environmental and Altruistic Attitudes on Willingness-to-Pay for Organic and Fair Trade Coffee in Flanders.\u003c/strong\u003e In \u003cem\u003eSustainability\u003c/em\u003e, vol. 10; 2018.\u003c/li\u003e\n\u003cli\u003eObadha M, Barasa E: \u003cstrong\u003ePilot discrete choice modelling survey for an MCDA study in Kenya.\u003c/strong\u003e V1 edition: Harvard Dataverse; 2024.\u003c/li\u003e\n\u003cli\u003eMarsh K, Byun J-H, Youngkong S, Willke R: \u003cstrong\u003eUsing Multi-Criteria Decision Analysis in Healthcare Decision Making: Approaches and Applications.\u003c/strong\u003e In \u003cem\u003eISPOR Virtual Short Course Program 12 - 15 October 2020\u003c/em\u003e; 2020.\u003c/li\u003e\n\u003cli\u003eObadha M, Mumbi A, Njuguna RG, Orangi S, Nguhiu P, Ngaiza G, Omollo H, Njeru N, Barasa E: \u003cstrong\u003eHTA16 Eliciting Preferences for Health Technology Assessment Criteria Using Analytic Hierarchy Process and Discrete Choice Experiment in Kenya.\u003c/strong\u003e \u003cem\u003eValue in Health \u003c/em\u003e2023, \u003cstrong\u003e26:\u003c/strong\u003eS261.\u003c/li\u003e\n\u003cli\u003eTerwindt F, Rajan D, Soucat A: \u003cstrong\u003ePriority-setting for national health policies, strategies and plans.\u003c/strong\u003e In \u003cem\u003eStrategizing national health in the 21st century: a handbook.\u003c/em\u003e \u003cem\u003eVolume\u003c/em\u003e 71. Edited by Schmets G, Rajan D, Kadandale S. Geneva: World Health Organization; 2016\u003c/li\u003e\n\u003cli\u003eSabir M, Ali Y, Muhammad N: \u003cstrong\u003eForecasting incidence of dengue and selecting best method for prevention.\u003c/strong\u003e \u003cem\u003eJ Pak Med Assoc \u003c/em\u003e2018, \u003cstrong\u003e68:\u003c/strong\u003e1383-1386.\u003c/li\u003e\n\u003cli\u003eBakhtiari A, Takian A, Majdzadeh R, Haghdoost AA: \u003cstrong\u003eAssessment and prioritization of the WHO \u0026ldquo;best buys\u0026rdquo; and other recommended interventions for the prevention and control of non-communicable diseases in Iran.\u003c/strong\u003e \u003cem\u003eBMC Public Health \u003c/em\u003e2020, \u003cstrong\u003e20:\u003c/strong\u003e333.\u003c/li\u003e\n\u003cli\u003eShahabi S, Pardhan S, Ahmadi Teymourlouy A, Skempes D, Shahali S, Mojgani P, Jalali M, Lankarani KB: \u003cstrong\u003ePrioritizing solutions to incorporate Prosthetics and Orthotics services into Iranian health benefits package: Using an analytic hierarchy process.\u003c/strong\u003e \u003cem\u003ePLOS ONE \u003c/em\u003e2021, \u003cstrong\u003e16:\u003c/strong\u003ee0253001.\u003c/li\u003e\n\u003cli\u003eIskrov G, Miteva-Katrandzhieva T, Stefanov R: \u003cstrong\u003eMulti-Criteria Decision Analysis for Assessment and Appraisal of Orphan Drugs.\u003c/strong\u003e \u003cem\u003eFrontiers in Public Health \u003c/em\u003e2016, \u003cstrong\u003e4\u003c/strong\u003e.\u003c/li\u003e\n\u003cli\u003eMiot J, Wagner M, Khoury H, Rindress D, Goetghebeur MM: \u003cstrong\u003eField testing of a multicriteria decision analysis (MCDA) framework for coverage of a screening test for cervical cancer in South Africa.\u003c/strong\u003e \u003cem\u003eCost Effectiveness and Resource Allocation \u003c/em\u003e2012, \u003cstrong\u003e10:\u003c/strong\u003e2.\u003c/li\u003e\n\u003cli\u003eMcCormick BJJ, Waiswa P, Nalwadda C, Sewankambo NK, Knobler SL: \u003cstrong\u003eSMART Vaccines 2.0 decision-support platform: a tool to facilitate and promote priority setting for sustainable vaccination in resource-limited settings.\u003c/strong\u003e \u003cem\u003eBMJ Global Health \u003c/em\u003e2020, \u003cstrong\u003e5:\u003c/strong\u003ee003587.\u003c/li\u003e\n\u003cli\u003eThokala P, Madhavan G: \u003cstrong\u003eStakeholder involvement in Multi-Criteria Decision Analysis.\u003c/strong\u003e \u003cem\u003eCost Effectiveness and Resource Allocation \u003c/em\u003e2018, \u003cstrong\u003e16:\u003c/strong\u003e0.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Africa, Discrete choice, Kenya, MCDA, Multiple Criteria Decision Analysis, Priority setting","lastPublishedDoi":"10.21203/rs.3.rs-6630410/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6630410/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eThe paper focuses on the process of identifying and selecting priority setting criteria for use in an exploratory quantitative multi-criteria decision analysis study for health benefit package design in Kenya.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eTo identify and select criteria, a four-stage approach was adapted i.e., raw data collection, data reduction, removing inappropriate criteria, and wording.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eIn stages one and two (raw data collection and reduction), 10 criteria that had been identified by the health benefit package advisory panel, when defining the universal health coverage essential benefit package, were used as a starting point. Stage three involved reducing the long list of 10 criteria to a shorter one using researchers\u0026rsquo; judgement while taking into account multiple factors. Three researchers commented on the 10 criteria and removed inappropriate ones e.g., splitting cost-effectiveness criterion into two \u0026ldquo;cost of intervention\u0026rdquo; and \u0026ldquo;effectiveness of intervention\u0026rdquo;. In stage four, the resulting criteria and levels were further refined using semi-structured interviews with 10 stakeholders, and a pilot discrete choice modelling survey with 24 stakeholders. Results and feedback from the semi-structured interviews and the pilot discrete choice modelling survey were used to develop a final list of six criteria and levels i.e., burden of disease, congruence with existing priorities, cost of intervention, effectiveness of intervention, equity, and health systems capacity.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003eThe study, which is part of a larger exploratory multi-criteria decision analysis exercise, provided insights into the priority setting criteria Kenyan stakeholders felt were important in health benefit package design.\u003c/p\u003e","manuscriptTitle":"Identifying and Selecting Priority Setting Criteria for an Exploratory Multi-Criteria Decision Analysis Study for Health Benefit Package Design in Kenya","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-05-16 09:07:54","doi":"10.21203/rs.3.rs-6630410/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6070bcf0-c081-497f-94c1-bdb4b7ebb43e","owner":[],"postedDate":"May 16th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-01-20T17:40:02+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-16 09:07:54","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6630410","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6630410","identity":"rs-6630410","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}