Permeability to value: a methodological framework for designing demand-driven value-based innovation procurements and its application

Permeability to value: a methodological framework for designing demand-driven value-based innovation procurements and its application | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Permeability to value: a methodological framework for designing demand-driven value-based innovation procurements and its application Rossana Alessandrello, Victòria Valls-Comamala, Ion Arrizabalaga, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5609699/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract To be fully capable to respond to health system arising needs not adequately addressed through the available existing solutions, public administrations are required to spur the development, implementation, adoption and scale up of new goods or services. Innovation procurement enables public sector to act either as the promoter of the research and development intended to create adequate solutions to the arising needs (pre-commercial procurements, PCP) or to act as early adopter of innovative solutions and services that are not yet available on large scale commercial basis and that still need to generate the evidence of the value they bring to impacted stakeholders (public procurement of innovation, PPI). This article proposes a methodological framework to design and implement demand-driven public contracts “permeable to value”, referring to the ability of the “value” to pass through the whole contract life. This framework has been defined following the methodology described by Alessandrello et al. 2021 [ 1 ], including literature review of recognized framings and PCP/PPI experiences followed by a prioritization process by stakeholders. Furthermore, it presents the results of the application of the proposed framework to three European Commission co-funded call for tenders: two cross-border PCP and one PPI. This methodological framework provides public buyers with a novel method to design and implement demand-driven public contracts “permeable to value”, that take into account the perspective of all the stakeholders impacted by such contracts. Overall, it has the potentiality to enable public buyers, to demonstrate the value generated from the implemented demand-driven innovation procurements. public procurement public procurement of innovation value-based procurement pre-commercial procurement health system value-based contracts outcomes-based contracts risk-sharing contracts value-based healthcare healthcare demand-driven theory of change Figures Figure 1 1 Introduction According to the European Commission (EC), public authorities yearly spend around 14% of the Gross Domestic Product (GDP) in purchasing works, goods and services [2]. In the case of the health services and medical goods, an average of 8,3% of GPD is spent [3]. The public procurement directive[4] defines “public contracts” as contracts for pecuniary interest concluded in writing between one or more economic operators and one or more contracting authorities and having as their object the execution of works, the supply of products or the provision of services. When the execution of works, the supply of products or the provision of services imply the demand of innovations to satisfy identified unmet needs [5], public authorities act as early adopters and assume the risks associated with deploying new and unproven technologies de-risking ex-ante the investments and boosting ex-post the acceleration of the demand adoption of the innovation. Different factors should be taken into account when designing innovation procurement to stimulate innovation and bring “value” to the public services the governments and the public administration deliver: unbundling versus bundling the research and development (R&D) phase with the commercialization phase (strictly linked to how far the innovation value is ex-post observable and measurable); the distribution of the decision rights between the buyers and the suppliers (strictly linked to the ex-ante uncertainty, the ex-post performance of the new technology and the contracts governance distributing the decision rights and risks between the buyers and suppliers); the requirements and the awarding criteria (strictly linked to the description of functionalities, performance and outcomes favouring technology agnosticism, standardization and interoperability); the tender procedure (strictly linked to the selection of the tender procedure aiming at reducing the risk of costly ex-post renegotiations especially for technically, legally and financially complex tenders for which ex-ante requirements description is potentially incomplete); competition design (strictly linked to the cost uncertainty, the asymmetric information on the value of the innovation including elements as value of the contract and number of contractors at each R&D phase) The concept of “value” is not always equally conceived by the different stakeholders involved in a public procurement procedure taking place in the healthcare sector. From the public services perspective [6], the “public value” is a matter of who consumes the services rather than of who produces them. Looking at the customer-supplier relationship [7], the “value” represents the benefits created by such relationship minus the sacrifices endured in obtaining those benefits. Such value can be created, captured, consumed and destroyed by both the customer and the supplier throughout the supply chain. In the context of the healthcare sector, in 2006 Michael Porter and Elisabeth Tiesberg introduced the concept of “value-based healthcare” as “framework for restructuring health systems around the globe with the overarching goal of value for patients”[8] and the value in healthcare was defined as health outcomes that matter to patients relative to costs to achieve them, with the aim to increase cost-effectiveness of the entire care pathway. As stated by the European Commission Expert Panel on Effective Ways of Investing in Health (EXPH)[9] this definition is too narrow and, more specifically in healthcare, should go beyond the focus on reducing costs and increase efficiency. EXPH suggests that first is needed to deeper understand what all involved stakeholders (including healthcare professionals, patients, citizens and societies in general) value the most. Therefore, the EXPH proposes a definition of “value-based healthcare (VBHC)” build on four value-pillars: Personal value: appropriate care for each individual to achieve personal goals Technical value: best possible outcomes with available resources Allocative value: equitable resources distribution across patients Societal value: contribution of healthcare to social participation and connectedness In public procurement [10] “best-value-for-money” traditionally refers to procedures where contract awarding decisions are shifted from mainly price-only criterion to multi-criteria approach including various non-monetary quality components. In January 2014 the European Parliament marked a milestone in public procurement practices introducing the Most Economically Advantageous Tender (MEAT) [2] as a method of assessment that could be used as “best-value-for-money” selection procedure, allowing the contracting authorities to award the contract based on aspects of the tender submission other than just price. The MEAT is based on: price, or cost, using a cost effectiveness approach such as life cycle costing, or the best price quality ratio (BPQR) to be assessed on the basis of award criteria linked to the subject matter of the contract. MEAT can include a wide range of criteria, as aesthetic and functional characteristics, consumer service, technical assistance, environmental sustainability, and disposal costs among others. However, operationalising these criteria and allocating scores to show the relative importance can be a very complex task, involving many difficult judgements. Procurement organisations may require time and effort to ensure the weights are appropriate to reflect the value of products or services they are procuring. Moreover, in 2021, EXPH [2] concluded that the greatest barrier to MEAT is the in silo budgeting within the health systems. In this context, this article proposes a methodological framework to define demand-driven public tenders aiming at securing that the generated “public value” matters to who consumes the resulting public services, that the customer-supplier relationship represents more benefices than sacrifices along the supply chain, that the resulting public services deliver what matters to the patients and, ultimately, that the ratio between the patient-relevant outcome measurements and the costs per patient in delivering those outcomes improves. Additionally, this article presents the application of the proposed framework to three European Commission co-funded call for tenders: two cross-border pre-commercial and one innovation procurement. 2 Methods In the deliverable “D5.4 A core set of outcome indicators” [11] of the Platform for Innovation of Procurement and Procurement of Innovation (PiPPi) (European Commission Grant Agreement (EC GA) 826157)), the “Theory of Change” is referred as a framework that can give support to the definition of the public contracts. Additionally, it conceptualizes the relationship between the activities of the health intervention to be carried out through PCP and PPI procedures to address the identified unmet needs, their results, and the demonstration of the long-term improvements compared with current practice. Consequently, the demonstration of the evidence of the impact generated through a determined health intervention will pass through the definition of an appropriate “Results Chain” [1], whose linear representation is inspired by the Theory of Change framework (as represented in Fig. 1). Results Chain elements can be defined as follows: Inputs: the financial, human, and material resources used for the development of the health intervention. Activities: actions taken or work performed through the inputs, funds, technical assistance and other types of resources mobilized to produce the expected outputs. Outputs: immediate results, like the products, capital goods and services resulting from the development of the intervention. Outputs may also include changes resulting from the intervention which are relevant to the achievement of outcomes (e.g.: healthcare professional training, new healthcare protocols in place). Outcomes: the likely or achieved medium-term and long-term effects of an intervention’s outputs. It is possible to refer to both no health-related outcomes (e.g.: increase guidelines adherence or healthcare worker performance) and health-related outcomes. The World Health Organization (WHO) defines health outcomes as a ”change in the health status of an individual, group or population which is attributable to a planned intervention or series of interventions, regardless of whether such an intervention was intended to change health status”. Impacts: effects on root causes and sustained significant changes produced by a development intervention, directly or indirectly, intended or unintended (e.g.: sustained drop in the incidence of a given disease). Results: the whole outputs, outcomes and impacts of a development intervention. To determine the challenge-agnostic results in the context of any procurement activity, the methodology developed by Alessandrello et al. 2021 [1] was applied. This methodology used the Theory of Change to establish results in the short, medium and long term period and applicable to any tender. These challenge-agnostic results were defined based on literature review of recognised existing framings and different public procurement of innovation and pre-commercial procurements documentation (See Table 1) while answering to the following two questions: How are the new digital services, acquired through PiPPi Community of Practice (CoP) activities increasing the value in the health system? To what extent is PiPPi CoP succeeding in their PCP/PPI procedures according to both internal and external stakeholders? Table 1 Recognized existing framings and PCPs/PPIs used to build PiPPi challenge-agnostic results. This table describes the recognized existing framings, PCPs and PPIs practical experiences that were identified for review to identify initial list of challenge-neutral results (outputs, outcomes and impact) that could contribute to answer the two key questions. The first question “How are the new digital services, acquired through PiPPi CoP activities, increasing the value in the health system?” was answered by consulting the recognized existing framings while PCP and PPI experience supported to answer the second question “To what extent is PiPPi CoP succeeding in their PCP/PPI procedures according to both internal and external stakeholders?” Recognized existing framings PCP & PPI experiences ICHOM web DECIPHER PCP (FP7 GA 288028) PROMS-PREMS (ACI Wales) STOPandGO PPI (FP7 GA 621013) MAST Manual [12] ANTISUPERBUGS PCP (H2020 GA 688878) CAHS Framework [13] LIVEINCITE PCP (H2020 GA 727558) NICE digital evidence standards framework CITATION Nat18 \l 1027 [14] STARS PCP (H2020 GA 727585) European Commission - Digital Transformation [15] RITMOCORE PPI (H2020 GA 727796) MEAT Framework [16] Observatori del Sistema de Salut de Catalunya COSMIN [17] Impact Evaluation Framework - HUS Virtual Hospital [18] Among the collected documentation, 122 challenge-agnostic results were defined by a group of PiPPi consortium members through a prioritization process made in two stages. CARE selection criteria were applied to each identified result by both internal and external stakeholder. The CARE selection criteria were: Clearness (the meaning of the result sublevel was clear and easy to be understood); Availability (the possibility to quantify the data making the result sublevel available); the Relevance (the result sublevel was relevant for the PiPPi CoP, the public procurement of innovation and the innovation in public procurement in the healthcare) and the Exemplarity (the result sublevel was sufficiently representative of the level and/or the sublevel intended to assess). In total, twenty-two internal and external stakeholders from six different countries responded the questionnaire. Participants came from different European Union (EU) countries (Spain (six), Italy (five), Austria (three), Sweden (three), Netherlands (three) and UK (two)) and were representing different disciplines and stakeholders categories (Innovation Project Management (five), Industry (five), Academia (five), Hospital Management (five), Consultant (one) and Patient (one)). Patient representative only responded to the patient-level outcomes. From this group of respondents, ten were part of PiPPi consortium partners and twelve were external. First, experts scored individual results in agreement with the criteria Clearness and Exemplarity. Second, experts scored and categorized individual results in agreement with their Relevance and Availability. Among the identified 122 items, 26 were classified as essential results to be taken into account when defining any PCP/PPI. PiPPi 122 results (outputs, outcomes and impacts) were divided into five categories depending on the main stakeholder level of a generic digital health intervention. Four of the categories correspond to the elements directly related to the actors mainly affected by the innovative interventions resulting from PCPs and PPIs procedures (patients, healthcare professionals, healthcare providers and healthcare system) and the fifth to the elements directly related to socioeconomic aspects. Additionally, each category/level is also divided in sub-levels (see Table 2). Table 2 Proposal of stakeholder levels, their definition and sublevels for result classification This table shows the proposal of stakeholders levels, their descriptions and sublevels as defined by PiPPi consortium. Four levels matches to the main actors impacted by the implementation of the innovative interventions and the fifth relates to socioeconomic elements. Level Description Sub-level Patient-level results It includes the measurement of patient experiences and results which are patient-centred, focused on their experiences and relevant results of the iteration with the health system. Patient-Reported Outcome Measures Patient-Reported Experience Measures Determinants of Health Long-Term treatment improvement Healthcare professional-level results A set of results regarding the culture of the organization and the healthcare professional skills. Benefits for the healthcare professional Workplace environment/culture results Healthcare provider-level results A set of results that can measure the activity and performance of the health service. Organizational aspects Costs Process Technological Aspects Health system-level results A set of results aimed to measure the activity of the health system including quality, efficiency and accessibility. Economic Sustainability Safety and Sustainability Long-Term treatment improvement Socio-economic-level results A set of results aimed to measure other factors, which can significantly affect health status. Social determinants Economic Evaluation and HTA From the 122 validated and selected results (Supplementary Table 1), a prioritization process allowed to define a set of 26 short, medium and long-term results classified as essential results that PiPPi consortium proposed to take into account when defining any demand-driven PCP/PPI (See Table 3). Table 3 Proposal of PiPPi essential set of results for a PCP/PPI per level and sublevel This table shows the proposal of 26 short, medium and long-term results classified as essential from the 122 initially validated list of results that PiPPi consortium recommends to consider when defining demand-driven PCP/PPI. Level Sublevel Result Patient Outcome measures reported by patients Health-related Quality of Life Experience outcomes reported by patients Acceptability of the solution Overall satisfaction with treatment Understanding treatment Long-term disease management support Health determinants Hospital-acquired infections Accuracy of diagnosis Quality Adjusted Life Years (QALY) Reduced clinical errors Long-term treatment improvement Mortality Disability Healthcare Professional Benefits for healthcare professionals Proportion of adherence of professionals to clinical guidelines Healthcare provider Organizational aspects Number of hospitalizations Timeout Number of re-hospitalizations / readmission rate Number of beds per hospitalized patient Costs Maintenance costs Process Workflow Technological aspects Production efficiency Data security Health system Safety and sustainability Data security Security environment Long-term treatment improvement Mortality Morbidity Disability Socioeconomic impact Economic and health technology assessment Cost-Utility Analysis; Cost-Effectiveness Analysis In 2022, for the very first time, the 122 challenge-agnostic results set was used to support the buyers of two European Commission funded pre-commercial procurements to design their demand-driven value-based innovation procurements permeable to value: oncNGS PCP (H2020 GA 874467) and TIQUE PCP (H2020 GA 965356). OncNGS is a cross border pre-commercial procurement whose lead procurer is Sciensano from Belgium, who is buying in name and on behalf of OncNGS buyers group, including Sciensano (BE), Alleanza Contro il Cancro (IT), Institut Curie (FR), Institut Catala d’Oncologia (ES), Institut Jules Bordet (BE), Ludwig-Maximilians-Universitaet Muenchen (DE), Charite – Universitaetsmedizin Berlin (DE) and Hospices Civils de Lyon (FR). The main goal of the oncNGS PCP is to develop an integrated solution for diagnostic, predictive, prognostic and theranostic analysis of liquid biopsies from solid tumours (including appropriate haematological indications such as lymphoma) using NGS technology. TIQUE is a cross border pre-commercial procurement whose lead procurer is Hospital de la Santa Creu i Sant Pau from Spain, who is buying in name and on behalf of TIQUE buyers group, including Hospital de la Santa Creu i Sant Pau from Spain (ES), Azienda Sanitaria Locale Avellino (IT) and Region Västerbotten (SE). The main goal of the TIQUE PCP is to develop integrated care approaches to transformation the health and care services for patients with advanced heart failure or at risk. The steps performed to apply PiPPi challenge-agnostic results were as follows: Step 1 - Taking into consideration all the inputs received from all the relevant stakeholders during the needs definition and the needs assessment, what matters to the patients and the potential improvements of the ratio between the patient-relevant outcome measurements and the costs per patient in delivering those outcomes, the agnostic PiPPi results set was analysed to identify and select the results that could be achieved addressing the given challenge and benefitting all the identified stakeholders along the value chain (starting from the results identified as essential, then continuing with the remaining ones). Step 2 - It was considered relevant to add few additional results not present in the original PiPPi results sets. Such addition was categorized per stakeholder to ensure not to lose the stakeholder relevance of the added results. Step 3 - After having gone through the challenges PiPPi selected results and the newly defined ones, it was identified whether such results were achievable as procurements short term outputs, mid-term outcomes or long term outcomes. Step 4 - Then it was analysed how each of the challenges-selected results and the newly defined ones may be achieved as short term outputs from the contract (e.g.: reduction of process complexity, acceptability of the solution, novel treatment opportunities, understanding care plan pathways), how they influence the PCP/PPI contract outputs definition (e.g.: requirements to be delivered during the contract life and validated by end users; end users satisfaction rates) and how such outputs could be measured and assessed during the contract life (e.g.: bid template, evaluation of the bid, awarding criteria, contract monitoring, contract governance, expertise necessary to evaluate and to monitor contract execution, outputs based payments and no-health outcomes based payments). Step 5 - Then it was analysed how each of the challenges-selected results and the newly defined ones may be achieved as midterm outcomes/no-health outcomes from the contract (e.g.: production efficiency, avoidance of unnecessary treatment, waiting time reduction), how they influence the PCP/PPI contract outcomes/no-health outcomes definition (e.g.: requirements to be delivered during the contract life and validated by end users; end users satisfaction rates; evaluation of the intervention and its implementation) and how such outcomes/no-health outcomes could be measured and assessed during the contract life (e.g.: bid template, evaluation of the bid, awarding criteria, contract monitoring, contract governance, expertise necessary to evaluate and to monitor contract execution, outputs based payments and no-health outcomes based payments, evidence generation and health outcomes payments, risk sharing agreement). Step 6 - Then it was analysed how each of the challenges-selected results and the newly defined ones may be achieved as long-term results from the contract (e.g.: improvements in health outcomes, cost utility analysis, increased equality in cancer care, 5-year survival rate after diagnosis), how they influence the PCP/PPI contract long-term results definition (e.g.: requirements to be delivered during the contract life and validated by end users; end users satisfaction rates, evaluation of the intervention and its implementation) and how such long-term results could be measured and assessed during the contract life during and beyond the contract life (e.g.: bid template, evaluation of the bid, awarding criteria, contract monitoring, contract governance, expertise necessary to evaluate and to monitor contract execution, outputs based payments and no-health outcomes based payments, evidence generation and health outcomes payments, risk sharing agreement). 3 Results Thanks to OncNGS and TIQUE pre-commercial procurements, these steps were applied for the very first time to define contracts “permeable to the value” across all their whole life from their publication to the completion of the R&D contracts execution. Furthermore, the learnings gathered in these two procurement procedures served to apply in 2023 the methodology to a PPI procedure focused on the personalised surgical process for joint replacements, activity making part of EcoQUIP Plus project (COSME GA 857790). The buyer of this procurement was Consorci Corporació Sanitària Parc Taulí (ES), who was looking for a solution to allow total monitoring and traceability of both prosthesis manufacturing and surgical processes, integrating with current 3D laboratory tools and incorporating and maintaining a quality system based on ISO13485. During the application of the six steps for designing demand-driven procurements “permeable to value” (Supplementary Table 1), it was recognised that this methodological framework directly impacts to at least five elements of the tender’s definition (Supplementary Tables 2 and 3): Description of the object of the contract : independently of whether contract concerns the execution of works or the supply of products or the provision of services, the object of the contract should be described in terms of impact to be generated, in terms of functional, performance and quality requirements, in terms of life-cycle and short, mid and long-terms results. The public buyer[ 19 ] sets minimum requirements in order to avoid an abusively low-performing tender, but is not overly prescriptive as regards the means of achieving a desired outcome. Economic operators enjoy openness and flexibility to reach the optimal performance. Technical offer/bid template : the technical proposal template or bid template is prepared to facilitate participating economic operators (bidders) to clear frame how the identified need(s) will be addressed. It prompts information about the work, product and/or service is planned to be delivered and how the evidence is planned to be generated. Associating short-term, mid-term and long-term outcomes to the bid template is a way to ensure that bidders align their proposals towards their achievement. Awarding Criteria : the awarding criteria [ 20 ] are the criteria that constitute the basis on which a contracting authority selects the best tender and awards a contract. These criteria must not be prejudicial to fair competition and ensure compliance with the principles of transparency, non-discrimination and equal treatment, with a view to ensuring objective comparison of tenders. The criteria chosen must be objective and objectively quantifiable, as: the delivery of short-term outcomes (key performance requirements and adherence to specifications, the measurements of patient-relevant outcome and of the costs per patient in delivering those outcomes), the technical, quality and organizational feasibility to deliver the mid-term outcomes and to generate evidence towards the achievement of the long-term outcomes, the plan for an active contribution of all the impacted stakeholders, including the patients. Contract monitoring : Contracts should be periodically monitored and the level of the achievement of the results (milestones, interim deliverables and short, mid and long-term results) should be evaluated. The monitoring and evaluation process consists in the assessment of quantitative and qualitative indicators obtained at different contract stages. Some indicators should be directly collected and recorded when accomplished, during the contract lifetime, and some other will be gathered in specific time-points in the contract duration. Thus, including measurable and assessable outcomes as part of the indicators/results that will monitored along contract duration is an additional manner to understand to what extent these outcomes are being achieved during contract duration and/or are to be achieved after contract completion. Expertise necessary to evaluate and to monitor : as already mentioned the “value” delivery will be guaranteed by considering the demand side as a whole, represented by all its different stakeholders’ categories. Only multidisciplinary teams, representing all the impacted stakeholders, with different skills, knowledge and expertise relevant to the object of the contract will be able to correctly assess the technical offers against the awarding criteria and to correctly monitor the contracts and evaluate their achievements. The necessary expertise should be identified upfront to ensure the preparedness of the buyer to evaluate and monitor when required. The detailed analysis of the expected short-term outputs/no-health outcomes of each of the procurement procedures enabled to identify what results were susceptible to be included in the object of the contract description, in the technical offer/bid template, in the awarding criteria, in the contract monitoring and in terms of the expertise necessary to assess and monitor bidder offers and contracts progresses. On the other hand, the detailed analysis of the expected long-term outcome/impact of each of the procurement procedures enabled to identify what quantitative/qualitative indicators were considered relevant to the buyers once the interventions may be ultimately implemented, adopted and scaled up beyond the PCP and PPI contracts termination. The results of the application of the six steps to the five elements were the following: Step 1 Results In 2022 and the first semester of 2023, the proposed results set proposed for any demand-driven PCP/PPI (Supplementary table 1 ) was applied to three contracts: two PCPs, OncNGS and TIQUE and one of the PPI procedures in EcoQUIP Plus. Step 2 Results Overall, the application of this methodology led to identify eleven new or modified results in OncNGS, eight in TIQUE and five in EcoQUIP Plus to be included or modified to the original PiPPi agnostic results set (See Tables 4 , 5 and 6 ). Table 4 OncNGS additional results set This table shows the eleven additional results identified after the application of the methodology to OncNGS project Level Sublevel Output / Outcome Patient Determinants of Health Novel treatment opportunities Determinants of Health Reduction of variability in patient results Healthcare Professional Benefits for the HCP Proportion of professional with opportunities to provide care to patients Healthcare provider Organizational aspects Diagnostics Organizational aspects Compliancy with new regulatory rules Organizational aspects Costs: Maintenance costs increase Organizational aspects Errors reduction / Increase in accuracy Organizational aspects Sensitivity Health System Economic Sustainability Diagnostics Economic Sustainability Prognosis Economic Sustainability Healthcare providers transparent budget monitoring Table 5 TIQUE additional results set This table shows the eight additional results identified after the application of the methodology to OncNGS project Level Sublevel Output / Outcome Patient Determinants of Health Initially defined in PiPPi as: Adverse drug effects. Updated and detailed in TIQUE as: Drug-related health problems (adverse effects, safety, drug-related morbidity) Patient-Reported Experience Measures Community linkages Healthcare Professionals Benefits for the HCP Proportion of professional with opportunities to provide care to patients Healthcare provider Organizational aspects Home Hospitalization Organizational aspects Number of visits in primary care Organizational aspects Unplanned potentially preventable inpatient episodes Organizational aspects Patient wills Health System Long-Term treatment improvement Survival rate after diagnosis Table 6 EcoQUIP Plus additional results set This table shows the five additional results identified after the application of the methodology to EcoQUIP Plus project Level Sublevel Output / Outcome Patient Determinants of Health Less surgery complications Healthcare Professionals Benefits for the HCP Proportion of professional with opportunities to provide care to patients Healthcare provider Costs 3D Equipment Water and Energy Reduction costs Process 3D equipment training and resources Software training and resources Technological aspects Ensure UE MDR 2017/745 and ISO 13485 completion Step 3 Results : In the relevant rows of the Supplementary Table 2 of the different contracts (OncNGS, TIQUE and EcoQUIP Plus), the first two columns contain the identified short and mid-term results, while the identified long-term results are in the in the relevant rows of the Supplementary Table 3. Step 4 Results : In the relevant rows of the Supplementary Table 2 of the different contracts (OncNGS, TIQUE and EcoQUIP Plus), the columns making explicit mention to “object of the contract”, “Bid Template”, “Awarding Criteria”, “Contract Monitoring”, “Expertise necessary to evaluate/ monitor” contain the identification of the tender elements the short-term outputs influence and what needs to be taken into account element by element. Step 5 Results : In the relevant rows of the Supplementary Table 2 of the different contracts (OncNGS, TIQUE and EcoQUIP Plus), the columns making explicit mention to “object of the contract”, “Bid Template”, “Awarding Criteria”, “Contract Monitoring”, “Expertise necessary to evaluate/ monitor” contain the identification of the tender elements the mid-term outcomes/no-health outcomes influence and what needs to be taken into account element by element. Step 6 Results : The Supplementary Table 3 contains the identified long-term outcomes of oncNGS, TIQUE and EcoQUIP Plus. In both OncNGS and TIQUE contracts, the governance and the risk sharing agreement have been designed towards the accomplishment of given performance indicators, the commencement of measuring to enable the evidence generation, the contract monitoring assessment and the outcome-based payments. In the case of EcoQUIP Plus contract, the governance has been designed towards the accomplishment of given performance indicators, the contract monitoring assessment and the commencement of measuring to enable the evidence generation. In summary, the proposed “Permeability to value” framework results from the application in six steps of PiPPi challenge-agnostic results set to five public tenders’ elements (see Table 7 ): Table 7 Methodological framework for designing demand-driven value-based innovation procurements This table shows the six steps to apply PiPPi challenge-agnostic results to any contract to be designed “permeable to the value” and what contract elements are impacted by such application “Permeability to value” framework Six steps to select/identify challenge-agnostic results sets Five public tenders elements impacted by the selected/identified results Step 1 From PiPPi results set selection of results that could be achieved addressing the given challenge and benefitting all the identified stakeholders along the value chain Step 2 Addition of relevant results not present in the original PiPPi results sets (addition categorised per stakeholder) Step 3 Identification of results as short term outputs, mid-term outcomes or long term outcomes. Step 4 Analysis of how short term influence the PCP/PPI contract outputs definition and how such outputs could be measured and assessed during the contract life Step 5 Analysis of how midterm outcomes/no-health influence the PCP/PPI contract outcomes/no-health outcomes definition and how such outcomes/no-health outcomes could be measured and assessed during the contract life Step 6 Analysis of how long-term results influence the PCP/PPI contract long-term results definition and how such long-term results could be measured and assessed during the contract life during and beyond the contract life 1. Description of the object of the contract 2. Technical offer/bid template 3. Awarding Criteria 4. Contract monitoring 5. Expertise necessary to evaluate the bids and to monitor the contracts 4 Discussion Ten years after the publication of the Public Procurement Directive 2014/24/EU [4], which gave contracting authorities in the Member States a new awarding procedure to pursue innovation, it is evident that operationalising awarding criteria and allocating scores against them can be a complex task that requires further action. Building upon existing recognised methodological frameworks as well as from the experience accrued from previous EU projects and PCP and PPI contracting practices, the described methodological approach proposes an applicable and usable system to apply from the preparation stages of the innovation procurement procedures. Such methodological approach leads to the definition of value-based contracts and enables procurement organisations and suppliers to demonstrate the real value of the solutions once implemented and adopted to a wider market as well as to patients, health care professionals, health care institutions, health care services and, ultimately, to the society as whole. Taking into account the starting point of the “permeability to value” methodology, that are the “theory of change” and the PiPPi challenge-agnostic results set, this methodology enables by default the identification of the “public values” that matters to who consumes the resulting public services, the benefices and the sacrifices along the healthcare delivery chain, what matters most to the patients, the patient-relevant outcome measurements and the costs per patient in delivering those outcomes improvements. The perspective of value differs depending on each targeted stakeholders’ point of view. The proposed methodological approach takes this subjective element into consideration and integrates it as cornerstone of the procurement of innovation process. Embedded within an improved cycle of best practices, the new methodological approach opens a constructive debate that permits to consolidate the knowledge and perspectives from different stakeholders into a robust contracting procedure. The effort to produce a collaborative framework of continuous improvement is linked to a shared commitment to participate in the continuous improvement of the management processes that supports the mainstreaming of innovation procurement strategic policy to create demand-driven adoption of innovative solutions for the transformation of the health and social systems. In accordance with the “Theory of Change”, the application of present methodological approach allows public organisations and suppliers to identify measurable results relevant to the affected stakeholders and assess them in short and midterm period. As follow up of current PCP and PPI procedures where the solutions will be ultimately deployed in procurement organisations’ settings, this methodology enables to eventually assess the long-term results of the actions and according to the stakeholder level (as described in Table 2). As a result, the tender will be “permeable to value”, in other words the “value” will have the ability to pass through the whole contract life. The “value” will be planned to influence the definition of the different aspects of the tender as the ex-ante description of the object of the contract, the requirements and the awarding criteria (in terms of functionalities, performance and outcomes favouring the technology neutrality), the template the bidders are required to use to submit their bids, the description of the contract execution with its deliverables and milestones (including what and how far the ex-post results will be observable and measurable), the contracts monitoring and the expertise necessary to evaluate the bids and monitor the contracts and the payment model (including the allocation between the buyers and suppliers of risks based on the achievements of defined ex-post measurable outcomes). All aspects that influence further the innovation adoption model and how and when the “value” is ultimately delivered to all the relevant stakeholders should be explicated by: - Unbundling versus bundling the research and development (R&D) phase with the commercialization phase; - defining the contracts governance allocating the decision rights and risks between the buyers and suppliers; - Selecting the tender procedure and designing the competition. “Value” will be guaranteed by considering both the demand side as a whole - represented by all the different stakeholders’ categories that will benefit from the implementation, adoption and scale up of the innovative technological intervention resulting from the PCP/PPI contracts (patients, healthcare professionals, healthcare providers, health systems) – including the supply side. This methodology is still under validation since two of the three mentioned contracts are in progress at the date of publication of this article: - OncNGS precommercial procedure [21] already successfully awarded the Framework Agreement, completed Phase 1 contract and Phase 2 contracts and is now contracting Phase 3. - TIQUE precommercial procedure [22] already successfully awarded the Framework Agreement, completed Phase 1 and Phase 2 contracts and awarded Phase 3 contracts, currently under execution. - The personalised surgical process for joint replacements of EcoQUIP Plus successfully awarded and executed the contract. Furthermore, the proposed methodology also showed to be easily adopted by any demand-driven innovation procurement procedure to assess the value of the resulting healthcare interventions brings to the different stakeholders, since it was also applied to two further public procurements of innovations launched by two of RaDAR PPI (COSME GA 101036228) (36) buyers looking at integrated solutions for rapid detection of antimicrobial resistance and patient management in hospital settings. During 2024 both Institut Català d'Oncologia and Università di Napoli Federico II launched their RaDAR PPI tenders (GA 101036228) applying this methodology and writing their contracts “permeable to value” [23, 24]. In the meantime, the authors of this article are actively boosting its use across the different demand driven innovation procurements they are defining at both European and local level with the aim at creating a corpus of public contracts enabling the validation of the methodology. The interaction with the affected stakeholders of each challenge is key to support the identification of results that can lead to improvement of the agnostic set of results. At the same time, the authors are working on defining a process to disseminate its wide use across all the upcoming healthcare demand-driven innovation contracts and to validate the methodology. Overall the proposed methodology demonstrated to be agnostic to the procedure, since it could be successfully used for both pre-commercial procurements and public procurement of innovations, and agnostic to the challenge, since it could be used on a wide range of diseases and patients' journeys (cancer diagnostic and treatment, health and social care of heart failure patients and prosthesis manufacturing and implant). However, there is the need to maintain and update PiPPi results set and put in place a process to periodically validate the updated set and the definition of the results. It is important to highlight that in case of the PCP procedures, the selected results can only be assessed in the short term and medium-term, but not in the long term. Thanks to the application of this methodology, in subsequent PPIs, the results of the procurements come not only from its application to the tender definition but also from the results of the previous PCPs already defined permeable to the value. The Value Based Procurement Subcommission of the Catalonian Health System Innovation & Transformation Departmental Commission [25] is working on the creation of an Observatory of the Value Based Procurements taking place in the Catalonian health system. This Observatory would offer to the authors the opportunity to assess the effectiveness of the proposed methodology by comparing “permeable to value” procurements with Catalonian value based contracts designed and launched without making use of this methodology. The “permeability to value” effectiveness could be measured by evaluating the degree of achievement of the expected results, the degree of viability and preparedness of any subsequent value based contract and the use of resources for its implementation. The measure of the “permeability to value” effectiveness could be of support to define the inclusion criteria for the application of this methodology to future public procurements of innovation. 5 Conclusions This article provides a methodological framework to design and implement demand-driven public contracts permeable to the value taking into account the perspective of involved stakeholders. The methodological framework has been already applied to five innovation public contracts in the healthcare sector (two of them under execution, one of them completed and two of them in the tendering phase). The application of the proposed methodological approach in further PCP and PPI initiatives will support the refinement, and ultimately the validation, of mechanisms that will allow public contracting authorities and the innovation ecosystem to demonstrate the value generated from the implemented demand-driven innovation procurement procedures across time for any given sector. The Value Based Procurement Subcommission of the Catalonian Health System Innovation & Transformation Departmental Commission considers the proposed “value permeability” framework a valid methodology to design demand-driven innovation procurements. Abbreviations BPQR: best price quality ratio CoP: Community of Practice EC: European Commission GA: Grant Agreement GDP: Gross Domestic Product MEAT: Most Economically Advantageous Tender PCP: Pre-commercial Procurement PiPPi: Platform for Innovation of Procurement and Procurement of Innovation PPI: Public Procurement of Innovation R&D: research and development Declarations Ethics approvals In case of oncNGS: the study to be performed within Phase 3 contract is a corroboration study and is not a clinical study. This study was reviewed and deemed exempt by SCIENSANO, oncNGS project coordinating entity and lead procurer. Samples to be used in the corroboration study can be both from biobanks and prospectively collected. Written informed consent is being obtained from all individual participants whose samples are prospectively collected and included in Phase 3 corroboration study. The prospective samples collection is being performed in line with the principles of the Declaration of Helsinki and approval was granted by the Ethics Committee of Hospital Universitari de Bellvitge (ES) (Date.25 th of February 2021 /No. PR381/20) In case of TIQUE: the study to be performed within Phase 3 contract is in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Hospital de la Santa Creu i Sant Pau (ES) (Date 25 th of May 2021 /No. IIBSP-TIQ-2021-44). Written informed consent is being obtained from all individual participants who will be included in Phase 3 study. In case of EcoQUIP Plus: the study performed within the contract was a cost analysis study. This study was reviewed and deemed exempt by Parc Taulí Institut d’Investigació I Innovació I3PT (ES). Competing interests The authors declare no competing interests. Open access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit https://creativecommons.org/licenses/by-nc-nd/4.0/ Acknowledgements We would like to thank the following for their assistance in the production of this proposal of methodological framework: Maria Sanchis-Amat, EIT Health, and Anna Garcia-Valls, Agència de Residus de Catalunya, for the support they gave while working in AQuAS - Agència de Qualitat i Avaluació Sanitàries de Catalunya in identifying the results of oncNGS and TIQUE respectively. All the patients and professionals coming from the list of entities below and who participated to the PiPPi CARE surveys and to validate the agnostic results set applied to this novel methodology to oncNGS, TIQUE, EcoQUIP Plus and RaDAR public tenders (in alphabetical order): Alleanza Contro Il Cancro (IT) Azienda Sanitaria Locale Avellino (IT) Charite - Universitaetsmedizin Berlin (DE) Erasmus Universitair Medisch Centrum Rotterdam (NL) Helsingin ja Uudenmaan Sairaanhoitopiirin Kuntayhtymä (FI) Hospices Civils De Lyon (FR) Hospital de la Santa Creu i Sant Pau (ES) Hospital Universitari Vall d'Hebron (ES) Institut Català d'Oncologia (ES) Institut Curie (FR) Institut Jules Bordet Asbl (BE) Karolinska University Hospital (SE) King's College Hospital Nhs Foundation Trust (UK) Ludwig-Maximilians-Universitaet Muenchen (DE) Medizinische Universitaet Wien (AU) Ospedale San Raffaele Srl (IT) Parc Taulí Institut d’Investigació I Innovació I3PT (ES) Region Vasterbotten (SE) Sciensano (BE) Università Di Napoli Federico II (IT) All members of the Value Based Procurement Subcommission of the Catalonian Health System Innovation & Tranformation Departamental Commission who reviewed the article and the methodology: (in alphabetical order) Cristina Adroher, Rossana Alessandrello, Cari Almazán, Neus Colet, Yolima Cossio, Ramon Maspons, Gemma Enfedaque, Antoni Gilabert, Josep Maria Giménez, Encarna Grifell, Cari Pontes, Laura Sampietro, Núria Umbert Funding The authors gratefully acknowledge funding from the European Union's Horizon 2020 research and innovation programme under Grant Agreements 826157 (Coordination and support action, PiPPi), 874467 (PCP, oncNGS), 965356 (PCP, TIQUE), 857790 (PPI, EcoQUIP Plus) and 101036228 (PPI, RaDAR). Author Contribution Rossana Alessandrello conceived the original idea, designed and developed the theoretical framework and directed the research. Rossana Alessandrello, Victòria Valls-Comamala, Ion Arrizabalaga, Uxío Meis-Pinheiro, Olman Elizondo-Cordero equally contributed to the manuscript. Ramon Maspons and Anna Garcia-Altés contributed to the final version of the manuscript and supervised the research. All authors provided critical feedback, discussed the results and commented on the manuscript and approved the version to be published. Data Availability All data generated or analysed during this study are included in this published article and its supplementary information files. The supplementary information associated to this article contains the complete set of proposed results for demand driven PCP/PPI and the application of the PiPPi result set to the call for tender documents of OncNGS, TIQUE and EcoQUIP Plus. References A. G. I. M. P. U. E. C. O. S.-A. M. M. R. Alessandrello R, “Teoria del canvi, resultats neutrals respecte al tipus de necessitats no satisfetes i permeabilitat de les compres públiques d'innovació al valor en l'àmbit de la salut,” vol. 104, 2021. “Expert Panel on effective ways of investing in health(EXPH),public procurement in healthcare systems,,” 2021. E. Comission, “PUBLIC PROCUREMENT IN HEALTHCARE SYSTEMS. Fact sheet accompanying the Opinion by the Expert Panel on Effective Ways of Investing in Health (EXPH),” Publication Office of the European Union, Brussels, 2021. European Parliament, “Directive 2014/24/EU of the European Parliament and of the Council of 26th of February on public procurement and repealing Directive 2004/18/EC,” 2014. O. Chiappinelli, L. M. Giuffrida and G. Spagnolo, “Public Procurement as an Innovation Policy: Where Do We Stand?,” Vols. ZEW Discussion Paper No. 23-002, 2023. J. &. H. O. Alford, "Public Value Pragmatism as the Next Phase of Public Management," vol. 38(2), 2008. T. R. H. G. G. Achim Walter, “Value Creation in Buyer–Seller Relationships: Theoretical Considerations and Empirical Results from a Supplier's Perspective,” vol. 30, no. 4, 2001. M. E. Porter, “What is value in Health Care?,” The New England Journal of Medicine, vol. 363, no. 26, p. 5, 2010. “Expert Panel on effective ways of investing in Health (EXPH). Defining value in "value-based healthcare",” 2019. N. Dimitri, “Best value for money” in procurement",” vol. 13, no. 2, 2013. AQuAS on behalf of PiPPi consortium, "D5.4. A core set of outcome indicators," 2021. MethoTelemed project, “The MAST Manual. Model for ASsessment of Telemedicine,” 2010. Canadian Academy of Health Sciences | Académie canadienne des sciences de la santé, “Making an Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research. Report of the Panel on the Return on Investments in Health Research,” Canadian Academy of Health Sciences | Académie canadienne des sciences de la santé, Ottawa, Canadà, 2009. National Institute for Health and Care Excellence, “Evidence standards framework for digital technologies,” NICE, 2018. European Commission, “A Europe fit for the digital age. Empowering people with a new generation of technologies,” European Commission, 2019. [Online]. Available: https://commission.europa.eu/strategy-and-policy/priorities-2019-2024/europe-fit-digital-age_en. J. C. C. P. Y. V. H. B. Götz Gerecke, “How Procurement unlocks value-based healthcare,” Boston Consulting Group & MedTech Europe, 2020. COSMIN initiative, “COnsensus-based Standards for the selection of health Measurement INstruments,” COSMIN initiative, [Online]. Available: https://www.cosmin.nl/. S. L.-T. P. (. V. H. 2. p. Arvonen, “We are getting there – Virtual Hospital 2.0 project summary.,” Helsinki and Uusimaa Hospital District (HUS), Helsinki, 2019. E. Commission, “Guidance on Innovation Procurement,” Brussels, 2021 . O. a. E. Commission, “Setting the Award Criteria,” SIGMA Public Procurement Briefs. OECD Publishing, Paris, 2011. OncNGS Consortium (Sciensano), “e-Notification (publicprocurement.be),” 2022. [Online]. Available: https://enot.publicprocurement.be/enot-war/preViewNotice.do?noticeId=452212. TIQUE Consortium (Fundació de Gestió Sanitària de l'Hospital de la Santa Creu i Sant Pau), “Plataforma de Serveis de Contractació Pública,” 2022. [Online]. Available: https://contractaciopublica.cat/en/detall-publicacio/f98c4c0d-5ade-19f6-a114-7b258c16a9d2/113124982. RaDAR Consortium (Institut Català d’Oncologia), “RaDAR call for tender launched by INSTITUT CATALÀ D'ONCOLOGIA. Servei d’abordatge integral per al control dels microorganismes resistents als antimicrobians,” Plataforma de Serveis de Contractació Pública, 2023. RaDAR Consortium (UNIVERSITÀ DI NAPOLI FEDERICO II), “RaDAR call for tender launched by UNIVERSITÀ DI NAPOLI FEDERICO II. Servizi informatici innovativi per la individuazione rapida di microrganismi multiresistenti ai farmaci e per la gestione e il controllo intelligenti della antibioticoresistenza,” Acquisti in rete certificata da Anac, 2024. Diari Oficial de la Generalitat de Catalunya, “ORDRE SLT/5/2022, de 25 de gener, per la qual es crea la Comissió Departamental d'Innovació i Transformació del Sistema de Salut,” 2022. Platform for Innovation of Procurement and Procurement of Innovation Platform for Innovation of Procurement and Procurement of Innovation (PiPPi project), “CORDIS. EU research results,” PiPPi consortium, 2018. [Online]. Available: https://cordis.europa.eu/project/id/826157. International Consortium for Health Outcomes Measurement, “International Consortium for Health Outcomes Measurement,” [Online]. Available: https://www.ichom.org/. Distributed European Community Individual Patient Healthcare Electronic Record (DECIPHER Project), “CORDIS. EU research results,” DECIPHER consortium, 2012. [Online]. Available: https://cordis.europa.eu/project/id/288028. Agency for Clinical Innovation , “Agency for Clinical Innovation (NSW government),” [Online]. Available: https://aci.health.nsw.gov.au/make-it-happen/prms. Sustainable Technology for Older People – Get Organised (STOP and Go Project), “CORDIS. EU research Results,” STOP and Go consortium, 2014. [Online]. Available: https://cordis.europa.eu/project/id/621013. ANTI-SUPERBUGS Precommercial procurement project, “CORDIS. EU research results,” ANTI-SUPERBUGS PPI consortium, 2016. [Online]. Available: https://cordis.europa.eu/project/id/688878. Lifestyle intervention in the perioperative process through digital service (LIVEINCITE PCP project), “CORDIS. EU research results,” LIVEINCITE PCP consortium, 2016. [Online]. Available: https://cordis.europa.eu/project/id/727558. Empowering Patients by Professional Stress Avoidance and Recovery Services (STARS project), “CORDIS. EU research results,” STARS consortium, [Online]. Available: https://cordis.europa.eu/project/id/727585/es. Arrhythmias monitoring and comprehensive care (Ritmocore Project), “CORDIS. EU research results,” Ritmocore consortium, 2016. [Online]. Available: https://cordis.europa.eu/project/id/727796. Agència de Qualitat i Avaluació Sanitàries de Catalunya (AQuAS), “Observatori del Sistema de Salut de Catalunya,” Agència de Qualitat i Avaluació Sanitàries de Catalunya (AQuAS), [Online]. Available: https://observatorisalut.gencat.cat/ca/inici. NGS diagnostics in 21st century oncology: the best, for all, at all times (oncNGS PCP project), “CORDIS. EU research results,” oncNGS PCP consortium, 2019. [Online]. Available: https://cordis.europa.eu/project/id/874467. InnovaTIve care services, enhanced with technology, to deliver QUick rEsponses for individuals with advanced heart failure and complex care needs through integrated care (TIQUE PCP project), “CORDIS. EU research results,” TIQUE PCP consortium, 2021. [Online]. Available: https://cordis.europa.eu/project/id/965356. EcoQuip+ project, "Delivering efficiency, quality and sustainability in healthcare," EcoQuip+ consortium, 2021. [Online]. Available: https://www.ecoquip.eu/. Additional Declarations No competing interests reported. Supplementary Files Supplementarytable1.docx Supplementarytable2.docx Supplementarytable3.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5609699","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":432462743,"identity":"74a7a428-eeb3-4a28-90fa-b925916f01a5","order_by":0,"name":"Rossana Alessandrello","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABCklEQVRIie3RsUrEMBjA8a8UOvXsWqngK0QOTgdNX6Uh0Km4uBQcrBTS5dDVAx/CR/iOgi7hXAtdLvgCFZcKHUw5uEEI11Ew/yUE8uNLCIDN9idzi92K4OAWLqcQZ08AE0j3G/AnkvowOa/W96of6PVRWyMm+Tt9jEr11QE9jQ3kRLJyPhP85niTJpjIlq+eX+chAj9bGkgITERO4bIX6RNkouWkyUATdIwkUNV3P9xpEnSabHjcZG6vSWwkIRPge/U4BTRBSsLMG6cwM1FlNBNvbCU9ot/Ck7BJFxeScL5E08X4+rMfbtmDdNW2y2kcPPGPJs/pVVUYxvyO7Q6SicfHTL9hs9ls/7gfM/Bmqu+WULoAAAAASUVORK5CYII=","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":true,"prefix":"","firstName":"Rossana","middleName":"","lastName":"Alessandrello","suffix":""},{"id":432462744,"identity":"0e880fdd-2309-4340-89f5-c4c5d7b86f8f","order_by":1,"name":"Victòria Valls-Comamala","email":"","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":false,"prefix":"","firstName":"Victòria","middleName":"","lastName":"Valls-Comamala","suffix":""},{"id":432462745,"identity":"2a46ec13-c708-4a41-900f-3a088e06b42e","order_by":2,"name":"Ion Arrizabalaga","email":"","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":false,"prefix":"","firstName":"Ion","middleName":"","lastName":"Arrizabalaga","suffix":""},{"id":432462746,"identity":"0b7ab9f3-acf1-4930-926f-85c3edda4219","order_by":3,"name":"Uxío Meis-Pinheiro","email":"","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":false,"prefix":"","firstName":"Uxío","middleName":"","lastName":"Meis-Pinheiro","suffix":""},{"id":432462747,"identity":"64e8fdd5-6e31-4194-9d90-62211767129e","order_by":4,"name":"Olman Elizondo-Cordero","email":"","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":false,"prefix":"","firstName":"Olman","middleName":"","lastName":"Elizondo-Cordero","suffix":""},{"id":432462748,"identity":"a6d458f0-c70c-42eb-94ab-1fcbcd07a18c","order_by":5,"name":"Ramon Maspons","email":"","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":false,"prefix":"","firstName":"Ramon","middleName":"","lastName":"Maspons","suffix":""},{"id":432462749,"identity":"cb5606d1-841a-42ea-b246-6d79b427b145","order_by":6,"name":"Anna Garcia-Altés","email":"","orcid":"","institution":"Agency for Health Quality and Assessment of Catalonia (AQuAS)","correspondingAuthor":false,"prefix":"","firstName":"Anna","middleName":"","lastName":"Garcia-Altés","suffix":""}],"badges":[],"createdAt":"2024-12-09 14:08:54","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5609699/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5609699/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":79259892,"identity":"e25e5138-f6c0-4095-9d63-1dfa23b21ec7","added_by":"auto","created_at":"2025-03-26 09:22:19","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":23670,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eResult chain.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eRepresentation of the Results Chain inspired by the Theory of Change used to demonstrate the evidence of impact generated by a health intervention.\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5609699/v1/f438ae0aa5b77d81f349d0f2.jpg"},{"id":83536471,"identity":"b9b9bb76-9378-4e23-97b0-86367c502d05","added_by":"auto","created_at":"2025-05-28 06:47:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":811511,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5609699/v1/559eec65-9762-4d8c-8d6e-ff2c38b1c4c1.pdf"},{"id":79259893,"identity":"8785440a-1bba-4a3c-a88e-15abf64caead","added_by":"auto","created_at":"2025-03-26 09:22:19","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":81051,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementarytable1.docx","url":"https://assets-eu.researchsquare.com/files/rs-5609699/v1/7692c46e5e62e52560607dca.docx"},{"id":79259903,"identity":"2d7d0369-4967-4167-bf5b-14413e5f2ab3","added_by":"auto","created_at":"2025-03-26 09:22:20","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":108299,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementarytable2.docx","url":"https://assets-eu.researchsquare.com/files/rs-5609699/v1/81c253cded97dcaed0a57e60.docx"},{"id":79261013,"identity":"5ce17f68-6696-4326-846e-02aec36d79ff","added_by":"auto","created_at":"2025-03-26 09:30:20","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":85410,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementarytable3.docx","url":"https://assets-eu.researchsquare.com/files/rs-5609699/v1/0de691d4e6dc823b7ddadcbd.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Permeability to value: a methodological framework for designing demand-driven value-based innovation procurements and its application","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eAccording to the European Commission (EC), public authorities yearly spend around 14% of the Gross Domestic Product (GDP) in purchasing works, goods and services [2]. In the case of the health services and medical goods, an average of 8,3% of GPD is spent [3].\u003c/p\u003e\n\u003cp\u003eThe public procurement directive[4] defines “public contracts” as contracts for pecuniary interest concluded in writing between one or more economic operators and one or more contracting authorities and having as their object the execution of works, the supply of products or the provision of services.\u003c/p\u003e\n\u003cp\u003eWhen the execution of works, the supply of products or the provision of services imply the demand of innovations to satisfy identified unmet needs [5], public authorities act as early adopters and assume the risks associated with deploying new and unproven technologies de-risking ex-ante the investments and boosting ex-post the acceleration of the demand adoption of the innovation. Different factors should be taken into account when designing innovation procurement to stimulate innovation and bring “value” to the public services the governments and the public administration deliver:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\n \u003cp\u003eunbundling versus bundling the research and development (R\u0026amp;D) phase with the commercialization phase (strictly linked to how far the innovation value is ex-post observable and measurable);\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003ethe distribution of the decision rights between the buyers and the suppliers (strictly linked to the ex-ante uncertainty, the ex-post performance of the new technology and the contracts governance distributing the decision rights and risks between the buyers and suppliers);\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003ethe requirements and the awarding criteria (strictly linked to the description of functionalities, performance and outcomes favouring technology agnosticism, standardization and interoperability);\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003ethe tender procedure (strictly linked to the selection of the tender procedure aiming at reducing the risk of costly ex-post renegotiations especially for technically, legally and financially complex tenders for which ex-ante requirements description is potentially incomplete);\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003ecompetition design (strictly linked to the cost uncertainty, the asymmetric information on the value of the innovation including elements as value of the contract and number of contractors at each R\u0026amp;D phase)\u003c/p\u003e\n \u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe concept of “value” is not always equally conceived by the different stakeholders involved in a public procurement procedure taking place in the healthcare sector. From the public services perspective [6], the “public value” is a matter of who consumes the services rather than of who produces them. Looking at the customer-supplier relationship [7], the “value” represents the benefits created by such relationship minus the sacrifices endured in obtaining those benefits. Such value can be created, captured, consumed and destroyed by both the customer and the supplier throughout the supply chain.\u003c/p\u003e\n\u003cp\u003eIn the context of the healthcare sector, in 2006 Michael Porter and Elisabeth Tiesberg introduced the concept of “value-based healthcare” as “framework for restructuring health systems around the globe with the overarching goal of value for patients”[8] and the value in healthcare was defined as health outcomes that matter to patients relative to costs to achieve them, with the aim to increase cost-effectiveness of the entire care pathway. As stated by the European Commission Expert Panel on Effective Ways of Investing in Health (EXPH)[9] this definition is too narrow and, more specifically in healthcare, should go beyond the focus on reducing costs and increase efficiency. EXPH suggests that first is needed to deeper understand what all involved stakeholders (including healthcare professionals, patients, citizens and societies in general) value the most. Therefore, the EXPH proposes a definition of “value-based healthcare (VBHC)” build on four value-pillars:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\n \u003cp\u003ePersonal value: appropriate care for each individual to achieve personal goals\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003eTechnical value: best possible outcomes with available resources\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003eAllocative value: equitable resources distribution across patients\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003eSocietal value: contribution of healthcare to social participation and connectedness\u003c/p\u003e\n \u003c/li\u003e\n\u003c/ul\u003e\n\u003cdiv\u003e\n \u003cp\u003eIn public procurement [10] “best-value-for-money” traditionally refers to procedures where contract awarding decisions are shifted from mainly price-only criterion to multi-criteria approach including various non-monetary quality components. In January 2014 the European Parliament marked a milestone in public procurement practices introducing the Most Economically Advantageous Tender (MEAT) [2] as a method of assessment that could be used as “best-value-for-money” selection procedure, allowing the contracting authorities to award the contract based on aspects of the tender submission other than just price. The MEAT is based on:\u003c/p\u003e\n\u003c/div\u003e\n\u003cul\u003e\n \u003cli\u003e\n \u003cp\u003eprice, or\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003ecost, using a cost effectiveness approach such as life cycle costing, or\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003ethe best price quality ratio (BPQR) to be assessed on the basis of award criteria linked to the subject matter of the contract.\u003c/p\u003e\n \u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eMEAT can include a wide range of criteria, as aesthetic and functional characteristics, consumer service, technical assistance, environmental sustainability, and disposal costs among others. However, operationalising these criteria and allocating scores to show the relative importance can be a very complex task, involving many difficult judgements. Procurement organisations may require time and effort to ensure the weights are appropriate to reflect the value of products or services they are procuring. Moreover, in 2021, EXPH [2] concluded that the greatest barrier to MEAT is the in silo budgeting within the health systems.\u003c/p\u003e\n\u003cp\u003eIn this context, this article proposes a methodological framework to define demand-driven public tenders aiming at securing that the generated “public value” matters to who consumes the resulting public services, that the customer-supplier relationship represents more benefices than sacrifices along the supply chain, that the resulting public services deliver what matters to the patients and, ultimately, that the ratio between the patient-relevant outcome measurements and the costs per patient in delivering those outcomes improves. Additionally, this article presents the application of the proposed framework to three European Commission co-funded call for tenders: two cross-border pre-commercial and one innovation procurement.\u003c/p\u003e"},{"header":"2 Methods","content":"\u003cp\u003eIn the deliverable \u0026ldquo;D5.4 A core set of outcome indicators\u0026rdquo; [11] of the Platform for Innovation of Procurement and Procurement of Innovation (PiPPi) (European Commission Grant Agreement (EC GA) 826157)), the \u0026ldquo;Theory of Change\u0026rdquo; is referred as a framework that can give support to the definition of the public contracts. Additionally, it conceptualizes the relationship between the activities of the health intervention to be carried out through PCP and PPI procedures to address the identified unmet needs, their results, and the demonstration of the long-term improvements compared with current practice.\u003c/p\u003e\n\u003cp\u003eConsequently, the demonstration of the evidence of the impact generated through a determined health intervention will pass through the definition of an appropriate \u0026ldquo;Results Chain\u0026rdquo; [1], whose linear representation is inspired by the Theory of Change framework (as represented in Fig. 1).\u003c/p\u003e\n\u003cp\u003eResults Chain elements can be defined as follows:\u0026nbsp;\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eInputs: the financial, human, and material resources used for the development of the health intervention.\u003c/li\u003e\n \u003cli\u003eActivities: actions \u0026nbsp; \u0026nbsp;taken \u0026nbsp; or \u0026nbsp; work \u0026nbsp; \u0026nbsp;performed through the inputs, funds, technical assistance and other types of resources mobilized to produce the expected outputs.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eOutputs: immediate results, like the products, capital goods and services resulting from the development of the intervention. Outputs may also include changes resulting from the intervention which are relevant to the achievement of outcomes (e.g.: healthcare professional training, new healthcare protocols in place).\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eOutcomes: the likely or achieved medium-term and long-term effects of an intervention\u0026rsquo;s outputs. It is possible to refer to both no health-related outcomes (e.g.: increase guidelines adherence or healthcare worker performance) and health-related outcomes. The World Health Organization (WHO) defines health outcomes as a \u0026rdquo;change in the health status of an individual, group or population which is attributable to a planned intervention or series of interventions, regardless of whether such an intervention was intended to change health status\u0026rdquo;.\u003c/li\u003e\n \u003cli\u003eImpacts: effects on root causes and sustained significant changes produced by a development intervention, directly or indirectly, intended or unintended (e.g.: sustained drop in the incidence of a given disease).\u003c/li\u003e\n \u003cli\u003eResults: the whole outputs, outcomes and impacts of a development intervention.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTo determine the challenge-agnostic results in the context of any procurement activity, the methodology developed by Alessandrello et al. 2021 [1] was applied. This methodology used the Theory of Change to establish results in the short, medium and long term period and applicable to any tender. These challenge-agnostic results were defined based on literature review of recognised existing framings and different public procurement of innovation and pre-commercial procurements documentation (See Table 1) while answering to the following two questions:\u0026nbsp;\u003c/p\u003e\n\u003col\u003e\n \u003cli\u003eHow are the new digital services, acquired through PiPPi Community of Practice (CoP) activities increasing the value in the health system?\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eTo what extent is PiPPi CoP succeeding in their PCP/PPI procedures according to both internal and external stakeholders?\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u003cstrong\u003eTable 1\u003c/strong\u003e Recognized existing framings and PCPs/PPIs used to build PiPPi challenge-agnostic results.\u003c/p\u003e\n\u003cp\u003eThis table describes the recognized existing framings, PCPs and PPIs practical experiences that were identified for review to identify initial list of challenge-neutral results (outputs, outcomes and impact) that could contribute to answer the two key questions. \u0026nbsp;The first question \u0026ldquo;How are the new digital services, acquired through PiPPi CoP activities, increasing the value in the health system?\u0026rdquo; was answered by consulting the recognized existing framings while PCP and PPI experience supported to answer the second question \u0026ldquo;To what extent is PiPPi CoP succeeding in their PCP/PPI procedures according to both internal and external stakeholders?\u0026rdquo;\u003c/p\u003e\n\u003ctable style=\"width: 4.1e+2pt;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eRecognized existing framings\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003ePCP \u0026amp; PPI experiences\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eICHOM web\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eDECIPHER PCP (FP7 GA 288028)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003ePROMS-PREMS (ACI Wales)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eSTOPandGO PPI (FP7 GA 621013)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eMAST Manual \u0026nbsp;[12]\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eANTISUPERBUGS PCP (H2020 GA 688878)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eCAHS Framework [13]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLIVEINCITE PCP (H2020 GA 727558)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eNICE digital evidence standards framework\n \u003c!--[if supportFields]\u003e\u003cspan style='mso-element: field-begin'\u003e\u003c/span\u003e\u003cspan style='color:windowtext'\u003e\u0026nbsp;CITATION Nat18 \\l 1027 \u003c/span\u003e\u003cspan style='mso-element:field-separator'\u003e\u003c/span\u003e\u003c![endif]--\u003e[14]\n \u003c!--[if supportFields]\u003e\u003cspan style='mso-element:field-end'\u003e\u003c/span\u003e\u003c![endif]--\u003e\n \u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eSTARS PCP (H2020 GA 727585)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eEuropean Commission - Digital Transformation [15]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eRITMOCORE PPI (H2020 GA 727796)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eMEAT Framework [16]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eObservatori del Sistema de Salut de Catalunya\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eCOSMIN [17]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eImpact Evaluation Framework - HUS Virtual Hospital [18]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eAmong the collected documentation, 122 challenge-agnostic results were defined by a group of PiPPi consortium members through a prioritization process made in two stages.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCARE selection criteria were applied to each identified result by both internal and external stakeholder. The CARE selection criteria were: Clearness (the meaning of the result sublevel was clear and easy to be understood); Availability (the possibility to quantify the data making the result sublevel available); the Relevance (the result sublevel was relevant for the PiPPi CoP, the public procurement of innovation and the innovation in public procurement in the healthcare) and the Exemplarity (the result sublevel was sufficiently representative of the level and/or the sublevel intended to assess).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn total, twenty-two internal and external stakeholders from six different countries responded the questionnaire. Participants came from different European Union (EU) countries (Spain (six), Italy (five), Austria (three), Sweden (three), Netherlands (three) and UK (two)) and were representing different disciplines and stakeholders categories (Innovation Project Management (five), Industry (five), Academia (five), Hospital Management (five), Consultant (one) and Patient (one)). Patient representative only responded to the patient-level outcomes. From this group of respondents, ten were part of PiPPi consortium partners and twelve were external.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFirst, experts scored individual results in agreement with the criteria Clearness and Exemplarity. Second, experts scored and categorized individual results in agreement with their Relevance and Availability. Among the identified 122 items, 26 were classified as essential results to be taken into account when defining any PCP/PPI.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ePiPPi 122 results (outputs, outcomes and impacts) were divided into five categories depending on the main stakeholder level of a generic digital health intervention. Four of the categories correspond to the elements directly related to the actors mainly affected by the innovative interventions resulting from PCPs and PPIs procedures (patients, healthcare professionals, healthcare providers and healthcare system) and the fifth to the elements directly related to socioeconomic aspects. Additionally, each category/level is also divided in sub-levels (see Table 2).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2\u003c/strong\u003e Proposal of stakeholder levels, their definition and sublevels for result classification\u003c/p\u003e\n\u003cp\u003eThis table shows the proposal of stakeholders levels, their descriptions and sublevels as defined by PiPPi consortium. Four levels matches to the main actors impacted by the implementation of the innovative interventions and the fifth relates to socioeconomic elements.\u0026nbsp;\u003c/p\u003e\n\u003ctable style=\"width: 4.3e+2pt;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eLevel\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eDescription\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eSub-level\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003ePatient-level results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003eIt includes the measurement of patient experiences and results which are patient-centred, focused on their experiences and relevant results of the iteration with the health system.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003ePatient-Reported Outcome Measures\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003ePatient-Reported Experience Measures\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eDeterminants of Health\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eLong-Term treatment improvement\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eHealthcare professional-level results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eA set of results regarding the culture of the organization and the healthcare professional skills.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eBenefits for the healthcare professional\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eWorkplace environment/culture results\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003eHealthcare provider-level results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003eA set of results that can measure the activity and performance of the health service.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eOrganizational aspects\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eCosts\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eProcess\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eTechnological Aspects\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\"\u003e\n \u003cp\u003eHealth system-level results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"3\"\u003e\n \u003cp\u003eA set of results aimed to measure the activity of the health system including quality, efficiency and accessibility.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eEconomic Sustainability\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eSafety and Sustainability\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eLong-Term treatment improvement\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eSocio-economic-level results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eA set of results aimed to measure other factors, which can significantly affect health status.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eSocial determinants\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eEconomic Evaluation and HTA\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eFrom the 122 validated and selected results (Supplementary Table 1), a prioritization process allowed to define a set of 26 short, medium and long-term results classified as essential results that PiPPi consortium proposed to take into account when defining any demand-driven PCP/PPI (See Table 3).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 3\u003c/strong\u003e Proposal of PiPPi essential set of results for a PCP/PPI per level and sublevel\u003c/p\u003e\n\u003cp\u003eThis table shows the proposal of 26 short, medium and long-term results classified as essential from the 122 initially validated list of results that PiPPi consortium recommends to consider when defining demand-driven PCP/PPI.\u0026nbsp;\u003c/p\u003e\n\u003ctable style=\"width: 100%;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eLevel\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eSublevel\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eResult\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"11\"\u003e\n \u003cp\u003ePatient\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eOutcome measures reported by patients\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eHealth-related Quality of Life\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003eExperience outcomes reported by patients\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eAcceptability of the solution\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eOverall satisfaction with treatment\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eUnderstanding treatment\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eLong-term disease management support\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003eHealth determinants\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eHospital-acquired infections\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eAccuracy of diagnosis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eQuality Adjusted Life Years (QALY)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eReduced clinical errors\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eLong-term treatment improvement\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eMortality\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eDisability\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eHealthcare Professional\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eBenefits for healthcare professionals\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eProportion of adherence of professionals to clinical guidelines\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"8\"\u003e\n \u003cp\u003eHealthcare provider\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"4\"\u003e\n \u003cp\u003eOrganizational aspects\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eNumber of hospitalizations\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eTimeout\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eNumber of re-hospitalizations / readmission rate\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eNumber of beds per hospitalized patient\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eCosts\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eMaintenance costs\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eProcess\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eWorkflow\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eTechnological aspects\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eProduction efficiency\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eData security\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"5\"\u003e\n \u003cp\u003eHealth system\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\"\u003e\n \u003cp\u003eSafety and sustainability\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eData security\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eSecurity environment\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\"\u003e\n \u003cp\u003eLong-term treatment improvement\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eMortality\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eMorbidity\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eDisability\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eSocioeconomic impact\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eEconomic and health technology assessment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eCost-Utility Analysis; Cost-Effectiveness Analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eIn 2022, for the very first time, the 122 challenge-agnostic results set was used to support the buyers of two European Commission funded pre-commercial procurements to design their demand-driven value-based innovation procurements permeable to value: oncNGS PCP (H2020 GA 874467) and TIQUE PCP (H2020 GA 965356).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOncNGS is a cross border pre-commercial procurement whose lead procurer is Sciensano from Belgium, who is buying in name and on behalf of OncNGS buyers group, including Sciensano (BE), Alleanza Contro il Cancro (IT), Institut Curie (FR), Institut Catala d\u0026rsquo;Oncologia (ES), Institut Jules Bordet (BE), Ludwig-Maximilians-Universitaet Muenchen (DE), Charite \u0026ndash; Universitaetsmedizin Berlin (DE) and Hospices Civils de Lyon (FR). The main goal of the oncNGS PCP is to develop an integrated solution for diagnostic, predictive, prognostic and theranostic analysis of liquid biopsies from solid tumours (including appropriate haematological indications such as lymphoma) using NGS technology.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTIQUE is a cross border pre-commercial procurement whose lead procurer is Hospital de la Santa Creu i Sant Pau from Spain, who is buying in name and on behalf of TIQUE buyers group, including Hospital de la Santa Creu i Sant Pau from Spain (ES), Azienda Sanitaria Locale Avellino (IT) and Region V\u0026auml;sterbotten (SE). The main goal of the TIQUE PCP is to develop integrated care approaches to transformation the health and care services for patients with advanced heart failure or at risk.\u003c/p\u003e\n\u003cp\u003eThe steps performed to apply PiPPi challenge-agnostic results were as follows:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStep 1\u003c/em\u003e - Taking into consideration all the inputs received from all the relevant stakeholders during the needs definition and the needs assessment, what matters to the patients and the potential improvements of the ratio between the patient-relevant outcome measurements and the costs per patient in delivering those outcomes, the agnostic PiPPi results set was analysed to identify and select the results that could be achieved addressing the given challenge and benefitting all the identified stakeholders along the value chain (starting from the results identified as essential, then continuing with the remaining ones).\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStep 2\u0026nbsp;\u003c/em\u003e- It was considered relevant to add few additional results not present in the original PiPPi results sets. Such addition was categorized per stakeholder to ensure not to lose the stakeholder relevance of the added results.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStep 3\u0026nbsp;\u003c/em\u003e- After having gone through the challenges PiPPi selected results and the newly defined ones, it was identified whether such results were achievable as procurements short term outputs, mid-term outcomes or long term outcomes.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStep 4 -\u0026nbsp;\u003c/em\u003eThen it was analysed how each of the challenges-selected results and the newly defined ones may be achieved as short term outputs from the contract (e.g.: reduction of process complexity, acceptability of the solution, novel treatment opportunities, understanding care plan pathways), how they influence the PCP/PPI contract outputs definition (e.g.: requirements to be delivered during the contract life and validated by end users; end users satisfaction rates) and how such outputs could be measured and assessed during the contract life (e.g.: bid template, evaluation of the bid, awarding criteria, contract monitoring, contract governance, expertise necessary to evaluate and to monitor contract execution, outputs based payments and no-health outcomes based payments).\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStep 5\u003c/em\u003e - Then it was analysed how each of the challenges-selected results and the newly defined ones may be achieved as midterm outcomes/no-health outcomes from the contract (e.g.: production efficiency, avoidance of unnecessary treatment, waiting time reduction), how they influence the PCP/PPI contract outcomes/no-health outcomes definition (e.g.: requirements to be delivered during the contract life and validated by end users; end users satisfaction rates; evaluation of the intervention and its implementation) and how such outcomes/no-health outcomes could be measured and assessed during the contract life (e.g.: bid template, evaluation of the bid, awarding criteria, contract monitoring, contract governance, expertise necessary to evaluate and to monitor contract execution, outputs based payments and no-health outcomes based payments, evidence generation and health outcomes payments, risk sharing agreement).\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStep 6\u0026nbsp;\u003c/em\u003e- Then it was analysed how each of the challenges-selected results and the newly defined ones may be achieved as long-term results from the contract (e.g.: improvements in health outcomes, cost utility analysis, increased equality in cancer care, 5-year survival rate after diagnosis), how they influence the PCP/PPI contract long-term results definition (e.g.: requirements to be delivered during the contract life and validated by end users; end users satisfaction rates, evaluation of the intervention and its implementation) and how such long-term results could be measured and assessed during the contract life during and beyond the contract life (e.g.: bid template, evaluation of the bid, awarding criteria, contract monitoring, contract governance, expertise necessary to evaluate and to monitor contract execution, outputs based payments and no-health outcomes based payments, evidence generation and health outcomes payments, risk sharing agreement).\u003c/p\u003e"},{"header":"3 Results","content":"\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThanks to OncNGS and TIQUE pre-commercial procurements, these steps were applied for the very first time to define contracts \u0026ldquo;permeable to the value\u0026rdquo; across all their whole life from their publication to the completion of the R\u0026amp;D contracts execution. Furthermore, the learnings gathered in these two procurement procedures served to apply in 2023 the methodology to a PPI procedure focused on the personalised surgical process for joint replacements, activity making part of EcoQUIP Plus project (COSME GA 857790). The buyer of this procurement was Consorci Corporaci\u0026oacute; Sanit\u0026agrave;ria Parc Taul\u0026iacute; (ES), who was looking for a solution to allow total monitoring and traceability of both prosthesis manufacturing and surgical processes, integrating with current 3D laboratory tools and incorporating and maintaining a quality system based on ISO13485.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003eDuring the application of the six steps for designing demand-driven procurements \u0026ldquo;permeable to value\u0026rdquo; (Supplementary Table\u0026nbsp;1), it was recognised that this methodological framework directly impacts to at least five elements of the tender\u0026rsquo;s definition (Supplementary Tables\u0026nbsp;2 and 3):\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eDescription of the object of the contract\u003c/em\u003e: independently of whether contract concerns the execution of works or the supply of products or the provision of services, the object of the contract should be described in terms of impact to be generated, in terms of functional, performance and quality requirements, in terms of life-cycle and short, mid and long-terms results. The public buyer[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] sets minimum requirements in order to avoid an abusively low-performing tender, but is not overly prescriptive as regards the means of achieving a desired outcome. Economic operators enjoy openness and flexibility to reach the optimal performance.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eTechnical offer/bid template\u003c/em\u003e: the technical proposal template or bid template is prepared to facilitate participating economic operators (bidders) to clear frame how the identified need(s) will be addressed. It prompts information about the work, product and/or service is planned to be delivered and how the evidence is planned to be generated. Associating short-term, mid-term and long-term outcomes to the bid template is a way to ensure that bidders align their proposals towards their achievement.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eAwarding Criteria\u003c/em\u003e: the awarding criteria [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] are the criteria that constitute the basis on which a contracting authority selects the best tender and awards a contract. These criteria must not be prejudicial to fair competition and ensure compliance with the principles of transparency, non-discrimination and equal treatment, with a view to ensuring objective comparison of tenders. The criteria chosen must be objective and objectively quantifiable, as: the delivery of short-term outcomes (key performance requirements and adherence to specifications, the measurements of patient-relevant outcome and of the costs per patient in delivering those outcomes), the technical, quality and organizational feasibility to deliver the mid-term outcomes and to generate evidence towards the achievement of the long-term outcomes, the plan for an active contribution of all the impacted stakeholders, including the patients.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eContract monitoring\u003c/em\u003e: Contracts should be periodically monitored and the level of the achievement of the results (milestones, interim deliverables and short, mid and long-term results) should be evaluated. The monitoring and evaluation process consists in the assessment of quantitative and qualitative indicators obtained at different contract stages. Some indicators should be directly collected and recorded when accomplished, during the contract lifetime, and some other will be gathered in specific time-points in the contract duration. Thus, including measurable and assessable outcomes as part of the indicators/results that will monitored along contract duration is an additional manner to understand to what extent these outcomes are being achieved during contract duration and/or are to be achieved after contract completion.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eExpertise necessary to evaluate and to monitor\u003c/em\u003e: as already mentioned the \u0026ldquo;value\u0026rdquo; delivery will be guaranteed by considering the demand side as a whole, represented by all its different stakeholders\u0026rsquo; categories. Only multidisciplinary teams, representing all the impacted stakeholders, with different skills, knowledge and expertise relevant to the object of the contract will be able to correctly assess the technical offers against the awarding criteria and to correctly monitor the contracts and evaluate their achievements. The necessary expertise should be identified upfront to ensure the preparedness of the buyer to evaluate and monitor when required.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eThe detailed analysis of the expected short-term outputs/no-health outcomes of each of the procurement procedures enabled to identify what results were susceptible to be included in the object of the contract description, in the technical offer/bid template, in the awarding criteria, in the contract monitoring and in terms of the expertise necessary to assess and monitor bidder offers and contracts progresses.\u003c/p\u003e \u003cp\u003eOn the other hand, the detailed analysis of the expected long-term outcome/impact of each of the procurement procedures enabled to identify what quantitative/qualitative indicators were considered relevant to the buyers once the interventions may be ultimately implemented, adopted and scaled up beyond the PCP and PPI contracts termination.\u003c/p\u003e \u003cp\u003eThe results of the application of the six steps to the five elements were the following:\u003c/p\u003e \u003cp\u003e \u003cem\u003eStep 1 Results\u003c/em\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eIn 2022 and the first semester of 2023, the proposed results set proposed for any demand-driven PCP/PPI (Supplementary table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) was applied to three contracts: two PCPs, OncNGS and TIQUE and one of the PPI procedures in EcoQUIP Plus.\u003c/p\u003e \u003cp\u003e \u003cem\u003eStep 2 Results\u003c/em\u003e \u003c/p\u003e \u003cp\u003eOverall, the application of this methodology led to identify eleven new or modified results in OncNGS, eight in TIQUE and five in EcoQUIP Plus to be included or modified to the original PiPPi agnostic results set (See Tables\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e, \u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e and \u003cspan refid=\"Tab6\" class=\"InternalRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eOncNGS additional results set This table shows the eleven additional results identified after the application of the methodology to OncNGS project\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLevel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSublevel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eOutput / Outcome\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003ePatient\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDeterminants of Health\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNovel treatment opportunities\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDeterminants of Health\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eReduction of variability in patient results\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHealthcare Professional\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBenefits for the HCP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eProportion of professional with opportunities to provide care to patients\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003eHealthcare provider\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDiagnostics\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCompliancy with new regulatory rules\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCosts: Maintenance costs increase\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eErrors reduction / Increase in accuracy\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSensitivity\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eHealth System\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEconomic Sustainability\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDiagnostics\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEconomic Sustainability\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePrognosis\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEconomic Sustainability\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eHealthcare providers transparent budget monitoring\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTIQUE additional results set This table shows the eight additional results identified after the application of the methodology to OncNGS project\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLevel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSublevel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eOutput / Outcome\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003ePatient\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDeterminants of Health\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eInitially defined in PiPPi as: Adverse drug effects.\u003c/p\u003e \u003cp\u003eUpdated and detailed in TIQUE as:\u003c/p\u003e \u003cp\u003eDrug-related health problems (adverse effects, safety, drug-related morbidity)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePatient-Reported Experience Measures\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCommunity linkages\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHealthcare Professionals\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBenefits for the HCP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eProportion of professional with opportunities to provide care to patients\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eHealthcare provider\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eHome Hospitalization\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNumber of visits in primary care\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eUnplanned potentially preventable inpatient episodes\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrganizational aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePatient wills\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHealth System\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLong-Term treatment improvement\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSurvival rate after diagnosis\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab6\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 6\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eEcoQUIP Plus additional results set This table shows the five additional results identified after the application of the methodology to EcoQUIP Plus project\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLevel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSublevel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eOutput / Outcome\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePatient\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDeterminants of Health\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLess surgery complications\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHealthcare Professionals\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBenefits for the HCP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eProportion of professional with opportunities to provide care to patients\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eHealthcare provider\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCosts\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3D Equipment\u003c/p\u003e \u003cp\u003eWater and Energy Reduction costs\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eProcess\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3D equipment training and resources\u003c/p\u003e \u003cp\u003eSoftware training and resources\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTechnological aspects\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eEnsure UE MDR 2017/745 and ISO 13485 completion\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eStep 3 Results\u003c/em\u003e:\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eIn the relevant rows of the Supplementary Table\u0026nbsp;2 of the different contracts (OncNGS, TIQUE and EcoQUIP Plus), the first two columns contain the identified short and mid-term results, while the identified long-term results are in the in the relevant rows of the Supplementary Table\u0026nbsp;3.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eStep 4 Results\u003c/em\u003e:\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eIn the relevant rows of the Supplementary Table\u0026nbsp;2 of the different contracts (OncNGS, TIQUE and EcoQUIP Plus), the columns making explicit mention to \u0026ldquo;object of the contract\u0026rdquo;, \u0026ldquo;Bid Template\u0026rdquo;, \u0026ldquo;Awarding Criteria\u0026rdquo;, \u0026ldquo;Contract Monitoring\u0026rdquo;, \u0026ldquo;Expertise necessary to evaluate/ monitor\u0026rdquo; contain the identification of the tender elements the short-term outputs influence and what needs to be taken into account element by element.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eStep 5 Results\u003c/em\u003e:\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eIn the relevant rows of the Supplementary Table\u0026nbsp;2 of the different contracts (OncNGS, TIQUE and EcoQUIP Plus), the columns making explicit mention to \u0026ldquo;object of the contract\u0026rdquo;, \u0026ldquo;Bid Template\u0026rdquo;, \u0026ldquo;Awarding Criteria\u0026rdquo;, \u0026ldquo;Contract Monitoring\u0026rdquo;, \u0026ldquo;Expertise necessary to evaluate/ monitor\u0026rdquo; contain the identification of the tender elements the mid-term outcomes/no-health outcomes influence and what needs to be taken into account element by element.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eStep 6 Results\u003c/em\u003e:\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eThe Supplementary Table\u0026nbsp;3 contains the identified long-term outcomes of oncNGS, TIQUE and EcoQUIP Plus.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eIn both OncNGS and TIQUE contracts, the governance and the risk sharing agreement have been designed towards the accomplishment of given performance indicators, the commencement of measuring to enable the evidence generation, the contract monitoring assessment and the outcome-based payments.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eIn the case of EcoQUIP Plus contract, the governance has been designed towards the accomplishment of given performance indicators, the contract monitoring assessment and the commencement of measuring to enable the evidence generation.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eIn summary, the proposed \u0026ldquo;Permeability to value\u0026rdquo; framework results from the application in six steps of PiPPi challenge-agnostic results set to five public tenders\u0026rsquo; elements (see Table\u0026nbsp;\u003cspan refid=\"Tab7\" class=\"InternalRef\"\u003e7\u003c/span\u003e):\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab7\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 7\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eMethodological framework for designing demand-driven value-based innovation procurements This table shows the six steps to apply PiPPi challenge-agnostic results to any contract to be designed \u0026ldquo;permeable to the value\u0026rdquo; and what contract elements are impacted by such application\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003e\u0026ldquo;Permeability to value\u0026rdquo; framework\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSix steps to select/identify\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003echallenge-agnostic results sets\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003eFive public tenders elements\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eimpacted by the selected/identified results\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStep 1 From PiPPi results set selection of results that could be achieved addressing the given challenge and benefitting all the identified stakeholders along the value chain\u003c/p\u003e \u003cp\u003eStep 2 Addition of relevant results not present in the original PiPPi results sets (addition categorised per stakeholder)\u003c/p\u003e \u003cp\u003eStep 3 Identification of results as short term outputs, mid-term outcomes or long term outcomes.\u003c/p\u003e \u003cp\u003eStep 4 Analysis of how short term influence the PCP/PPI contract outputs definition and how such outputs could be measured and assessed during the contract life\u003c/p\u003e \u003cp\u003eStep 5 Analysis of how midterm outcomes/no-health influence the PCP/PPI contract outcomes/no-health outcomes definition and how such outcomes/no-health outcomes could be measured and assessed during the contract life\u003c/p\u003e \u003cp\u003eStep 6 Analysis of how long-term results influence the PCP/PPI contract long-term results definition and how such long-term results could be measured and assessed during the contract life during and beyond the contract life\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1. Description of the object of the contract\u003c/p\u003e \u003cp\u003e2. Technical offer/bid template\u003c/p\u003e \u003cp\u003e3. Awarding Criteria\u003c/p\u003e \u003cp\u003e4. Contract monitoring\u003c/p\u003e \u003cp\u003e5. Expertise necessary to evaluate the bids and to monitor the contracts\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"4 Discussion","content":"\u003cp\u003eTen years after the publication of the Public Procurement Directive 2014/24/EU\u0026nbsp;[4], which gave contracting authorities in the Member States a new awarding procedure to pursue innovation, it is evident that operationalising awarding criteria and allocating scores against them can be a complex task that requires further action.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eBuilding upon existing recognised methodological frameworks as well as from the experience accrued from previous EU projects and PCP and PPI contracting practices, the described methodological approach proposes an applicable and usable system to apply from the preparation stages of the innovation procurement procedures. Such methodological approach leads to the definition of value-based contracts and enables procurement organisations and suppliers to demonstrate the real value of the solutions once implemented and adopted to a wider market as well as to patients, health care professionals, health care institutions, health care services and, ultimately, to the society as whole.\u003c/p\u003e\n\u003cp\u003eTaking into account the starting point of the “permeability to value” methodology, that are the “theory of change” and the PiPPi challenge-agnostic results set, this methodology enables by default the identification of the “public values” that matters to who consumes the resulting public services, the benefices and the sacrifices along the healthcare delivery chain, what matters most to the patients, the patient-relevant outcome measurements and the costs per patient in delivering those outcomes improvements.\u003c/p\u003e\n\u003cp\u003eThe perspective of value differs depending on each targeted stakeholders’ point of view. The proposed methodological approach takes this subjective element into consideration and integrates it as cornerstone of the procurement of innovation process. Embedded within an improved cycle of best practices, the new methodological approach opens a constructive debate that permits to consolidate the knowledge and perspectives from different stakeholders into a robust contracting procedure.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe effort to produce a collaborative framework of continuous improvement is linked to a shared commitment to participate in the continuous improvement of the management processes that supports the mainstreaming of innovation procurement strategic policy to create demand-driven adoption of innovative solutions for the transformation of the health and social systems.\u003c/p\u003e\n\u003cp\u003eIn accordance with the “Theory of Change”, the application of present methodological approach allows public organisations and suppliers to identify measurable results relevant to the affected stakeholders and assess them in short and midterm period. As follow up of current PCP and PPI procedures where the solutions will be ultimately deployed in procurement organisations’ settings, this methodology enables to eventually assess the long-term results of the actions and according to the stakeholder level (as described in Table 2).\u003c/p\u003e\n\u003cp\u003eAs a result, the tender will be “permeable to value”, in other words the “value” will have the ability to pass through the whole contract life. The “value” will be planned to influence the definition of the different aspects of the tender as the ex-ante description of the object of the contract, the requirements and the awarding criteria (in terms of functionalities, performance and outcomes favouring the technology neutrality), the template the bidders are required to use to submit their bids, the description of the contract execution with its deliverables and milestones (including what and how far the ex-post results will be observable and measurable), the contracts monitoring and the expertise necessary to evaluate the bids and monitor the contracts and the payment model (including the allocation between the buyers and suppliers of risks based on the achievements of defined ex-post measurable outcomes). All aspects that influence further the innovation adoption model and how and when the “value” is ultimately delivered to all the relevant stakeholders should be explicated by:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e-\u0026nbsp; \u0026nbsp;Unbundling versus bundling the research and development (R\u0026amp;D) phase with the commercialization phase;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e-\u0026nbsp; \u0026nbsp;defining the contracts governance allocating the decision rights and risks between the buyers and suppliers;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e-\u0026nbsp; \u0026nbsp;Selecting the tender procedure and designing the competition.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e“Value” will be guaranteed by considering both the demand side as a whole - represented by all the different stakeholders’ categories that will benefit from the implementation, adoption and scale up of the innovative technological intervention resulting from the PCP/PPI contracts (patients, healthcare professionals, healthcare providers, health systems) – including the supply side.\u003c/p\u003e\n\u003cp\u003eThis methodology is still under validation since two of the three mentioned contracts are in progress at the date of publication of this article:\u003c/p\u003e\n\u003cp\u003e- OncNGS precommercial procedure \u003cstrong\u003e[21]\u003c/strong\u003e already successfully awarded the Framework Agreement, completed Phase 1 contract and Phase 2 contracts and is now contracting Phase 3.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e-\u0026nbsp; \u0026nbsp;TIQUE precommercial procedure \u003cstrong\u003e[22]\u003c/strong\u003e already successfully awarded the Framework Agreement, completed Phase 1 and Phase 2 contracts and awarded Phase 3 contracts, currently under execution.\u003c/p\u003e\n\u003cp\u003e-\u0026nbsp; \u0026nbsp;The personalised surgical process for joint replacements of EcoQUIP Plus successfully awarded and executed the contract.\u003c/p\u003e\n\u003cp\u003eFurthermore, the proposed methodology also showed to be easily adopted by any demand-driven innovation procurement procedure to assess the value of the resulting healthcare interventions brings to the different stakeholders, since it was also applied to two further public procurements of innovations launched by two of\u0026nbsp;\u003cstrong\u003eRaDAR PPI\u003c/strong\u003e (COSME GA 101036228) (36) buyers looking at integrated solutions\u0026nbsp;for rapid detection of antimicrobial resistance and patient management in hospital settings. During 2024 both Institut Català d'Oncologia and Università di Napoli Federico II launched their RaDAR PPI tenders (GA 101036228) applying this methodology and writing their contracts “permeable to value” [23, 24].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn the meantime, the authors of this article are actively boosting its use across the different demand driven innovation procurements they are defining at both European and local level with the aim at creating a corpus of public contracts enabling the validation of the methodology. The interaction with the affected stakeholders of each challenge is key to support the identification of results that can lead to improvement of the agnostic set of results. At the same time, the authors are working on defining a process to disseminate its wide use across all the upcoming healthcare demand-driven innovation contracts and to validate the methodology.\u003c/p\u003e\n\u003cp\u003eOverall the proposed methodology demonstrated to be agnostic to the procedure, since it could be successfully used for both pre-commercial procurements and public procurement of innovations, and agnostic to the challenge, since it could be used on a wide range of diseases and patients' journeys (cancer diagnostic and treatment, health and social care of heart failure patients and prosthesis manufacturing and implant). However, there is the need to maintain and update PiPPi results set and put in place a process to periodically validate the updated set and the definition of the results.\u003c/p\u003e\n\u003cp\u003eIt is important to highlight that in case of the PCP procedures, the selected results can only be assessed in the short term and medium-term, but not in the long term. Thanks to the application of this methodology, in subsequent PPIs, the results of the procurements come not only from its application to the tender definition but also from the results of the previous PCPs already defined permeable to the value.\u003c/p\u003e\n\u003cp\u003eThe Value Based Procurement Subcommission of the Catalonian Health System Innovation \u0026amp; Transformation Departmental Commission [25] is working on the creation of an Observatory of the Value Based Procurements taking place in the Catalonian health system. This Observatory would offer to the authors the opportunity to assess the effectiveness of the proposed methodology by comparing “permeable to value” procurements with Catalonian value based contracts designed and launched without making use of this methodology. The “permeability to value” effectiveness could be measured by evaluating the degree of achievement of the expected results, the degree of viability and preparedness of any subsequent value based contract and the use of resources for its implementation. The measure of the “permeability to value” effectiveness could be of support to define the inclusion criteria for the application of this methodology to future public procurements of innovation.\u003c/p\u003e"},{"header":"5 Conclusions","content":"\u003cp\u003eThis article provides a methodological framework to design and implement demand-driven public contracts permeable to the value taking into account the perspective of involved stakeholders. The methodological framework has been already applied to five innovation public contracts in the healthcare sector (two of them under execution, one of them completed and two of them in the tendering phase).\u003c/p\u003e \u003cp\u003eThe application of the proposed methodological approach in further PCP and PPI initiatives will support the refinement, and ultimately the validation, of mechanisms that will allow public contracting authorities and the innovation ecosystem to demonstrate the value generated from the implemented demand-driven innovation procurement procedures across time for any given sector.\u003c/p\u003e \u003cp\u003eThe Value Based Procurement Subcommission of the Catalonian Health System Innovation \u0026amp; Transformation Departmental Commission considers the proposed \u0026ldquo;value permeability\u0026rdquo; framework a valid methodology to design demand-driven innovation procurements.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eBPQR: best price quality ratio\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCoP: Community of Practice\u003c/p\u003e\n\u003cp\u003eEC: European Commission\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eGA: Grant Agreement\u003c/p\u003e\n\u003cp\u003eGDP: Gross Domestic Product\u003c/p\u003e\n\u003cp\u003eMEAT: Most Economically Advantageous Tender\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ePCP: Pre-commercial Procurement\u003c/p\u003e\n\u003cp\u003ePiPPi:\u0026nbsp;Platform for Innovation of Procurement and Procurement of Innovation\u003c/p\u003e\n\u003cp\u003ePPI: Public Procurement of Innovation\u003c/p\u003e\n\u003cp\u003eR\u0026amp;D: research and development\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approvals\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn case of oncNGS: the study to be performed within Phase 3 contract is a corroboration study and is not a clinical study.\u0026nbsp;This study was reviewed and deemed exempt by\u0026nbsp;SCIENSANO, oncNGS project coordinating entity and lead procurer.\u0026nbsp;Samples to be used in the corroboration study can be both from biobanks and prospectively collected. Written informed consent is being obtained from all individual participants whose samples are prospectively collected and included in Phase 3 corroboration study. The prospective samples collection is being\u0026nbsp;performed in line with the principles of the Declaration of Helsinki and approval\u0026nbsp;was\u0026nbsp;granted by the Ethics Committee of\u0026nbsp;Hospital Universitari de Bellvitge (ES)\u0026nbsp;(Date.25\u003csup\u003eth\u003c/sup\u003e of February 2021 /No. PR381/20)\u003c/p\u003e\n\u003cp\u003eIn case of TIQUE: the study to be performed within Phase 3 contract is in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of\u0026nbsp;Hospital de la Santa Creu i Sant Pau (ES)\u0026nbsp;(Date 25\u003csup\u003eth\u003c/sup\u003e of May 2021 /No. IIBSP-TIQ-2021-44). Written informed consent is being obtained from all individual participants who will be included in Phase 3 study.\u003c/p\u003e\n\u003cp\u003eIn case of EcoQUIP Plus: the study performed within the contract was a cost analysis study. This study was reviewed and deemed exempt by\u0026nbsp;Parc Taul\u0026iacute; Institut d\u0026rsquo;Investigaci\u0026oacute; I Innovaci\u0026oacute; I3PT (ES).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOpen access\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article\u0026rsquo;s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article\u0026rsquo;s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit https://creativecommons.org/licenses/by-nc-nd/4.0/\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe would like to thank the following for their assistance in the production of this proposal of methodological framework:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eMaria Sanchis-Amat, EIT Health, and Anna Garcia-Valls, Ag\u0026egrave;ncia de Residus de Catalunya, for the support they gave while working in AQuAS - Ag\u0026egrave;ncia de Qualitat i Avaluaci\u0026oacute; Sanit\u0026agrave;ries de Catalunya in identifying the results of oncNGS and TIQUE respectively.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eAll the patients and professionals coming from the list of entities below and who participated to the PiPPi CARE surveys and to validate the agnostic results set applied to this novel methodology to oncNGS, TIQUE, EcoQUIP Plus and RaDAR public tenders (in alphabetical order):\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAlleanza Contro Il Cancro (IT)\u003c/p\u003e\n\u003cp\u003eAzienda Sanitaria Locale Avellino (IT)\u003c/p\u003e\n\u003cp\u003eCharite - Universitaetsmedizin Berlin (DE)\u003c/p\u003e\n\u003cp\u003eErasmus Universitair Medisch Centrum Rotterdam (NL)\u003c/p\u003e\n\u003cp\u003eHelsingin ja Uudenmaan Sairaanhoitopiirin Kuntayhtym\u0026auml; (FI)\u003c/p\u003e\n\u003cp\u003eHospices Civils De Lyon (FR)\u003c/p\u003e\n\u003cp\u003eHospital de la Santa Creu i Sant Pau (ES)\u003c/p\u003e\n\u003cp\u003eHospital Universitari Vall d\u0026apos;Hebron (ES)\u003c/p\u003e\n\u003cp\u003eInstitut Catal\u0026agrave; d\u0026apos;Oncologia (ES)\u003c/p\u003e\n\u003cp\u003eInstitut Curie (FR)\u003c/p\u003e\n\u003cp\u003eInstitut Jules Bordet Asbl (BE)\u003c/p\u003e\n\u003cp\u003eKarolinska University Hospital (SE)\u003c/p\u003e\n\u003cp\u003eKing\u0026apos;s College Hospital Nhs Foundation Trust (UK)\u003c/p\u003e\n\u003cp\u003eLudwig-Maximilians-Universitaet Muenchen (DE)\u003c/p\u003e\n\u003cp\u003eMedizinische Universitaet Wien (AU)\u003c/p\u003e\n\u003cp\u003eOspedale San Raffaele Srl (IT)\u003c/p\u003e\n\u003cp\u003eParc Taul\u0026iacute; Institut d\u0026rsquo;Investigaci\u0026oacute; I Innovaci\u0026oacute; I3PT (ES)\u003c/p\u003e\n\u003cp\u003eRegion Vasterbotten (SE)\u003c/p\u003e\n\u003cp\u003eSciensano (BE)\u003c/p\u003e\n\u003cp\u003eUniversit\u0026agrave; Di Napoli Federico II (IT)\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eAll members of the Value Based Procurement Subcommission of the Catalonian Health System Innovation \u0026amp; Tranformation Departamental Commission who reviewed the article and the methodology: (in alphabetical order) Cristina Adroher, Rossana Alessandrello, Cari Almaz\u0026aacute;n, Neus Colet, Yolima Cossio, Ramon Maspons, Gemma Enfedaque, Antoni Gilabert, Josep Maria Gim\u0026eacute;nez, Encarna Grifell, Cari Pontes, Laura Sampietro, N\u0026uacute;ria Umbert\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eThe authors gratefully acknowledge funding from the European Union\u0026apos;s Horizon 2020 research and innovation programme under Grant Agreements 826157 (Coordination and support action, PiPPi), 874467 (PCP, oncNGS), 965356 (PCP, TIQUE), 857790 (PPI, EcoQUIP Plus) and 101036228 (PPI, RaDAR).\u003c/p\u003e\n\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\n\u003cp\u003eRossana Alessandrello conceived the original idea, designed and developed the theoretical framework and directed the research. Rossana Alessandrello, Vict\u0026ograve;ria Valls-Comamala, Ion Arrizabalaga, Ux\u0026iacute;o Meis-Pinheiro, Olman Elizondo-Cordero equally contributed to the manuscript. Ramon Maspons and Anna Garcia-Alt\u0026eacute;s contributed to the final version of the manuscript and supervised the research. All authors provided critical feedback, discussed the results and commented on the manuscript and approved the version to be published.\u003c/p\u003e\n\u003ch2\u003eData Availability\u003c/h2\u003e\n\u003cp\u003eAll data generated or analysed during this study are included in this published article and its supplementary information files. The supplementary information associated to this article contains the complete set of proposed results for demand driven PCP/PPI and the application of the PiPPi result set to the call for tender documents of OncNGS, TIQUE and EcoQUIP Plus.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eA. G. I. M. P. U. E. C. O. S.-A. M. M. R. Alessandrello R, \u0026ldquo;Teoria del canvi, resultats neutrals respecte al tipus de necessitats no satisfetes i permeabilitat de les compres p\u0026uacute;bliques d\u0026apos;innovaci\u0026oacute; al valor en l\u0026apos;\u0026agrave;mbit de la salut,\u0026rdquo; vol. 104, 2021. \u003c/li\u003e\n\u003cli\u003e\u0026ldquo;Expert Panel on effective ways of investing in health(EXPH),public procurement in healthcare systems,,\u0026rdquo; 2021.\u003c/li\u003e\n\u003cli\u003eE. Comission, \u0026ldquo;PUBLIC PROCUREMENT IN HEALTHCARE SYSTEMS. Fact sheet accompanying the Opinion by the Expert Panel on Effective Ways of Investing in Health (EXPH),\u0026rdquo; Publication Office of the European Union, Brussels, 2021.\u003c/li\u003e\n\u003cli\u003eEuropean Parliament, \u0026ldquo;Directive 2014/24/EU of the European Parliament and of the Council of 26th of February on public procurement and repealing Directive 2004/18/EC,\u0026rdquo; 2014.\u003c/li\u003e\n\u003cli\u003eO. Chiappinelli, L. M. Giuffrida and G. Spagnolo, \u0026ldquo;Public Procurement as an Innovation Policy: Where Do We Stand?,\u0026rdquo; Vols. ZEW Discussion Paper No. 23-002, 2023. \u003c/li\u003e\n\u003cli\u003eJ. \u0026amp;. H. O. Alford, \u0026quot;Public Value Pragmatism as the Next Phase of Public Management,\u0026quot; vol. 38(2), 2008. \u003c/li\u003e\n\u003cli\u003eT. R. H. G. G. Achim Walter, \u0026ldquo;Value Creation in Buyer\u0026ndash;Seller Relationships: Theoretical Considerations and Empirical Results from a Supplier\u0026apos;s Perspective,\u0026rdquo; vol. 30, no. 4, 2001. \u003c/li\u003e\n\u003cli\u003eM. E. Porter, \u0026ldquo;What is value in Health Care?,\u0026rdquo; The New England Journal of Medicine, vol. 363, no. 26, p. 5, 2010. \u003c/li\u003e\n\u003cli\u003e\u0026ldquo;Expert Panel on effective ways of investing in Health (EXPH). Defining value in \u0026quot;value-based healthcare\u0026quot;,\u0026rdquo; 2019.\u003c/li\u003e\n\u003cli\u003eN. Dimitri, \u0026ldquo;Best value for money\u0026rdquo; in procurement\u0026quot;,\u0026rdquo; vol. 13, no. 2, 2013. \u003c/li\u003e\n\u003cli\u003eAQuAS on behalf of PiPPi consortium, \u0026quot;D5.4. A core set of outcome indicators,\u0026quot; 2021.\u003c/li\u003e\n\u003cli\u003eMethoTelemed project, \u0026ldquo;The MAST Manual. Model for ASsessment of Telemedicine,\u0026rdquo; 2010.\u003c/li\u003e\n\u003cli\u003eCanadian Academy of Health Sciences | Acad\u0026eacute;mie canadienne des sciences de la sant\u0026eacute;, \u0026ldquo;Making an Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research. Report of the Panel on the Return on Investments in Health Research,\u0026rdquo; Canadian Academy of Health Sciences | Acad\u0026eacute;mie canadienne des sciences de la sant\u0026eacute;, Ottawa, Canad\u0026agrave;, 2009.\u003c/li\u003e\n\u003cli\u003eNational Institute for Health and Care Excellence, \u0026ldquo;Evidence standards framework for digital technologies,\u0026rdquo; NICE, 2018.\u003c/li\u003e\n\u003cli\u003eEuropean Commission, \u0026ldquo;A Europe fit for the digital age. Empowering people with a new generation of technologies,\u0026rdquo; European Commission, 2019. [Online]. Available: https://commission.europa.eu/strategy-and-policy/priorities-2019-2024/europe-fit-digital-age_en.\u003c/li\u003e\n\u003cli\u003eJ. C. C. P. Y. V. H. B. G\u0026ouml;tz Gerecke, \u0026ldquo;How Procurement unlocks value-based healthcare,\u0026rdquo; Boston Consulting Group \u0026amp; MedTech Europe, 2020.\u003c/li\u003e\n\u003cli\u003eCOSMIN initiative, \u0026ldquo;COnsensus-based Standards for the selection of health Measurement INstruments,\u0026rdquo; COSMIN initiative, [Online]. Available: https://www.cosmin.nl/.\u003c/li\u003e\n\u003cli\u003eS. L.-T. P. (. V. H. 2. p. Arvonen, \u0026ldquo;We are getting there \u0026ndash; Virtual Hospital 2.0 project summary.,\u0026rdquo; Helsinki and Uusimaa Hospital District (HUS), Helsinki, 2019.\u003c/li\u003e\n\u003cli\u003eE. Commission, \u0026ldquo;Guidance on Innovation Procurement,\u0026rdquo; Brussels, 2021 .\u003c/li\u003e\n\u003cli\u003eO. a. E. Commission, \u0026ldquo;Setting the Award Criteria,\u0026rdquo; SIGMA Public Procurement Briefs. OECD Publishing, Paris, 2011.\u003c/li\u003e\n\u003cli\u003eOncNGS Consortium (Sciensano), \u0026ldquo;e-Notification (publicprocurement.be),\u0026rdquo; 2022. [Online]. Available: https://enot.publicprocurement.be/enot-war/preViewNotice.do?noticeId=452212.\u003c/li\u003e\n\u003cli\u003eTIQUE Consortium (Fundaci\u0026oacute; de Gesti\u0026oacute; Sanit\u0026agrave;ria de l\u0026apos;Hospital de la Santa Creu i Sant Pau), \u0026ldquo;Plataforma de Serveis de Contractaci\u0026oacute; P\u0026uacute;blica,\u0026rdquo; 2022. [Online]. Available: https://contractaciopublica.cat/en/detall-publicacio/f98c4c0d-5ade-19f6-a114-7b258c16a9d2/113124982.\u003c/li\u003e\n\u003cli\u003eRaDAR Consortium (Institut Catal\u0026agrave; d\u0026rsquo;Oncologia), \u0026ldquo;RaDAR call for tender launched by INSTITUT CATAL\u0026Agrave; D\u0026apos;ONCOLOGIA. Servei d\u0026rsquo;abordatge integral per al control dels microorganismes resistents als antimicrobians,\u0026rdquo; Plataforma de Serveis de Contractaci\u0026oacute; P\u0026uacute;blica, 2023.\u003c/li\u003e\n\u003cli\u003eRaDAR Consortium (UNIVERSIT\u0026Agrave; DI NAPOLI FEDERICO II), \u0026ldquo;RaDAR call for tender launched by UNIVERSIT\u0026Agrave; DI NAPOLI FEDERICO II. Servizi informatici innovativi per la individuazione rapida di microrganismi multiresistenti ai farmaci e per la gestione e il controllo intelligenti della antibioticoresistenza,\u0026rdquo; Acquisti in rete certificata da Anac, 2024.\u003c/li\u003e\n\u003cli\u003eDiari Oficial de la Generalitat de Catalunya, \u0026ldquo;ORDRE SLT/5/2022, de 25 de gener, per la qual es crea la Comissi\u0026oacute; Departamental d\u0026apos;Innovaci\u0026oacute; i Transformaci\u0026oacute; del Sistema de Salut,\u0026rdquo; 2022.\u003c/li\u003e\n\u003cli\u003ePlatform for Innovation of Procurement and Procurement of Innovation Platform for Innovation of Procurement and Procurement of Innovation (PiPPi project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; PiPPi consortium, 2018. [Online]. Available: https://cordis.europa.eu/project/id/826157.\u003c/li\u003e\n\u003cli\u003eInternational Consortium for Health Outcomes Measurement, \u0026ldquo;International Consortium for Health Outcomes Measurement,\u0026rdquo; [Online]. Available: https://www.ichom.org/.\u003c/li\u003e\n\u003cli\u003eDistributed European Community Individual Patient Healthcare Electronic Record (DECIPHER Project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; DECIPHER consortium, 2012. [Online]. Available: https://cordis.europa.eu/project/id/288028.\u003c/li\u003e\n\u003cli\u003eAgency for Clinical Innovation , \u0026ldquo;Agency for Clinical Innovation (NSW government),\u0026rdquo; [Online]. Available: https://aci.health.nsw.gov.au/make-it-happen/prms.\u003c/li\u003e\n\u003cli\u003eSustainable Technology for Older People \u0026ndash; Get Organised (STOP and Go Project), \u0026ldquo;CORDIS. EU research Results,\u0026rdquo; STOP and Go consortium, 2014. [Online]. Available: https://cordis.europa.eu/project/id/621013.\u003c/li\u003e\n\u003cli\u003eANTI-SUPERBUGS Precommercial procurement project, \u0026ldquo;CORDIS. EU research results,\u0026rdquo; ANTI-SUPERBUGS PPI consortium, 2016. [Online]. Available: https://cordis.europa.eu/project/id/688878.\u003c/li\u003e\n\u003cli\u003eLifestyle intervention in the perioperative process through digital service (LIVEINCITE PCP project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; LIVEINCITE PCP consortium, 2016. [Online]. Available: https://cordis.europa.eu/project/id/727558.\u003c/li\u003e\n\u003cli\u003eEmpowering Patients by Professional Stress Avoidance and Recovery Services (STARS project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; STARS consortium, [Online]. Available: https://cordis.europa.eu/project/id/727585/es.\u003c/li\u003e\n\u003cli\u003eArrhythmias monitoring and comprehensive care (Ritmocore Project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; Ritmocore consortium, 2016. [Online]. Available: https://cordis.europa.eu/project/id/727796.\u003c/li\u003e\n\u003cli\u003eAg\u0026egrave;ncia de Qualitat i Avaluaci\u0026oacute; Sanit\u0026agrave;ries de Catalunya (AQuAS), \u0026ldquo;Observatori del Sistema de Salut de Catalunya,\u0026rdquo; Ag\u0026egrave;ncia de Qualitat i Avaluaci\u0026oacute; Sanit\u0026agrave;ries de Catalunya (AQuAS), [Online]. Available: https://observatorisalut.gencat.cat/ca/inici.\u003c/li\u003e\n\u003cli\u003eNGS diagnostics in 21st century oncology: the best, for all, at all times (oncNGS PCP project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; oncNGS PCP consortium, 2019. [Online]. Available: https://cordis.europa.eu/project/id/874467.\u003c/li\u003e\n\u003cli\u003eInnovaTIve care services, enhanced with technology, to deliver QUick rEsponses for individuals with advanced heart failure and complex care needs through integrated care (TIQUE PCP project), \u0026ldquo;CORDIS. EU research results,\u0026rdquo; TIQUE PCP consortium, 2021. [Online]. Available: https://cordis.europa.eu/project/id/965356.\u003c/li\u003e\n\u003cli\u003eEcoQuip+ project, \u0026quot;Delivering efficiency, quality and sustainability in healthcare,\u0026quot; EcoQuip+ consortium, 2021. [Online]. Available: https://www.ecoquip.eu/.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"public procurement, public procurement of innovation, value-based procurement, pre-commercial procurement, health system, value-based contracts, outcomes-based contracts, risk-sharing contracts, value-based healthcare, healthcare, demand-driven, theory of change","lastPublishedDoi":"10.21203/rs.3.rs-5609699/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5609699/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTo be fully capable to respond to health system arising needs not adequately addressed through the available existing solutions, public administrations are required to spur the development, implementation, adoption and scale up of new goods or services. Innovation procurement enables public sector to act either as the promoter of the research and development intended to create adequate solutions to the arising needs (pre-commercial procurements, PCP) or to act as early adopter of innovative solutions and services that are not yet available on large scale commercial basis and that still need to generate the evidence of the value they bring to impacted stakeholders (public procurement of innovation, PPI).\u003c/p\u003e \u003cp\u003eThis article proposes a methodological framework to design and implement demand-driven public contracts \u0026ldquo;permeable to value\u0026rdquo;, referring to the ability of the \u0026ldquo;value\u0026rdquo; to pass through the whole contract life. This framework has been defined following the methodology described by Alessandrello et al. 2021 [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e], including literature review of recognized framings and PCP/PPI experiences followed by a prioritization process by stakeholders. Furthermore, it presents the results of the application of the proposed framework to three European Commission co-funded call for tenders: two cross-border PCP and one PPI.\u003c/p\u003e \u003cp\u003eThis methodological framework provides public buyers with a novel method to design and implement demand-driven public contracts \u0026ldquo;permeable to value\u0026rdquo;, that take into account the perspective of all the stakeholders impacted by such contracts. Overall, it has the potentiality to enable public buyers, to demonstrate the value generated from the implemented demand-driven innovation procurements.\u003c/p\u003e","manuscriptTitle":"Permeability to value: a methodological framework for designing demand-driven value-based innovation procurements and its application","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-26 09:22:14","doi":"10.21203/rs.3.rs-5609699/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"404b70be-7440-4e66-abee-d99e8fcc5092","owner":[],"postedDate":"March 26th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-05-28T06:38:48+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-26 09:22:14","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5609699","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5609699","identity":"rs-5609699","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}