Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances?

Keywords Start for Life (SfL), Family Hubs, Early childhood development, 1,001 critical days, Parental mental health, Parent-infant relationships, Public health evaluation, Mixed-methods evaluation, Cost-benefit analysis, Health inequalities ALL Metrics - Views Downloads How to cite this article J B, M S, L D et al. Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.14019.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Study Protocol [version 1; peer review: 3 approved with reservations] Barlow J1, Sleed M https://orcid.org/0000-0002-2321-2116 2, Day L3, [...] Davey C3, Conti G https://orcid.org/0000-0003-4699-2490 4,5, Deighton J2, Edbrooke-Childs J2, Weber Costa G4, Reynolds S https://orcid.org/0009-0004-1431-514X 2, Cann H https://orcid.org/0000-0001-6208-4041 1, Crossan L2, Fearon P6Barlow J1, Sleed M https://orcid.org/0000-0002-2321-2116 2, [...] Day L3, Davey C3, Conti G https://orcid.org/0000-0003-4699-2490 4,5, Deighton J2, Edbrooke-Childs J2, Weber Costa G4, Reynolds S https://orcid.org/0009-0004-1431-514X 2, Cann H https://orcid.org/0000-0001-6208-4041 1, Crossan L2, Fearon P6 PUBLISHED 18 Dec 2025 Author details Author details 1 University of Oxford Department of Social Policy and Intervention, Oxford, England, UK 2 Anna Freud Centre, London, England, UK 3 Ecorys, Birmingham, UK 4 Department of Economics, University College London, London, England, UK 5 Social Research Institute, University College London, London, England, UK 6 University of Cambridge Department of Psychology, Cambridge, England, UK 2 Anna Freud Centre, London, England, UK 3 Ecorys, Birmingham, UK 4 Department of Economics, University College London, London, England, UK 5 Social Research Institute, University College London, London, England, UK 6 University of Cambridge Department of Psychology, Cambridge, England, UK Barlow J Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Sleed M Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Review & Editing Day L Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Visualization, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Visualization, Writing – Review & Editing Davey C Roles: Data Curation, Formal Analysis, Methodology, Project Administration, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Methodology, Project Administration, Writing – Review & Editing Conti G Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Deighton J Roles: Formal Analysis, Investigation, Methodology Roles: Formal Analysis, Investigation, Methodology Edbrooke-Childs J Roles: Conceptualization, Funding Acquisition, Methodology Roles: Conceptualization, Funding Acquisition, Methodology Weber Costa G Roles: Data Curation, Formal Analysis, Investigation, Methodology, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Methodology, Writing – Review & Editing Reynolds S Roles: Methodology, Project Administration, Visualization Roles: Methodology, Project Administration, Visualization Cann H Roles: Data Curation, Formal Analysis, Project Administration, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Project Administration, Writing – Review & Editing Crossan L Roles: Methodology, Project Administration, Visualization Roles: Methodology, Project Administration, Visualization Fearon P Roles: Conceptualization, Funding Acquisition, Methodology Roles: Conceptualization, Funding Acquisition, Methodology OPEN PEER REVIEW REVIEWER STATUS This article is included in the Policy Research Programme gateway. Launched in 2021, the joint Department of Health and Social Care and Department for Education Family Hubs and Start for Life (SfL) programme aims to improve support through pregnancy and early childhood. Funding is provided to 75 local authorities (LAs) with high levels of deprivation. As part of this, £170m DHSC funding was allocated for Start for Life services which focus on the 1,001 days between pregnancy and age 2. In January 2025, £57 million was announced for the continuation of Start for Life services in 2025/26. The programme seeks to improve key outcomes for parents and infants. This paper describes the national evaluation of the programme. This study uses a mixed-method multilevel triangulation design to assess the impact and cost-benefit of SfL, and to determine which interventions are effective, for whom, and in what contexts. The evaluation consists of two main components: 1) a process and outcome evaluation: to assess changes at both service and individual levels. Qualitative data will be analysed thematically, while quantitative process data will be examined descriptively and triangulated with longitudinal qualitative data and broader process evaluation findings. System-level factors (e.g., workforce attitudes and practices) will be explored to determine their impact on parents and infants; 2) an impact and economic evaluation: this component will use the Synthetic Difference-in-Differences method to construct counterfactuals and evaluate whether the programme affected key parent’s and children’s outcomes, using both individual and LA-level data; it will also include a cost analysis, in which the study will monetise SfL benefits and compare them against the cost of the programme to assess whether it is more effective than usual care, with associated net cost savings. In 2021, the Department for Health and Social Care and Department for Education launched the Family Hubs and Start for Life programme. This programme aims to improve support for families through pregnancy and early childhood. 75 local authorities (LAs) with high levels of deprivation received funding. £170m of this funding was allocated to help LAs develop Start for Life services, focussing on the 1,001 days between pregnancy and age 2. In January 2025, £57 million was announced to help these services continue in 2025/26. The programme aims to help improve key health and wellbeing outcomes for parents and infants. This research uses a range of methods to assess the impact of the Start for Life programme, compare the costs to benefits, and determine which interventions are effective and in what circumstances. There are two main strands to the evaluation: 1) assessing changes for parents, infants and staff at services and individual levels. This will be explored by using interviews and surveys; 2) an impact and economic evaluation. This strand will evaluate whether the programme affects key health and wellbeing outcomes for parents and infants. It will also include a cost-benefit analysis to see if the additional cost of the programme leads to an improvement in outcomes, compared to normal services. Start for Life (SfL), Family Hubs, Early childhood development, 1,001 critical days, Parental mental health, Parent-infant relationships, Public health evaluation, Mixed-methods evaluation, Cost-benefit analysis, Health inequalities Corresponding Author(s) Barlow J ([email protected]) Grant information: This project is funded by the National Institute for Health Research (NIHR) under its [NIHR Policy Research Programme (NIHR204678)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © Crown copyright, 2025 J B et al.. This open access work is licensed under the Open Government Licence v3.0 How to cite: J B, M S, L D et al. Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.14019.1) First published: 18 Dec 2025, 5:123 (https://doi.org/10.3310/nihropenres.14019.1) Latest published: 18 Dec 2025, 5:123 (https://doi.org/10.3310/nihropenres.14019.1) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The use of a mixed-method, multi-level triangulation design will enable us to explore what works for whom, under what circumstances, in addition to the factors associated with optimal outcomes; Assessment of the causal impact of SfL will be enhanced through the use of the Synthetic Difference-in-Differences method, which combines the strengths of traditional synthetic control and difference-in-difference approaches to provide credible estimates of treatment effects; The collection of longitudinal qualitative data from participating families will enable us to examine the views of families about SfL over time; The system-level process evaluation will enable us to map the development of local SfL programmes across all 75 LAs drawing on a national survey of strategic leads and analysis of the national MI dataset shared by DHSC as well as to capture more detail through a smaller number of deep-dive case studies; Although the process evaluation will provide a rich picture with regard to SfL services and stakeholder views about its implementation, and also some evidence regarding impact on a small number of outcomes, the limited number of case study sites may preclude wider generalisation. To mitigate this risk, semi-structured interviews will be conducted with strategic leads for SfL on a rolling basis, covering all LAs over the course of years 1-3 of the evaluation, alongside a national survey of strategic leads. The 1,001 days between pregnancy and age 2 set the foundations for an individual’s cognitive, emotional, and physical development, shaping long-term health, educational, and social outcomes1. This period represents an important window of opportunity to optimise outcomes for all children. However, negative early experiences, compounded by social inequalities, can lead to disparities in early development that persist and widen across later childhood, adolescence and into adulthood2. In response to this, in 2022 the government introduced the Family Hubs and Start for Life (SfL) programme, a c.£300 million package aimed at transforming support for families in 75 local authorities (LAs) in England with high levels of deprivation during this foundational period. The programme was extended over 25/26 with an investment of a further £126 million. The Family Hubs and Start for Life programme, jointly led by the Department for Education (DfE) and the Department of Health and Social Care (DHSC), is central to delivering on the Government’s ambition to raise the healthiest generation of children, supporting the commitment in the Plan for Change (December 2024) to strengthen and join up family support to improve support through pregnancy and early childhood. The programme seeks to integrate and enhance existing services through family hubs, ensuring parents and carers can access the support they need, when they need it. Its overarching goal is to improve health and education outcomes and reduce inequalities among babies, children, and families across England. The programme’s objectives include to: 1. Improve immediate and long-term health, well-being and education outcomes by providing universal and targeted support to parents and carers 2. Contribute to a reduction in health and education inequalities for babies, children, and families across England through improving access to services for hard-to-reach groups and those in most need of additional support 3. Contribute to the evidence base for what works, for whom and in which contexts regarding the health, well-being and education outcomes for babies, children and families. In England, there has been sustained cross-governmental focus on identifying early years services and policies that will level up the opportunities for children, regardless of their background3. These policies represent a response to the evidence that by age 5, some children are already so far behind in terms of development that they will be unlikely to close the gap. The existing literature supports the use of a range of interventions during the 1,001 days4, and many of these have been included in the Healthy Child Programme, which begins in pregnancy and is underpinned by a model of proportionate universalism5. However, evidence regarding the actual benefits of population and area-level models of provision such as SfL during the early years are limited in the UK to the Sure Start Initiative6,7, and, more recently, A Better Start (process data only)8, and to evaluations of single national programmes (e.g. FNP)9. The current proposal will address this evidence gap by evaluating the implementation, impact and cost-benefit of the Start for Life funding across England. Initial involvement included appointing a Peer Researcher as Co-Investigator, consulting with parent/carer champions at Anna Freud, and recruiting members to the Parent/Carer Advisory Group (PCAG). Early PCAG activities focused on identifying individual goals for involvement, refining sampling criteria, engaging community gatekeepers, and reviewing interview schedules with researchers. The peer researcher was involved in applying for funding, which included identifying the importance of setting research questions that focussed on examining the experiences of parents/carers and how these may change over time. The Peer Researcher co-facilitates PCAG involvement, with members contributing to the review of study materials, piloting interviews, refining data collection tools, supporting transcript analysis, and co-creating the parent/carer survey, which some members presented to the DHSC. Their input has been consistent throughout, particularly in shaping inclusive language and ensuring the study reflects diverse parenting experiences. Outcome measures (PHQ-9, GAD-7, MORS) were selected based on Start for Life routine measures, with the Peer Researcher and PCAG advising on their implementation to ensure sensitivity, accessibility, and inclusivity, while also addressing safeguarding considerations. Sampling criteria were developed collaboratively during the scoping phase to ensure inclusion of families representative of local populations and priority groups, with input from the Peer Researcher, PCAG, and parent/carer champions helping to identify underrepresented communities and refine recruitment strategies. PCAG members also supported outreach and contributed ideas for long-term engagement to ensure participants feel valued throughout the study. Finally, PCAG members are actively involved in shaping dissemination plans, advising on preferred formats, languages, and outreach methods, and will help raise awareness and share findings within local communities and parenting networks to ensure relevance and reach across diverse audiences. This study will employ a mixed-method multilevel triangulation design to assess the impact of SfL at individual, service and population levels. The aim of the evaluation is to determine which interventions and delivery elements have an impact, for whom, and in what contexts through the collection of process and impact data at the above levels. The quantitative and qualitative evidence generated will enable us to assess the factors that are associated with the optimisation of service provision and best outcomes for families, as well as the characteristics of less optimal service models. Specifically: 1. The system-level process evaluation will capture an overview of local SfL programmes, drawing on primary and secondary data from all 75 LAs to: a) understand key factors informing programme and service design and delivery, and b) inform the selection of a sub-set of LAs for ‘deep dive’ exploration. 2. The individual-level process evaluation will explore families’ needs, expectations, access to, and experiences of services in the 1,001 days. This will be done through a longitudinal case series design10. 3. The impact and economic evaluation will assess the benefits of SfL using both individual and LA-level data from a range of national data sources, and the Synthetic Difference-in-Differences method to construct counterfactuals. We will combine primary and secondary data collection and analysis, drawing on theory-based and framework analyses to assess programme effectiveness and outcomes. Informed written consent will be obtained from all participants taking part in interviews. The following sections have been organised according to the above levels of data collection. 1. Process and outcome evaluation System-level process evaluation: sampling for the system-level process evaluation will be twofold and will include: i) primary and secondary data collection with all 75 funded LAs, including a cross-sectional survey of all SfL leads, a rolling programme of qualitative interviews with all SfL leads phased over years 1–3 of the evaluation, and descriptive analysis of MI data reported by LAs; ii) primary and secondary data collection in 10–12 deep dive case study LAs, sampled purposively with attention to maturity (a mix of Trailblazers and non-Trailblazers), LA characteristics, and meeting data requirements for a robust process evaluation. The phasing for the latter is as follows: 4 LAs – longitudinal (years 1, 2 and 3); 3–4 LAs – cross-sectional (year 2 only); 3–4 LAs – cross-sectional (year 3 only). The sampling of professionals for qualitative research will be undertaken purposively to achieve a representative cross-section of strategic and operational-level SfL personnel drawn from the 10–12 LAs. We anticipate conducting qualitative interviews with 10–20 professionals per case study LA, supplemented with local workforce surveys, although this will be determined by the size and composition of SfL delivery in each site. Individual-level process evaluation: this sample will be nested within the ‘deep-dive’ longitudinal case study LAs; a total of 60 families will be recruited in pregnancy from the 4 LAs and followed up longitudinally (n = 15 families per LA). Eligible families will be identified through local community groups and settings (such as children’s centres, community-based antenatal services, and online fora). A combination of self-referrals and active targeted recruitment by the research team will be used to purposively sample participants as we aim to recruit families prior to and regardless of their knowledge of or access to any Start for Life offers. The unit of referral will be a “family”, and this can include any composition of family unit, as long as there is at least one caregiver and one unborn child. The sampling criteria will be developed in the scoping phase with the PPI group and through stakeholder consultation to enable the sampling of families that are representative of the local population and priority groups that the Start for Life interventions aim to support (e.g., families from Black and minoritized ethnic groups and/or neurodivergent parents/carers). The criteria will be informed by the theory of change as well as local needs and priorities. Each year, an additional sample of 8–10 parents in different stages of the perinatal period will be recruited to participate in a single interview about service experience. They will be purposively sampled to capture additional cross-sectional data about parents’ experiences of a broad range of infant feeding, parental mental health, and parent-infant relationship support services. For these interviews, we will purposively sample service users who may not be represented by the case study sample. Outcome evaluation for individuals living in funded Sf Lareas: We will look at individual-level pre- to post-intervention scores and indices of reliable change across all SfL services where routine outcome measures of parental mental health and parent-infant relationships are collected and reported. We will also use a cross-sectional survey to collect self-reported outcomes and experiences from a sample of n=1200 parents living in SfL-funded LAs. The selection of sites will be based on triangulation of data from multiple sources, including financial reporting, maturity self-assessment ratings, and MI data. 2. Impact and economic evaluation We will estimate the causal impact of the SfL on key outcome indicators from individual and LA-level data using the Synthetic Difference-in-Differences (SDID) method. Synthetic control methods in general rely on the existence of several untreated units, composing what is known as the ‘donor pool’. The SDID assigns weights to untreated units such that the weighted average of their pre-treatment outcomes closely tracks the outcome trajectory of the treated units, while also weighting pre-treatment time periods11. LAs that did not receive SfL funding will be initially included in the donor pool. The cost analysis will build on the findings from the impact evaluation to compare SfL benefits with the costs of the programme. A range of quantitative and qualitative data is being collected as follows: 1. Process and outcome evaluation For the purpose of the service-level evaluation, we will collate summary-level Management Information (MI) data for all 75 programme LAs on a rolling basis. Data collection will also involve qualitative interviews with strategic-level personnel and focus groups with operational personnel engaged in SfL planning and delivery, which will focus on capturing their views about the programme, its implementation, and impact on families using semi-structured topics guides. Interviews with caregivers’ will aim to capture their perceptions regarding their needs, expectations, access to and experiences of support in relation to breastfeeding, parental mental health and parent-infant relationships, and the overall SfL offer. Workforce surveys (i.e. short surveys exploring perceptions of programme coherence; effectiveness, knowledge and awareness of SfL offers and their implementation, self-reported practice outcomes and multi-disciplinary working); documentary analysis will involve extraction of key information from all tools and frameworks, workforce planning and training documents, and service-level data (e.g. Red Books). SfL teams will be responsible for gathering consent for contact by the evaluation team to administer the quantitative and qualitative data collection – either on a rolling basis, or via a single consent-gathering exercise prior to the first data collection point. We will also seek consent for data linkage. Following consent to contact, SfL teams will supply sample lists and URLs to the evaluation team. Surveys will be administered by sending an online survey link to all consenting parents and carers, with URLs. It will be enabled for self-completion in 7 languages. Response tracking and two rounds of reminders with a telephone booster will be used if necessary. Telephone surveys will be deployed for non-responders to maximise the overall response rate and minimise sample bias. For the 4 longitudinal case study LAs, we will liaise with SfL leads to plan and coordinate primary data collection, and surveys will be web-based and administered with a diverse workforce sample for case study LAs, with sample-building and cascading coordinated via SfL and Family Hubs teams. For the purpose of the individual-level process data, caregivers in each case study family will be invited to take part in a yearly in-depth in-person interview and telephone interviews every 3 months. Caregivers will also complete the same standardized questionnaires that will be used for the individual-level impact evaluation to enable data triangulation. To maximise the retention rates and ensure high quality interviews, the same researchers will follow up the same families over time, thus building up a rapport and engaging the families in meaningful interviews over the study period. Support will be available to ensure accessibility (including interpreters, mobile devices and data for online meetings, or travel expenses for face-to-face meetings). Generous participant payments will be made for every part of the research in which a family participates, in the form of shopping vouchers to compensate for their time. The PPI group will advise on engagement and retention in the study on an ongoing basis. For the purpose of the individual-level data from families living in funded SfL areas, we will assess parental depression (PHQ-912), anxiety (GAD-713), and parent-infant bonding (MORS14) pre- and post-intervention for a limited number of interventions using locally-collected data, where available. The cross-sectional survey of parents and carers will capture their need for support with infant feeding, parental mental health and parent-infant relationships, the extent to which they feel that these needs have been met by local services, and, where they have accessed such services, their experiences of these. The survey will also include the PHQ-9, GAD-7, and MORS. 2. Impact and economic evaluation We will use national microdata at the individual and LA-level. The main source of data will be the Fingertips Child and Maternal Health Data, available from the Office for Health Improvement and Disparities public website, which includes key measures such as vaccination coverage, health visiting metrics, breastfeeding rates at 6–8 weeks and ASQ-3 scores. Individual-level data from the Education & Child Health Insights from Linked Data (ECHILD) database will also be used15. The ECHILD contains linked administrative data from NHS hospitals, as well as education and social care services, covering all children in England. It includes data from: Community Services Data Set (CSDS), which records information such as mandated health visits and referrals to community health services; Maternity Services Data Set (MSDS), which includes variables such as APGAR score, birth weight, and first feed status; Mental Health Services Data Set (MHSDS), which includes information on secondary care for parents and children, for mental health and learning disabilities (such as referral, reason, and treatment); Hospital Episode Statistics (HES), which includes information on admitted patient care, critical care, accidents and emergencies, outpatient care and mortality records; Children In Need (CIN)/Children Looked After (CLA), with records of all children referred to children’s social care services, including those looked after by a Local Authority. Cost data will be drawn from SfL administrative records and the best available sources (e.g. the NHS National Cost Collection for the costs of NHS services, or previous studies). 1. Process and outcome evaluation Qualitative and quantitative data from both service and individual-level process data collection: the qualitative data will be analysed thematically, and the quantitative data will be examined descriptively, including patterns of change over time in parental mental health. This will be triangulated with the longitudinal qualitative data and the wider process evaluation data to explore the impact of system-level factors (e.g., changes in workforce attitudes and practices) on parents. Individual-level data from families living in funded SfL areas: this data will be analysed using a range of descriptive and inferential statistics to assess reliable change over time (i.e. pre to post) for each of the three main outcomes being measured, and, for the survey responses, to assess differences across groups. 2. Impact and economic evaluation Impact evaluation: to estimate the causal impact of SfL on parents’ and children’s outcomes, we will use the Synthetic Difference-in-Differences (SDID) method16. The SDID combines features of traditional synthetic control and difference-in-differences approaches by considering both time and unit weights in the estimation of treatment effects. Specifically, the method assigns weights to untreated units and pre-treatment time periods such that the weighted average of the untreated pre-treatment outcomes closely tracks the trajectory of the treated units, and the weighted average of pre-treatment periods closely resembles post-treatment outcomes for untreated units. Treatment effects are then estimated via a doubly-weighted two-way fixed effects regression, where both unit and time weights are applied. We will also examine whether effects differ by relevant groups (e.g. socioeconomic status, ethnicity), and include sibling fixed effects in the analysis of individual-level data. Following best practice, we will perform extensive robustness tests, and also assess the sensitivity of the results to the use of alternative methods. Cost analysis: drawing on findings from the process and impact evaluations, we will monetize short-term benefits of SfL by combining the estimates obtained from the best available sources (e.g. the NHS National Cost Collection for the costs of NHS services, or previous studies) and compare these against the cost of the programme, to assess whether SfL is more effective than usual care, with associated net cost savings. We will follow NICE guidelines (including recommended discount rate and performing extensive sensitivity analysis) and carry out a Cost-Benefit Analysis to compute the Return on Investment (RoI) from the perspective of those providing and using the services. For the CBA, we will supplement the short-term outcomes from the impact evaluation with medium-term health and developmental outcomes (from the National Pupil Database and the National Child Measurement Programme), using surrogate-index methods to account for not-yet-realised returns17. Upon consultation with DHSC, we will also provide an assessment of the trajectories of the returns, including the likely break-even point18. 3. A synthesis of the data across the workstreams will enable us to address the overarching questions described above and will be undertaken in conjunction with our PPI co-applicants. Specifically: a. System-level process data will be combined with impact data to assess the factors that are associated with optimal services and the best outcomes for families, and to elicit best practices that can be applied across the 75 funded areas and beyond. b. System and service-level data will be used to identify the factors that drive and shape service delivery, and the individual/family process-level data will be used to identify the acceptability of services to families, and stakeholder views regarding factors that affect uptake and impact. c. We will test the overall Theory of Change for the SfL programme, drawing on all data sources to explain the observed impacts at programme and strand levels, and synergies between SfL and Family Hubs. The main ethical considerations will be concerned with a) participant burdens and/or risks b) the physical and psychological safety of the study researchers; and c) consent, data protection/ confidentiality. We will ensure that all researchers abide by the recognised GDPR, GCP, and Safeguarding procedures, and that they have been appropriately trained and certified. The final outputs, engagement and dissemination plans will be co-produced with families, and will aim to identify key audiences and channels, and leverage maximum research impact through a range of methods including policy workshops, practice guidelines and parent-friendly information. We will also connect with networks established via the national centre for family hubs and AFC's Early Years in Mind network. Participants and the wider population will be kept informed about the progress of the study and findings throughout the duration of the study using a range of accessible methods (i.e. newsletters, leaflets, website, blog and social media) and where relevant different languages and formats. We will support our PPI group to perform a role as Learning Champions, raising awareness of the evaluation within the SfL LAs, and disseminating more widely via parenting networks as multipliers. Through this approach we will aim to boost awareness and reach with culturally and linguistically diverse communities and with families who may not be accessing SfL services. The findings of the research will be disseminated to relevant audiences using conference presentations, and publications in academic, policy and methods journals. We will also hold at least one roundtable session, bringing together civil servants, professional associations, third sector partners, and parent and carer representatives to consider the implications for policy and practice. Ethics committee approval has been secured from the University of Oxford and IDREC. The study also has ethical approval from the Hampstead Research Ethics Committee (24/LO/0371). Informed written consent will be obtained from all participants. The findings of the research will be disseminated to relevant audiences using conference presentations, and publications in academic, policy and methods journals. We will also hold at least one roundtable session, bringing together civil servants, professional associations, third sector partners, and parent and carer representatives to consider the implications for policy and practice. This study/project is funded by the NIHR. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. There will be no data sharing because: a) the interview and survey data cannot be sufficiently pseudonymised to be made publicly available, and explicit consent for data to be shared in this way was not collected. The remaining data is already publicly available. Faculty Opinions recommendedReferences - 1. Shonkoff JP: Capitalizing on advances in science to reduce the health consequences of early childhood adversity. JAMA Pediatr. 2016; 170(10): 1003–1007. PubMed Abstract | Publisher Full Text - 2. Andrew J, Robinson D, Hutchinson J: Closing the gap: trends in education attainment and disadvantage. London: Education Policy Institute, 2017. Reference Source - 3. Including for example: NHS Long Term Plan; the Maternity Transformation Programme; Transforming Children and Young People’s Mental Health; No child left behind (Public Health England (PHE) 2020) and Childhood Vulnerability in England (Children’s Commissioner 2019). - 4. Asmussen K, Bains Z: What works to enhance the effectiveness of the healthy child programme: an evidence update. London: Early Intervention Foundation, 2018. Reference Source - 5. Department of Health and Social Care: Healthy child programme: 0 to 19 health visitor and school nurse commissioning. (n.d.). Reference Source - 6. Cattan S, Conti G, Farquharson C, et al.: The health impacts of universal Early Childhood Interventions: evidence from Sure Start. (IFS Working Paper W22/43). Institute for Fiscal Studies, 2022. Reference Source - 7. Carneiro P, Cattan S, Conti G, et al.: The effect of Sure Start on youth misbehaviour, crime and contacts with children's social care. (IFS Report R338). Institute for Fiscal Studies, 2024. Reference Source - 8. The National Lottery Community Fund: A Better Start evaluation documents. (n.d.). Reference Source - 9. Robling M, Lugg-Widger FV, Cannings-John R, et al.: Nurse-led home-visitation programme for first-time mothers in reducing maltreatment and improving child health and development (BB:2-6): longer-term outcomes from a randomised cohort using data linkage. BMJ Open. 2022; 12(2): e049960. PubMed Abstract | Publisher Full Text | Free Full Text - 10. Grant A, Bugge C, Wells M: Designing process evaluations using case study to explore the context of complex interventions evaluated in trials. Trials. 2020; 21(1): 982. PubMed Abstract | Publisher Full Text | Free Full Text - 11. Abadie A: Using synthetic controls: feasibility, data requirements, and methodological aspects. J Econ Lit. 2021; 59(2): 391–425. Publisher Full Text - 12. Kroenke K, Spitzer RL, Williams JB: The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001; 16(9): 606–613. PubMed Abstract | Publisher Full Text | Free Full Text - 13. Spitzer RL, Kroenke K, Williams JBW, et al.: A brief measure for assessing Generalized Anxiety Disorder: the GAD-7. Arch Intern Med. 2006; 166(10): 1092–1097. PubMed Abstract | Publisher Full Text - 14. Condon JT, Corkindale CJ: The assessment of parent-to-infant attachment: development of a self-report questionnaire instrument. J Reprod Infant Psychol. 1998; 16(1): 57–76. Publisher Full Text - 15. Ramzan F, McGrath-Lone L, Blackburn R, et al.: Education and Child Health Insights from Linked Data (ECHILD): an introductory guide for researchers. University College London, 2023. Publisher Full Text - 16. Arkhangelsky D, Athey S, Hirshberg DA, et al.: Synthetic difference-in-differences. Am Econ Rev. 2021; 111(12): 4088–4118. Publisher Full Text - 17. Athey S, Chetty R, Imbens GW, et al.: The surrogate index: combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. (NBER Working Paper No. 26463). National Bureau of Economic Research, 2019. Publisher Full Text - 18. Battocchi K, Dillon E, Hei M, et al.: Estimating the long-term effects of novel treatments. Adv Neural Inf Process Syst. 2021; 34. : 2925–2935. Reference Source Author details Author details 1 University of Oxford Department of Social Policy and Intervention, Oxford, England, UK 2 Anna Freud Centre, London, England, UK 3 Ecorys, Birmingham, UK 4 Department of Economics, University College London, London, England, UK 5 Social Research Institute, University College London, London, England, UK 6 University of Cambridge Department of Psychology, Cambridge, England, UK 2 Anna Freud Centre, London, England, UK 3 Ecorys, Birmingham, UK 4 Department of Economics, University College London, London, England, UK 5 Social Research Institute, University College London, London, England, UK 6 University of Cambridge Department of Psychology, Cambridge, England, UK Barlow J Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Sleed M Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Visualization, Writing – Review & Editing Day L Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Visualization, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Visualization, Writing – Review & Editing Davey C Roles: Data Curation, Formal Analysis, Methodology, Project Administration, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Methodology, Project Administration, Writing – Review & Editing Conti G Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Writing – Review & Editing Deighton J Roles: Formal Analysis, Investigation, Methodology Roles: Formal Analysis, Investigation, Methodology Edbrooke-Childs J Roles: Conceptualization, Funding Acquisition, Methodology Roles: Conceptualization, Funding Acquisition, Methodology Weber Costa G Roles: Data Curation, Formal Analysis, Investigation, Methodology, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Methodology, Writing – Review & Editing Reynolds S Roles: Methodology, Project Administration, Visualization Roles: Methodology, Project Administration, Visualization Cann H Roles: Data Curation, Formal Analysis, Project Administration, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Project Administration, Writing – Review & Editing Crossan L Roles: Methodology, Project Administration, Visualization Roles: Methodology, Project Administration, Visualization Fearon P Roles: Conceptualization, Funding Acquisition, Methodology Roles: Conceptualization, Funding Acquisition, Methodology Competing interests No competing interests were disclosed. Grant information This project is funded by the National Institute for Health Research (NIHR) under its [NIHR Policy Research Programme (NIHR204678)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright © Crown copyright, 2025 J B et al.. This open access work is licensed under the Open Government Licence v3.0 metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article J B, M S, L D et al. Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.14019.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 1 VERSION 1 PUBLISHED 18 Dec 2025 Views 0 How to cite this report: Kankaanpää E. Reviewer Report For: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.15246.r39178) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-39178 https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-39178 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 16 Feb 2026 Approved with Reservations VIEWS 0 Reviewer’s assessment of the protocol: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? Overall assessment The researchers aim to conduct an evaluation study to find out ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? Overall assessment The researchers aim to conduct an evaluation study to find out ... Continue reading Reviewer’s assessment of the protocol: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? Overall assessment The researchers aim to conduct an evaluation study to find out if the Start for Life project works, which parts of the intervention are successful (and what are not), to whom and what is required from the organization and employees to make it work. The objective has been expressed in evaluation research concepts, too. The study is a combination of implementation study, impact (effectiveness) assessment and economic evaluation (cost-benefit analysis). To be useful, both impact assessment and economic evaluation need a comparison group. This is an ambitious research project, especially once the project has been started already. At this moment, it is difficult to build a study design that would support robust results. The study has multiple research objectives and applies multi-level assessment and mixed methods. The variety is difficult to handle, and it is a little bit difficult to follow the protocol. My suggestion is that you structure the manuscript according to the implementation, impact and economic evaluation approaches. You might benefit from the SPIRIT (2025) STATEMENT FOR PROTOCOLS (available https://www.consort-spirit.org/). Although the recommendation has been written for RCT designs it contains a lot of useful information. For example, how to use figures to help the reader. Such as a figures of the contents of the Start for Life intervention – how families go through the process. This information is something that you also need for the comparison. What kind of services do families receive and when in other LAs (“donor pool”). Also, the recruitment of participants (LAs, families) could be presented in figures. For example, how and when LAs (intervention and comparison) are recruited? Only some of them are recruited to “deeper” investigation, when and what are the criteria in choosing these LAs. Time frame is interesting, too. The SfL has been going on since 2021 and for families the participation time is two years. When do you expect to find an impact in LAs and/or for families? How does this timeframe affect your evaluation? Rationale/Objectives and the study design The research gap is described as follows: “evidence regarding the actual benefits of population and area-level models of provision such as SfL during the early years are limited”. If I understood the program correctly it is basically integration of already existing services. Then, this study will aim to evaluate the impact of the program. The authors state that they aim to “determine which interventions are effective and in what circumstances”. This might not be possible with this intervention and study design. To be able to evaluate the impact there should be a comparison group. Also pre-post evaluation can help, how the LAs changed their service provision, how families use the services differently after the program was implemented? Synthetic control will alleviate this problem, but it will certainly be difficult to define the service integration in control LAs. Recommendation: Data will be gathered about LAs (administrative data, interviews, surveys) and families (interviews, surveys, data collected while using the services). The data description should be more detailed on all levels. The participating 75 LAs had to apply to get the funding. How are they different from other LAs? How many families in these LAs had babies born, how many used the services etc. What kind of data do you collect on families? Will you have access to data that professionals have filed? If consent is needed, you might not get all data. How to deal with this problem? Recommendation: Specifically, for the cost-benefit analysis you should give more information. How you collect the data for the synthetic LAs and families? Do you plan to collect the survey data from families in non SfL areas? The data demands for synthetic control are demanding (see Abadie 2021). Cost-benefit analysis You state that you are conducting a cost-benefit analysis (CBA). CBA compares the costs of an intervention to the monetary value of the benefits (health, wellbeing, utility). https://www.yhec.co.uk/glossary-term/cost-benefit-analysis/. Now the description of how the benefits are valued in monetary terms is missing. The CBA is worthwhile to be reported as such, not only as providing information for the return on investment analysis. How do you conduct the ROI analysis for families? (“the perspective of those providing and using the services”)? The authors have to - better describe the intervention (LAs and families) - give more details on data, especially on the synthetic control - also the methods need more details and concreteness. Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? Overall assessment The researchers aim to conduct an evaluation study to find out if the Start for Life project works, which parts of the intervention are successful (and what are not), to whom and what is required from the organization and employees to make it work. The objective has been expressed in evaluation research concepts, too. The study is a combination of implementation study, impact (effectiveness) assessment and economic evaluation (cost-benefit analysis). To be useful, both impact assessment and economic evaluation need a comparison group. This is an ambitious research project, especially once the project has been started already. At this moment, it is difficult to build a study design that would support robust results. The study has multiple research objectives and applies multi-level assessment and mixed methods. The variety is difficult to handle, and it is a little bit difficult to follow the protocol. My suggestion is that you structure the manuscript according to the implementation, impact and economic evaluation approaches. You might benefit from the SPIRIT (2025) STATEMENT FOR PROTOCOLS (available https://www.consort-spirit.org/). Although the recommendation has been written for RCT designs it contains a lot of useful information. For example, how to use figures to help the reader. Such as a figures of the contents of the Start for Life intervention – how families go through the process. This information is something that you also need for the comparison. What kind of services do families receive and when in other LAs (“donor pool”). Also, the recruitment of participants (LAs, families) could be presented in figures. For example, how and when LAs (intervention and comparison) are recruited? Only some of them are recruited to “deeper” investigation, when and what are the criteria in choosing these LAs. Time frame is interesting, too. The SfL has been going on since 2021 and for families the participation time is two years. When do you expect to find an impact in LAs and/or for families? How does this timeframe affect your evaluation? Rationale/Objectives and the study design The research gap is described as follows: “evidence regarding the actual benefits of population and area-level models of provision such as SfL during the early years are limited”. If I understood the program correctly it is basically integration of already existing services. Then, this study will aim to evaluate the impact of the program. The authors state that they aim to “determine which interventions are effective and in what circumstances”. This might not be possible with this intervention and study design. To be able to evaluate the impact there should be a comparison group. Also pre-post evaluation can help, how the LAs changed their service provision, how families use the services differently after the program was implemented? Synthetic control will alleviate this problem, but it will certainly be difficult to define the service integration in control LAs. Recommendation: - The authors should more precisely describe what changes, what is the intervention? Is it integration which might vary in different LAs? Or will something change in the contents of the services provided to families? - Once this is more clearly described you can judge if your evaluation plan can reach the objectives. Data will be gathered about LAs (administrative data, interviews, surveys) and families (interviews, surveys, data collected while using the services). The data description should be more detailed on all levels. The participating 75 LAs had to apply to get the funding. How are they different from other LAs? How many families in these LAs had babies born, how many used the services etc. What kind of data do you collect on families? Will you have access to data that professionals have filed? If consent is needed, you might not get all data. How to deal with this problem? Recommendation: - Please, show the data description in tables. Different “levels”, different sources, how many participants, what kind of data collection method (e.g. questionnaires). Specifically, for the cost-benefit analysis you should give more information. How you collect the data for the synthetic LAs and families? Do you plan to collect the survey data from families in non SfL areas? The data demands for synthetic control are demanding (see Abadie 2021). Cost-benefit analysis You state that you are conducting a cost-benefit analysis (CBA). CBA compares the costs of an intervention to the monetary value of the benefits (health, wellbeing, utility). https://www.yhec.co.uk/glossary-term/cost-benefit-analysis/. Now the description of how the benefits are valued in monetary terms is missing. The CBA is worthwhile to be reported as such, not only as providing information for the return on investment analysis. How do you conduct the ROI analysis for families? (“the perspective of those providing and using the services”)? The authors have to - better describe the intervention (LAs and families) - give more details on data, especially on the synthetic control - also the methods need more details and concreteness. - Is the rationale for, and objectives of, the study clearly described? Partly - Is the study design appropriate for the research question? Partly - Are sufficient details of the methods provided to allow replication by others? No - Are the datasets clearly presented in a useable and accessible format? No Competing Interests: No competing interests were disclosed. Reviewer Expertise: Health economics, evaluation studies CITE HOW TO CITE THIS REPORT Kankaanpää E. Reviewer Report For: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.15246.r39178) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-39178 https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-39178 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Chopra G. Reviewer Report For: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.15246.r38962) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38962 https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38962 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 22 Jan 2026 Approved with Reservations VIEWS 0 The article presents a detailed protocol for the national evaluation of England’s Start for Life programme, a large, place-based public investment aimed at improving outcomes for parents and infants during pregnancy and the first two years of life. The programme ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close The article presents a detailed protocol for the national evaluation of England’s Start for Life programme, a large, place-based public investment aimed at improving outcomes for parents and infants during pregnancy and the first two years of life. The programme is delivered through Family Hubs and integrated local early years systems across 75 local authorities, with an explicit focus on reducing inequalities and strengthening support during the critical early years. The proposed evaluation is ambitious in scope and adopts a mixed-methods, multi-level design. It combines system-level process evaluation, qualitative data-includes longitudinal and cross-sectional case studies and interviews with families, parent surveys, analysis of routine service measures, and linkage to administrative datasets. The protocol also includes an economic evaluation component, aiming to assess value for money and longer-term benefits. A key stated aim of the evaluation is to understand not only whether the programme is associated with improved outcomes, but also how and why changes occur, for whom, and in which local contexts. The protocol emphasises learning for policy and practice, strong patient and public involvement (PPI), and the generation of evidence that can inform future early childhood systems reform at scale. MAJOR Strengths 1. High policy relevance and national importance The evaluation focuses on a major national investment in early childhood and family support. Given the scale of public funding and the strategic importance of the first 1,001 days, the study addresses questions that are highly relevant for policymakers concerned with effectiveness, equity, and sustainability. 2. Strong systems-oriented perspective A notable strength is the evaluation’s focus on the wider early years system rather than isolated services. By examining governance, workforce practices, service integration, and family pathways, the protocol reflects the complexity of real-world delivery and aligns well with current thinking on early childhood systems strengthening. 3. Appropriate use of mixed methods The combination of qualitative case studies, longitudinal interviews with families, surveys, and administrative data is well suited to capturing both implementation processes and outcomes. In particular, the longitudinal qualitative component has the potential to generate valuable insights into how families experience services over time and how change occurs in practice. 4. Meaningful involvement of parents and carers The protocol demonstrates strong commitment to patient and public involvement throughout the evaluation. Involving parents and carers in study design, data collection, interpretation, and dissemination is a clear strength and enhances the relevance, ethical quality, and credibility of the findings. Major areas for improvement 1. Limited clarity on causal inference and impact estimation The protocol proposes the use of Synthetic Difference-in-Differences (SDID) alongside linked administrative data to estimate programme impacts. While this is an appropriate and increasingly used quasi-experimental approach, key methodological details are insufficiently specified. In particular, the protocol does not clearly describe how comparison local authorities will be constructed and weighted, how staggered implementation across areas will be addressed, or how variation in programme intensity and exposure will be incorporated into the analysis. In addition, there is limited discussion of how assumptions underlying SDID will be tested or how sensitive results will be to modelling choices. This lack of detail makes it difficult to assess the robustness and credibility of the proposed impact estimates. It would be helpful to provide clearer methodological detail on the construction of comparison groups, treatment timing, handling of heterogeneous exposure, and planned robustness and sensitivity analyses. The protocol should also adopt cautious language regarding causal claims, given the non-randomised nature of the programme. 2. Unclear definition of the intervention and primary estimands It is not always clear what exactly is being evaluated. The protocol appears to simultaneously assess the effects of funding allocation, local delivery models, service integration, and individual service components. Without clearly defined primary estimands, there is a risk that findings will be difficult to interpret or compare across local authorities. It would be useful to clearly specify what constitutes the “intervention” (e.g. funding eligibility, service intensity, or model type) and identify a small number of priority evaluation questions and outcomes to anchor the analysis. 3. Limited attention to long-term outcomes and trajectories Although the programme targets the first 1,001 days-a period known to influence long-term developmental, educational, and health outcomes-the evaluation focuses largely on short-term indicators. The protocol relies on proxy measures rather than outlining a clear strategy for longer-term follow-up of children and families. It would be useful to position the current evaluation as the foundation of a longer-term learning and evaluation system, with explicit plans for future data linkage or cohort follow-up where feasible. As the programme was launched relatively recently (in 2021), this is a critical moment to embed mechanisms that enable the study of long-term outcomes. Doing so would allow the evaluation to move beyond short-term effects and align more clearly with the programme’s primary objective of assessing sustained impact over time. MINOR COMMENTS 1. Outcome proliferation and prioritisation The protocol includes a broad range of outcomes across parental wellbeing, parent–infant relationships, service use, and child development. However, the absence of a clear outcome hierarchy risks diffuse interpretation. It would be useful to pre-specify primary and secondary outcomes to strengthen analytical focus and interpretability. 2. Limitations of routine service data Routine data sources are central to the evaluation but may vary in completeness, quality, and consistency across local authorities. These limitations are not fully discussed. Describe strategies for handling missing or uneven data. 3. Economic evaluation assumptions While the inclusion of cost-benefit analysis is appropriate, the protocol would benefit from clearer specification of assumptions, including perspective, time horizon, and treatment of uncertainty. Provide additional detail on how economic findings will be interpreted and communicated to policymakers. Overall assessment Overall, this protocol outlines an ambitious and policy-relevant evaluation of a complex national early childhood programme. Its strengths lie in its systems perspective, mixed-methods design, and strong stakeholder involvement. However, the evaluation would be strengthened by: The proposed evaluation is ambitious in scope and adopts a mixed-methods, multi-level design. It combines system-level process evaluation, qualitative data-includes longitudinal and cross-sectional case studies and interviews with families, parent surveys, analysis of routine service measures, and linkage to administrative datasets. The protocol also includes an economic evaluation component, aiming to assess value for money and longer-term benefits. A key stated aim of the evaluation is to understand not only whether the programme is associated with improved outcomes, but also how and why changes occur, for whom, and in which local contexts. The protocol emphasises learning for policy and practice, strong patient and public involvement (PPI), and the generation of evidence that can inform future early childhood systems reform at scale. MAJOR Strengths 1. High policy relevance and national importance The evaluation focuses on a major national investment in early childhood and family support. Given the scale of public funding and the strategic importance of the first 1,001 days, the study addresses questions that are highly relevant for policymakers concerned with effectiveness, equity, and sustainability. 2. Strong systems-oriented perspective A notable strength is the evaluation’s focus on the wider early years system rather than isolated services. By examining governance, workforce practices, service integration, and family pathways, the protocol reflects the complexity of real-world delivery and aligns well with current thinking on early childhood systems strengthening. 3. Appropriate use of mixed methods The combination of qualitative case studies, longitudinal interviews with families, surveys, and administrative data is well suited to capturing both implementation processes and outcomes. In particular, the longitudinal qualitative component has the potential to generate valuable insights into how families experience services over time and how change occurs in practice. 4. Meaningful involvement of parents and carers The protocol demonstrates strong commitment to patient and public involvement throughout the evaluation. Involving parents and carers in study design, data collection, interpretation, and dissemination is a clear strength and enhances the relevance, ethical quality, and credibility of the findings. Major areas for improvement 1. Limited clarity on causal inference and impact estimation The protocol proposes the use of Synthetic Difference-in-Differences (SDID) alongside linked administrative data to estimate programme impacts. While this is an appropriate and increasingly used quasi-experimental approach, key methodological details are insufficiently specified. In particular, the protocol does not clearly describe how comparison local authorities will be constructed and weighted, how staggered implementation across areas will be addressed, or how variation in programme intensity and exposure will be incorporated into the analysis. In addition, there is limited discussion of how assumptions underlying SDID will be tested or how sensitive results will be to modelling choices. This lack of detail makes it difficult to assess the robustness and credibility of the proposed impact estimates. It would be helpful to provide clearer methodological detail on the construction of comparison groups, treatment timing, handling of heterogeneous exposure, and planned robustness and sensitivity analyses. The protocol should also adopt cautious language regarding causal claims, given the non-randomised nature of the programme. 2. Unclear definition of the intervention and primary estimands It is not always clear what exactly is being evaluated. The protocol appears to simultaneously assess the effects of funding allocation, local delivery models, service integration, and individual service components. Without clearly defined primary estimands, there is a risk that findings will be difficult to interpret or compare across local authorities. It would be useful to clearly specify what constitutes the “intervention” (e.g. funding eligibility, service intensity, or model type) and identify a small number of priority evaluation questions and outcomes to anchor the analysis. 3. Limited attention to long-term outcomes and trajectories Although the programme targets the first 1,001 days-a period known to influence long-term developmental, educational, and health outcomes-the evaluation focuses largely on short-term indicators. The protocol relies on proxy measures rather than outlining a clear strategy for longer-term follow-up of children and families. It would be useful to position the current evaluation as the foundation of a longer-term learning and evaluation system, with explicit plans for future data linkage or cohort follow-up where feasible. As the programme was launched relatively recently (in 2021), this is a critical moment to embed mechanisms that enable the study of long-term outcomes. Doing so would allow the evaluation to move beyond short-term effects and align more clearly with the programme’s primary objective of assessing sustained impact over time. MINOR COMMENTS 1. Outcome proliferation and prioritisation The protocol includes a broad range of outcomes across parental wellbeing, parent–infant relationships, service use, and child development. However, the absence of a clear outcome hierarchy risks diffuse interpretation. It would be useful to pre-specify primary and secondary outcomes to strengthen analytical focus and interpretability. 2. Limitations of routine service data Routine data sources are central to the evaluation but may vary in completeness, quality, and consistency across local authorities. These limitations are not fully discussed. Describe strategies for handling missing or uneven data. 3. Economic evaluation assumptions While the inclusion of cost-benefit analysis is appropriate, the protocol would benefit from clearer specification of assumptions, including perspective, time horizon, and treatment of uncertainty. Provide additional detail on how economic findings will be interpreted and communicated to policymakers. Overall assessment Overall, this protocol outlines an ambitious and policy-relevant evaluation of a complex national early childhood programme. Its strengths lie in its systems perspective, mixed-methods design, and strong stakeholder involvement. However, the evaluation would be strengthened by: - Being more cautious about causal claims, - Clearly defining what is being evaluated, - Planning more explicitly for long-term learning, - Improving clarity around methods and priorities. - Is the rationale for, and objectives of, the study clearly described? Yes - Is the study design appropriate for the research question? Partly - Are sufficient details of the methods provided to allow replication by others? Partly - Are the datasets clearly presented in a useable and accessible format? Not applicable Competing Interests: No competing interests were disclosed. Reviewer Expertise: Professor in Child Development with 40 years experience in teaching, research, advising policies and programmes for children. CITE HOW TO CITE THIS REPORT Chopra G. Reviewer Report For: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.15246.r38962) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38962 https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38962 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: García JL. Reviewer Report For: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.15246.r38960) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38960 https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38960 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 07 Jan 2026 Approved with Reservations VIEWS 0 1. Summary of the article This manuscript presents a protocol for the national evaluation of England’s Start for Life (SfL) and Family Hubsprogramme, a large-scale policy initiative targeting parents and children during the first 1,000 days of life. The programme ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close This manuscript presents a protocol for the national evaluation of England’s Start for Life (SfL) and Family Hubsprogramme, a large-scale policy initiative targeting parents and children during the first 1,000 days of life. The programme ... Continue reading 1. Summary of the article This manuscript presents a protocol for the national evaluation of England’s Start for Life (SfL) and Family Hubsprogramme, a large-scale policy initiative targeting parents and children during the first 1,000 days of life. The programme provides funding to 75 local authorities (LAs) with the aim of improving early childhood development, parental wellbeing, and family support through integrated local services. The proposed evaluation adopts a mixed-methods design. It combines (i) a system-level process evaluation across all funded LAs; (ii) in-depth qualitative case studies in a subset of LAs, including a longitudinal family case series; (iii) analysis of individual-level outcomes using routine service measures and a cross-sectional survey; and (iv) an impact and economic evaluation relying primarily on Synthetic Difference-in-Differences (SDID) using administrative and population-level data. The authors aim to assess implementation, estimate impacts on short- and medium-run outcomes, examine heterogeneity in effects, and evaluate cost-effectiveness. The policy relevance of the programme is clear, and the ambition to combine process, impact, and economic evaluation is commendable. However, several core elements of the research design are insufficiently specified or fundamentally limit the scientific strength of the conclusions the evaluation seeks to draw. 2. Overall assessment The protocol addresses an important policy area and reflects substantial investment in data collection and stakeholder engagement. The descriptive and process-evaluation components are well motivated and likely to be informative for policymakers. However, two central design limitations substantially weaken the scientific contribution of the evaluation as currently proposed: 3. Evaluation of key scientific elements A. Clarity of research questions and estimands Assessment: partly The manuscript clearly states broad goals (“what works, for whom, and under what circumstances”) and identifies a wide range of outcomes of interest. However, it does not clearly define the primary causal estimands. Specifically, it remains unclear whether the evaluation seeks to estimate: Nor are primary outcomes distinguished from secondary or exploratory outcomes. How authors can address this Must be addressed for scientific soundness: Yes B. Credibility of the causal design Assessment: no (for strong causal claims) The most significant limitation of the protocol is the lack of randomization or quasi-randomized assignment at the LA level or below. While the authors propose SDID to construct counterfactual trends using unfunded LAs, this approach does not compensate for the absence of an experimental or plausibly exogenous source of variation. SfL funding was targeted toward more deprived LAs and implemented alongside other national policies. In this context, SDID estimates rely on strong assumptions regarding donor pool comparability, absence of unobserved time-varying confounders, and limited interference across LAs. As written, the design supports associational evidence with robustness checks rather than credible causal inference. How authors can address this Must be addressed for scientific soundness: Yes C. Specification of the SDID approach and inference Assessment: partly Although SDID is named as the primary estimator, the protocol does not provide sufficient detail on: Without these details, it is difficult to assess whether the proposed estimates will be interpretable or robust. How authors can address this Must be addressed for scientific soundness: Yes D. Outcome selection, multiplicity, and measurement Assessment: partly The protocol includes a wide array of outcomes drawn from administrative data, routine service measures, and surveys. However: How authors can address this Must be addressed for scientific soundness: Yes E. Long-term follow-up and scientific contribution Assessment: no A major missed opportunity is the absence of a clear strategy for long-term follow-up of children exposed to the programme. The evaluation focuses on short- and medium-run outcomes and relies on surrogate indices to infer long-run benefits. This is inconsistent with the programme’s emphasis on the first 1,000 days and with the standards of evidence in the early-childhood literature. Without deliberate cohort construction and plans for longitudinal linkage, the evaluation risks yielding results that are policy-relevant in the short run but limited in lasting scientific value. How authors can address this Must be addressed for scientific soundness: Yes F. Economic evaluation Assessment: partly The proposed cost-benefit analysis is conceptually appropriate but underspecified. Key elements such as perspective, valuation sources, uncertainty propagation, and time horizon require clarification. How authors can address this Must be addressed for scientific soundness: Partly 4. Required revisions for scientific soundness The following issues must be addressed before the protocol can support scientifically credible conclusions: 5. Conclusion This protocol outlines an ambitious and policy-relevant evaluation of a major early-life intervention. However, in its current form, the design does not support strong causal claims, nor does it fully exploit the opportunity to generate lasting scientific evidence on interventions during the first 1,000 days of life. Addressing the issues outlined above would substantially strengthen both the credibility and the long-term contribution of the study. This manuscript presents a protocol for the national evaluation of England’s Start for Life (SfL) and Family Hubsprogramme, a large-scale policy initiative targeting parents and children during the first 1,000 days of life. The programme provides funding to 75 local authorities (LAs) with the aim of improving early childhood development, parental wellbeing, and family support through integrated local services. The proposed evaluation adopts a mixed-methods design. It combines (i) a system-level process evaluation across all funded LAs; (ii) in-depth qualitative case studies in a subset of LAs, including a longitudinal family case series; (iii) analysis of individual-level outcomes using routine service measures and a cross-sectional survey; and (iv) an impact and economic evaluation relying primarily on Synthetic Difference-in-Differences (SDID) using administrative and population-level data. The authors aim to assess implementation, estimate impacts on short- and medium-run outcomes, examine heterogeneity in effects, and evaluate cost-effectiveness. The policy relevance of the programme is clear, and the ambition to combine process, impact, and economic evaluation is commendable. However, several core elements of the research design are insufficiently specified or fundamentally limit the scientific strength of the conclusions the evaluation seeks to draw. 2. Overall assessment The protocol addresses an important policy area and reflects substantial investment in data collection and stakeholder engagement. The descriptive and process-evaluation components are well motivated and likely to be informative for policymakers. However, two central design limitations substantially weaken the scientific contribution of the evaluation as currently proposed: - The absence of randomization or quasi-randomized assignment at the LA or more granular level, which severely limits causal interpretation despite the use of SDID methods. - The lack of a concrete long-term follow-up strategy, despite the programme’s explicit focus on the first 1,000 days and the well-established importance of long-run outcomes in early-childhood interventions. 3. Evaluation of key scientific elements A. Clarity of research questions and estimands Assessment: partly The manuscript clearly states broad goals (“what works, for whom, and under what circumstances”) and identifies a wide range of outcomes of interest. However, it does not clearly define the primary causal estimands. Specifically, it remains unclear whether the evaluation seeks to estimate: - the effect of SfL funding eligibility, - the effect of actual spending or service intensity, - the effect of specific service components, or - the effect of locally determined implementation models. Nor are primary outcomes distinguished from secondary or exploratory outcomes. How authors can address this - Explicitly define a small set of primary estimands, including the unit of analysis, timing, and population. - Clarify the operational definition of “treatment” used in the causal analysis. - Distinguish confirmatory outcomes from exploratory analyses. Must be addressed for scientific soundness: Yes B. Credibility of the causal design Assessment: no (for strong causal claims) The most significant limitation of the protocol is the lack of randomization or quasi-randomized assignment at the LA level or below. While the authors propose SDID to construct counterfactual trends using unfunded LAs, this approach does not compensate for the absence of an experimental or plausibly exogenous source of variation. SfL funding was targeted toward more deprived LAs and implemented alongside other national policies. In this context, SDID estimates rely on strong assumptions regarding donor pool comparability, absence of unobserved time-varying confounders, and limited interference across LAs. As written, the design supports associational evidence with robustness checks rather than credible causal inference. How authors can address this - Explicitly acknowledge the limits of causal identification under the current design. - Explore whether randomization or quasi-randomization is still feasible for future phases (e.g., phased rollouts, oversubscription lotteries, eligibility thresholds, encouragement designs). - Clarify whether within-LA variation in timing or intensity can be exploited for more credible identification. - Align the language of conclusions with what the design can support. Must be addressed for scientific soundness: Yes C. Specification of the SDID approach and inference Assessment: partly Although SDID is named as the primary estimator, the protocol does not provide sufficient detail on: - treatment timing and exposure, - donor pool restrictions and overlap diagnostics, - handling of staggered implementation and heterogeneous effects, - inference procedures (placebo tests, clustering, uncertainty quantification). Without these details, it is difficult to assess whether the proposed estimates will be interpretable or robust. How authors can address this - Clearly define treatment timing and intensity. - Pre-specify donor pool construction rules and diagnostic criteria. - Describe inference methods and sensitivity analyses. - Clarify how staggered adoption will be handled. Must be addressed for scientific soundness: Yes D. Outcome selection, multiplicity, and measurement Assessment: partly The protocol includes a wide array of outcomes drawn from administrative data, routine service measures, and surveys. However: - No hierarchy of outcomes is specified. - Multiplicity is not addressed. - Routine outcome measures are likely subject to selection and missingness that are not discussed. - The role of the cross-sectional survey in causal inference is unclear. How authors can address this - Define a limited set of primary outcomes. - Pre-specify a multiplicity adjustment strategy. - Discuss selection into routine measurement and propose methods to address it. - Clarify the estimand supported by the survey data. Must be addressed for scientific soundness: Yes E. Long-term follow-up and scientific contribution Assessment: no A major missed opportunity is the absence of a clear strategy for long-term follow-up of children exposed to the programme. The evaluation focuses on short- and medium-run outcomes and relies on surrogate indices to infer long-run benefits. This is inconsistent with the programme’s emphasis on the first 1,000 days and with the standards of evidence in the early-childhood literature. Without deliberate cohort construction and plans for longitudinal linkage, the evaluation risks yielding results that are policy-relevant in the short run but limited in lasting scientific value. How authors can address this - Articulate a plan to identify cohorts of exposed children and link them to future education, health, and economic outcomes using administrative data. - Specify governance, consent, and infrastructure steps required for long-term follow-up. - Frame the evaluation as the foundation of a long-run research platform rather than a stand-alone study. Must be addressed for scientific soundness: Yes F. Economic evaluation Assessment: partly The proposed cost-benefit analysis is conceptually appropriate but underspecified. Key elements such as perspective, valuation sources, uncertainty propagation, and time horizon require clarification. How authors can address this - Specify perspectives (e.g., public sector, societal). - Clearly map outcomes to monetized benefits. - Describe sensitivity and uncertainty analyses. - Clarify how long-run benefits will be projected. Must be addressed for scientific soundness: Partly 4. Required revisions for scientific soundness The following issues must be addressed before the protocol can support scientifically credible conclusions: - Clear definition of treatment, timing, and primary estimands. - Explicit acknowledgment of the limits imposed by the lack of randomization, and alignment of claims accordingly. - Detailed specification of the SDID design, donor pool, and inference. - Pre-specification of primary outcomes and multiplicity handling. - A concrete strategy for long-term follow-up of exposed cohorts. 5. Conclusion This protocol outlines an ambitious and policy-relevant evaluation of a major early-life intervention. However, in its current form, the design does not support strong causal claims, nor does it fully exploit the opportunity to generate lasting scientific evidence on interventions during the first 1,000 days of life. Addressing the issues outlined above would substantially strengthen both the credibility and the long-term contribution of the study. - Is the rationale for, and objectives of, the study clearly described? Yes - Is the study design appropriate for the research question? Partly - Are sufficient details of the methods provided to allow replication by others? Yes - Are the datasets clearly presented in a useable and accessible format? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: labor and public economics CITE HOW TO CITE THIS REPORT García JL. Reviewer Report For: Protocol for the Start for Life National Evaluation: What works, for whom, under what circumstances? [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:123 (https://doi.org/10.3310/nihropenres.15246.r38960) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38960 https://openresearch.nihr.ac.uk/articles/5-123/v1#referee-response-38960 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | ||| |---|---|---|---| | 1 | 2 | 3 | | | Version 1 18 Dec 25 | read | read | read | Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...