Developing methods for systems-informed Health Impact Assessment (System-HIA)

health impact assessment, complex systems, systems approaches, HIA practice ALL Metrics - Views Downloads How to cite this article Inyang E, Tweed E, Pollack R et al. Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.13621.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Research Article [version 1; peer review: 3 approved with reservations] Elizabeth Inyang https://orcid.org/0009-0009-2678-4230 1, Emily Tweed1, Roxana Pollack1, [...] Margaret Douglas https://orcid.org/0000-0003-1560-5451 2-4, Liz Green https://orcid.org/0000-0001-9218-3526 4-6, Petra Meier https://orcid.org/0000-0001-5354-1933 1, Monica O’Mullane7, Corinna Elsenbroich https://orcid.org/0000-0003-1153-4326 1Elizabeth Inyang https://orcid.org/0009-0009-2678-4230 1, Emily Tweed1, [...] Roxana Pollack1, Margaret Douglas https://orcid.org/0000-0003-1560-5451 2-4, Liz Green https://orcid.org/0000-0001-9218-3526 4-6, Petra Meier https://orcid.org/0000-0001-5354-1933 1, Monica O’Mullane7, Corinna Elsenbroich https://orcid.org/0000-0003-1153-4326 1 PUBLISHED 11 Feb 2025 Author details Author details 1 MRC/CSO SPHSU, University of Glasgow, Glasgow, Scotland, G12 8TB, UK 2 Public Health Scotland, NHS Scotland, Glasgow, Scotland, UK 3 School of Health and Wellbeing, University of Glasgow, Glasgow, UK 4 Department of International Health, Care and Public Health Research Institute – CAPHRI, Maastricht University, Maastricht, The Netherlands 5 Public Health Wales, NHS Wales, Cardiff, UK 6 University of Liverpool Department of Geography and Planning, Liverpool, England, UK 7 School of Public Health, University College Cork, Cork, Ireland 2 Public Health Scotland, NHS Scotland, Glasgow, Scotland, UK 3 School of Health and Wellbeing, University of Glasgow, Glasgow, UK 4 Department of International Health, Care and Public Health Research Institute – CAPHRI, Maastricht University, Maastricht, The Netherlands 5 Public Health Wales, NHS Wales, Cardiff, UK 6 University of Liverpool Department of Geography and Planning, Liverpool, England, UK 7 School of Public Health, University College Cork, Cork, Ireland Elizabeth Inyang Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Emily Tweed Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roxana Pollack Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Margaret Douglas Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Liz Green Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Petra Meier Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Monica O’Mullane Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Corinna Elsenbroich Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing OPEN PEER REVIEW REVIEWER STATUS Health Impact Assessment (HIA) is a process for systematically assessing the potential health and wellbeing impacts of a policy, plan, or project, including recommendations on how those impacts may be managed. Increasingly policy is seen as interventions in complex systems. Methodologically guidance for undertaking HIA do not explicitly draw on complex systems approaches. This study explores the potential contribution of systems approaches to HIA, with a view to informing practical guidance on conducting HIA. The study was registered with Research Registry (reference: researchregistry9504). Scoping review, Stakeholder Workshops. 25 eligible sources were identified applying systems approaches in IA, mostly quantitative modelling approaches, such as system dynamics, though several advocated for the application of systems thinking more broadly. HIA is inherently seen as systems oriented, given the involvement of multiple stakeholders and the focus on the wider determinants of health. Workshop participants showed a range of reactions towards applying the systems tools. Major concerns were HIA already being under resourced and systems methods potentially adding pressure and reducing stakeholder buy-in for HIA. Whilst broadening perspectives was seen as enriching, the difficulty to apply system tools directly would require adaptation to the HIA process. Participants valued the tools for their capacity to structure discussion or stakeholder engagement, and their capacity to communicate findings. There was concern about adding to the structured HIA process. This is the first study systematically assessing potential value and practical implications of applying systems informed tools as part of HIA. Although there is some conceptual overlap between HIA and systems approaches, there are practical challenges in incorporating new systems tools into HIA practice. Systems tools have a potential role in the initial setting of boundaries for a HIA, and as a lens to find opportunities to embed HIA in institutional decision-making processes. Health Impact Assessment (HIA) is a process used in public health to examine how initiatives from outside the health sector might affect health and wellbeing across different groups in the population. Public health is increasingly using methods from a discipline called ‘systems science’ to understand how multiple different factors from across society interact over time to shape our health. Systems science approaches for understanding complex situations could therefore be useful when carrying out HIA. To understand whether systems approaches might be helpful as part of HIA, we: Looked at previous research to see if systems tools have previously been used in impact assessments looking at health and other outcomes. Held eight online workshops with 27 participants who are involved in HIA, to test out ten systems tools in different HIA scenarios. Held two further workshops with our research team to apply two specific tools to examples of previously completed HIA, to see what difference using the tools might have made. We found that systems approaches had rarely been used previously in HIA but had been used in other types of impact assessment. Most participants thought that HIA shared a similar worldview to systems approaches, because both look at many different factors affecting health. The systems tools were seen as valuable for broadening perspectives, structuring discussions, and involving different people, but were difficult to apply directly to HIA. Participants were concerned about making the HIA process more complicated because of the limited time and resources available. We believe this study is the first to investigate whether and how systems approaches might be useful to HIA. We concluded that they are potentially useful, especially in the early stages of the process or for understanding the ‘bigger picture’ of how HIA can contribute to decision-making, but are likely to require further adaptation. health impact assessment, complex systems, systems approaches, HIA practice Corresponding Author(s) Elizabeth Inyang ([email protected]) Corinna Elsenbroich ([email protected]) Corinna Elsenbroich ([email protected]) Grant information: This project is funded by the National Institute for Health and Care Research (NIHR) (Grant Reference Number NIHR153474). This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Public Health Research programme or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © Crown copyright, 2025 Inyang E et al.. This open access work is licensed under the Open Government Licence v3.0 How to cite: Inyang E, Tweed E, Pollack R et al. Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.13621.1) First published: 11 Feb 2025, 5:11 (https://doi.org/10.3310/nihropenres.13621.1) Latest published: 11 Feb 2025, 5:11 (https://doi.org/10.3310/nihropenres.13621.1) This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Public Health Research programme or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Health Impact Assessment (HIA) is defined as ‘a process which systematically judges the potential, and sometimes unintended, effects of a project, program, plan, policy, or strategy on the health of the population and the distribution of those effects within the population’1. It is a core public health approach, used on a prospective basis to influence healthy public policy by generating evidence and making recommendations as to how negative impacts on health and wellbeing can be avoided or mitigated, and positive impacts enhanced. HIAs are carried out in a range of contexts, including local and national government, the private sector, the third sector, and public health agencies, and vary in scope, scale, and comprehensiveness according to circumstances (such as resources available and the issues at stake). They may be mandatory (based on requirements set out in legislation or regulations); commissioned or led by decision-makers to inform decision-making; or undertaken by organisations or communities who wish to influence the decision, as a form of advocacy. A range of guidance for undertaking HIA exists, though typically follows a defined sequence of steps as illustrated in Figure 1. Public health practice is increasingly adopting concepts and methods from systems thinking, a discipline which aims to recognise the world as a complex system – a system of multiple interacting components; dynamically changing over time and displaying feedback loops, adaptation, and emergence. Calls to adopt systems approaches to public health have proliferated in recent years2–6. Many of the proposals examined by HIA may be conceptualised as interventions in complex systems. For example, a policy such as a minimum income guarantee may impact on population health and health inequalities via a number of complex, dynamic effect pathways over the short-term (e.g., affordability of key health determinants such as high quality housing and nutritious food), medium-term (e.g., household financial security and labour market choices), and long-term (e.g., life course effects from child poverty). All of these impacts will depend on interactions between the intervention and existing systems (e.g., the economy, benefits system, and labour market). To our knowledge, no existing HIA guidance makes explicit reference to systems approaches and their application to the HIA process, and examples of individual HIA which have done so are scarce. Several researchers have identified challenges within HIA which may be amenable to systems approaches (such as improving understanding of causal pathways in order to better assess potential impacts and make more effective recommendations; of interdependencies, uncertainties, and multiple exposures; and of the complex knowledge systems in which decision making takes place)7,8. Consideration of whether and how these two methodological traditions could be combined is timely, in light of new statutory duties for HIA in Wales and Scotland, forthcoming updates to HIA guidance in a number of UK nations, and ongoing interest in systems approaches in public health. We therefore sought to assess the value and practical implications of applying systems tools to the HIA process. Aim: to explore and test the potential contribution of systems approaches to Health Impact Assessment (HIA), with a view to informing practical guidance on conducting HIA.

to undertake stakeholder workshops, case studies, and a scoping review to investigate the following questions: Our advisory group included a public representative, recruited via the NIHR People in Research website, who was able to provide guidance on study design and delivery in general as well as the recruitment and support of public representatives for study workshops. Given the technical nature of the topics discussed at workshops for this methodological research project, we recruited public participants from an existing public panel which had previously taken part in a Health Impact Assessment exercise and were therefore familiar with the methods. We recruited three public participants though due to ill-health, only two were able to participate. All public representatives were offered preparatory pre-meetings with the joint principal investigator prior to each event to and were also encouraged to make contact should questions or issues arise at any other time. Public representatives were compensated for their time (including preparation and pre-meetings) according to NIHR guidelines. With regard to our research topic, equity is a core value underpinning HIA practice, as identified in the International Association for Impact Assessment HIA Best Practice Principles1. How systems approaches might help further realise and support this value was therefore an important consideration as part of the research: for instance, the ability of existing systems tools to capture potential differential impacts of a given proposal on different population groups was a key topic for discussion during the stakeholder workshops. With regard to research conduct, we sought to ensure diversity and minimise bias in our recruitment to the workshops by issuing open invitations to existing communities of practice as well as using professional networks and ‘word-of-mouth’. The online format reduced participation burden and enabled a much greater geographical diversity and international representation than would otherwise have been the case. Our public participants were recruited either by open advertisement on the NIHR People in Research portal (advisory group member) or a public panel convened for an existing HIA (workshop participants): together they represented diverse identities and life experiences relating to economic precarity, chronic illness, minority ethnic status, and the asylum system. Our recruitment materials emphasised our commitment to meeting accessibility needs and all participants – both professional and public – were asked about any such requirements in advance of events to enable accommodations. The project had three phases: a scoping review, a set of stakeholder workshops, and two follow up co-investigator workshops. We had initially planned to conduct pilot studies during the third phase, but this turned out not to be feasible for reasons described below. We undertook a rapid scoping review of existing peer-reviewed and grey literature to identify examples of the use of systems approaches in impact assessments, including those not explicitly including health. The research question for the review was: how have systems approaches been applied in impact assessment activities to date? The review protocol – including detailed search strategies and inclusion criteria – is provided in in the OSF repository (https://doi.org/10.17605/OSF.IO/U76VW). We searched five bibliographic databases (Medline, Scopus, Social Science Citation Index, Science Citation Index, and ASSIA) on 18th April 2023, using terms relevant to impact assessment and systems approaches. No limit was applied by date or geographical setting, though full texts had to be available in English. Eligible article types included primary research, literature reviews, editorials, position papers, and commentary. To identify grey literature, we searched selected websites relevant to impact assessment using terms relevant to systems approaches and invited suggestions for specific examples from our co-investigators and study advisory group. Potentially relevant citations from eligible studies were also screened for inclusion, using a snowballing approach. To be eligible for inclusion, studies had to engage with systems approaches either by making a claim about their application; engaging with at least two of the core characteristics of complex systems from 9; or using one of a list of methods characteristics of systems approaches (as defined in the protocol). They additionally had to discuss or apply these elements as part of impact assessment or impact appraisal. Citations retrieved in searches were de-deduplicated and imported into Rayyan for screening. Screening was carried out by all reviewers (RP, ET, VW) via a two-stage process based on (1) title and abstract and (2) full text. An initial set of 30 citations was screened at each stage to pilot and refine the inclusion/exclusion criteria. Following this pilot stage, 10% of citations at each stage were screened by two reviewers to check reliability and identify any further issues requiring protocol refinement: the remainder were screened by a single reviewer. All included articles were extracted by one reviewer using a structured template in Excel and the extraction then checked by a second reviewer. Where systematic or scoping reviews were identified, the included studies were screened for potential inclusion using the above criteria. Quality assessment was not undertaken given that this was a scoping review focused on understanding the extent to which these techniques have been applied in practice. During this phase, we held a series of online workshops with stakeholders from the HIA community to consider how systems tools could be applied to the HIA process. The workshops were intended as the primary ‘test-bed’ for exploring potential applications of systems approaches and methods within HIA, supplemented by pre- and post-workshop activities undertaken by participants and project researchers. Each of the four workshops was held on two separate occasions to maximise the possibility of the same people attending all three workshops and accommodating different timezones, resulting in a total of eight completed workshops. Workshops were held online via Microsoft Teams and recordings subsequently professionally transcribed. Each of the first three workshops was 2.5 hours; the final session was 1 hour. Workshops were intended to be sequential, with participants expected to attend at least the first three, with the fourth being optional. We recruited participants for the workshops via a range of different routes and with an international scope, outlined in Table 1. Our focus was on recruiting: 1. HIA specialists, whose work primarily consists of undertaking HIA 2. Public health practitioners working in local and national settings, who may use HIA as part of their wider portfolio of activities 3. Public representatives, who may participate in and/or be affected by the outcomes of HIA 4. Third sector organisations, with expertise in sectors which may be the focus of HIA 5. Stakeholders from non-health sectors, whose proposals may be the subject of HIA 6. Specialists in impact assessment methods from other disciplines, such as environmental impact assessment or social impact assessment The final set of participants comprised 27 individuals from across these categories. Consent was obtained by email before the workshops. In advance of the workshops, project staff reviewed a range of sources to collate practical tools relevant to HIA that would serve as the basis for engagement with workshop participants. We identified the UK Government toolkit on Systems Thinking for Civil Servants as a key source for this purpose, since it was designed for use in the kinds of policy contexts where HIA may be deployed but supplemented this with additional tools identified by co-investigators and our advisory group. Briefing materials for participants on the key concepts of systems thinking and a set of ten core tools which we would explore in detail in the workshops were circulated in advance. The tools which formed the basis for engagement at the workshops are listed in Table 2. We also identified three key vignettes of real-world HIA which participants could use when testing the tools of interest: these are detailed in Table 3. The use of vignettes was to enable workshop participants to contextualise how different systems tools could be used in the different stages of HIA and to provide a proposal against which the tool could be tested. The choice of vignettes was based on capturing the diversity of policy sectors (food/education; energy; and spatial planning/local economic development); levels of governance (national; regional; local); and of intervention types (introduction of new universal policy; withdrawal of existing infrastructure; change in land use) to which HIA can be applied. Details of the vignettes were circulated in advance alongside the briefing materials for participants. | Proposal | Policy sector | Level of governance | Details | |---|---|---|---| | Universal free school meals for primary school children | Food & education | National | See a description of the full vignettes in the Systems-HIA Workshop Vignettes document at https://doi.org/10.17605/OSF.IO/U76VW | | Retirement or retrofitting of coal- fired power plant | Energy | Regional | | | Development of retail park on site of former steelworks | Spatial planning & local economic development | Local | The purpose and activities of each workshop are described in Table 2. For the first three workshops, there were typically three breakout groups in each, with each group assigned a vignette to use for the duration of that workshop. Participants were allocated to breakout rooms in advance to ensure a balance of professional backgrounds and geographical areas of practice, and to ensure that participants had the opportunity to test tools against different HIA vignettes over the course of the three workshops. We initially planned to allow participants to choose between systems tools for each exercise: however, it was time-consuming and less informative than expected so from the second iteration of the first workshop onwards, each breakout group was assigned a tool. During workshops, we used the online whiteboard tool Miro to structure group exercises. We used the online note-making tool Padlet to seek reflections from participants prior to the first workshop (on the challenges they experienced in HIA and their experience of systems thinking to date) and subsequent to each of the first three workshops (on their experience of testing the tools and their views on the application of systems approaches to HIA). After each workshop, facilitators completed a structured debriefing template to capture their reflections and experiences of the workshops and identify learning for subsequent workshops and phases of the project. Together, the transcripts; completed Miro boards; Padlet outputs; and facilitator reflections formed the basis of the collated data from workshops: we analysed these using thematic analysis in NVivo. During this phase, we had originally anticipated piloting systems tools on a prospective basis as part of real-world HIA undertaken by our practice partners. This proved not to be appropriate for both scientific and practical reasons. Firstly, the workshops had identified a number of reservations about directly applying existing systems tools to HIA practice (as detailed below under results), and had highlighted that systems tools were likely to require significant adaptation prior to use in HIA. Secondly, despite scoping a range of pilot opportunities from the beginning of the project, we did not identify any prospective HIA which would be feasible to use as pilot studies – either due to a mismatch with project timescales or other sensitivities around delivery. Instead, we undertook the following: A day-long co-investigator workshop applying specific systems tools to two completed HIA undertaken by practice partners (on the health impacts of a policy commitment to building 50,000 new affordable homes in Scotland and of Brexit in Wales10,11: During this session, the lead practitioner for each HIA presented on the policy proposal and the group as a whole selected a specific systems tool to apply to a particular stage of the HIA; we then worked through that tool in detail before reflecting on the experience and in particular, what difference this process may have made to the outcome of that stage in comparison with the completed HIA. A half-day co-investigator workshop exploring: Identification of potential partners for further research on adapting systems tools to HIA practice and their prospective piloting These workshops were transcribed and analysed thematically alongside the stakeholder workshops. An advisory group was convened for the project, with a membership spanning expertise in systems science; public health; and health impact assessment, as well as a public representative. The group met twice during the duration of the project with email updates on a bimonthly basis. The purpose of the group was to provide advice on all aspects of the study by incorporating perspectives from a range of relevant stakeholders and scientific experts, to maximise the usefulness and impact of the project for policy, practice, and research. Ethical approval for the study was obtained from the University of Glasgow College of Social Sciences Research Ethics Committee for Non-Clinical Research involving Human Participants/Data (reference number 400220331). Date of approval 19/07/2023. The study was registered with Research Registry (reference: researchregistry9504). In this section, we first describe the findings of the scoping review, followed by those from the stakeholder workshops and co-investigator workshops. The scoping review identified 25 sources for inclusion: 16 from bibliographic databases; four from grey literature searches or stakeholder feedback; and five via snowballing (Figure 2). These sources are summarised in the Systems-HIA Characteristics of studies document (Systems-HIA Characteristics of studies), with a more detailed description of findings provided in the Systems-HIA Scoping Review document (both documents are available in the OSF Systems-HIA repository https://doi.org/10.17605/OSF.IO/U76VW). Publication dates spanned 2004 to 2022, with no clear trend in frequency over time. Policy sectors of interest included environment; agriculture; housing; tourism and leisure; and transport. Twelve countries were identified in those sources with a specific geographical focus, with all inhabited continents represented at least once. The most common forms of impact assessment were environmental impact assessment or strategic environmental assessment; three studies examined the application of systems approaches to health impact assessment12. Of the included sources, 14 reported case studies of the application of systems approaches to impact assessment; nine described theoretical or conceptual considerations in integrating the two disciplines; and two were examples of guidance on impact assessment which acknowledged systems approaches. Many sources explicitly advocated for greater application of systems approaches to impact assessment12–23, including two examples of guidance documents16,17. The most common systems approach described in the sources was systems thinking in general terms (featuring in twelve sources), followed by systems dynamics modelling (featuring in nine sources), soft systems approaches (four sources), and causal loop diagrams or other mapping approaches (three sources). Relatively few sources described the use of participatory approaches. The most common motivation described for using systems approaches in impact assessment was the identification of a policy proposal or problem as part of a complex system and therefore the perceived limitations of conventional linear approaches to understanding it12–30. Several papers highlighted the commonalities between resilience thinking and systems thinking14,20: the influence of the former within environmental and sustainability assessment traditions may explain why these account for the majority of examples identified. Other motivations included: testing assumptions and/or different policy scenarios10,13,19,21,24,28; integrating potentially competing considerations within the impact assessment process14,21,27; identifying characteristics of causal relationships including factors which promote or hinder change23,24,31;21; enabling integrated consideration (and mitigation) of cumulative impacts from 13,17,18,22,31; concordance with key concepts in impact assessment including socio-ecological models of human environmental impact and resilience13,14,17,20,32; communicating13,17,27; or modelling the interactions between stakeholders25. Where participatory systems approaches were used, authors highlighted the value of these in exploring multiple perspectives, organising collective knowledge, and ensuring stakeholder ownership22,24–28,30. Among the applied case studies, the way in which systems approaches were used in impact assessment varied: some advanced novel approaches to impact assessment which integrated systems approaches25 or included systems tools at specific points in the process (e.g., 26); whereas others used systems methods as a stand-alone analytical tool (e.g., 29) or, more rarely, made claims about the use of systems approaches which were not borne out by the data presented33. Few sources explicitly considered how systems tools might be applied to the distinct stages of impact assessment. Where this could be ascertained, most applied systems approaches at the a[ppraisal stage?]12,21,22,26–28,31,34, though some sources mentioned boundary-setting or problem definition as part of scoping17,27,30. Only one source identified impact assessment practice itself as part of a complex system to which systems insights could be applied20. Challenges highlighted included: lack of suitable data for quantitative systems modelling and the robustness of resultant findings10,19,22; negative perceptions of systems thinking by IA practitioners as overly abstract or theoretical13; limited familiarity or skills in systems thinking among IA practitioners14,15,21,22; real or perceived increase in time and budget required for systems-informed assessments15,22,26; failure to connect systems thinking with other conceptual traditions relevant to impact assessment such as resilience thinking14; difficulties among practitioners and/or stakeholders in adopting a systems thinking ‘mindset’26; and potential conflict with established impact assessment methods where scoping tends to result in narrower and reductionist approaches to issues13 Workshop participants showed high levels of engagement with the practical tasks of applying tools to HIA vignettes as well as reflecting on their usefulness. In general terms, participants reacted positively to tools where they could identify immediate benefit but resisted where tools were perceived as adding to complexity to the HIA. In general, participants preferred more structured tools, such as the Perspective Diagram, Context Diagram and the 12 Question tool. Systems tools were seen as beneficial where a clear purpose could be easily discerned, e.g. building an understanding of stakeholder perspectives (eg.12 Questions tool), or as a framework for writing up findings (e.g. CATWOE). The most common reason for rejecting a tool was that it was difficult to apply. This was either because a direct HIA focus was missing from the tool, or applying it was perceived as complicating things that were already routinely done in HIA practice (e.g. considering population subgroups). HIA practitioners commonly use checklists of populations and determinants of health to prompt consideration of potential impacts, including differential impacts, on health. Whilst the tools were often focused on eliciting the same outcomes, the absence of such prompts was a seen as a challenge in the application of the tools and there was agreement that it would take time and training to gain confidence in using the tool. Departure from existing approaches to HIA – which provide structure and transparency – was also seen as potentially worsening existing challenges in engaging stakeholders in the process. On the other hand, at least one participant mentioned that increasing policymaker familiarity with systems approaches might be an asset and enhance stakeholder buy-in to the process. A recurring theme from participants was that significant adaptation would be required before any of the systems methods presented could be integrated into HIA practice. There were also some concerns voiced about how evidence could be sourced and integrated with the tools in a systematic way, given that ethical use of value is a core value of HIA1. Participating HIA practitioners worked in a range of contexts, from large public health teams which prioritised community engagement to consultancy firms where one person could be carrying out the HIA with no public engagement. Broadly, those who worked in established public health teams which used established HIA guidance and checklists were less keen to adopt the tools and more likely to perceive them as overly complicated than independent practitioners, who approached them more flexibly and found them useful for providing a structured broad perspective. Participants’ familiarity with systems approaches was also relevant, with those who had used systems tools before being much more inclined to see their benefits. Similarly, participants responded most positively to the Theory of Change tool, because they were already familiar with the theory of change flow charts from other contexts, and found it easier to see how they could potentially use the tool. The Participant Feedback document (OSF Systems-HIA repository, https://doi.org/10.17605/OSF.IO/U76VW) provides detailed feedback on each of the specific tools examined in the workshops. When participants were asked to consider at what stage of the HIA process each tool might be applied, the most common responses were scoping and recommendations. The choice of the scoping stage, often mentioned along with screening, was linked to the capacity of the tools to explore boundaries, for example CATWOE, Context Diagram, Rich Picture. The choice of recommendations tended to reflect the power of systems tools to communicate or help structure complex situations, for example Context Diagram, Theory of Change, Causal Loop Diagram. Participants frequently identified more than one stage where a tool could be useful though were also often ambivalent and unable to provide detail on how the tool could be used. This was linked to a general lack of confidence on how tools could be applied within the HIA process. Answering this question was also complicated by a lack of standardised language in HIA relating to stages and therefore a lack of shared understanding among the participants: for example, UK participants often spoke of screening and scoping in one phrase, not making strong distinctions, while others noted that they used different terms, which included identification, appraisal, assessment or evaluation. Our choice to present the systems tools in their existing form – i.e., not tailored to HIA and agnostic as to how they may be used – was motivated by a wish to avoid biasing participant feedback, but was identified by participants as particularly challenging. Facilitators reflected that participants struggled with how to apply the tool to the agreed purpose of identifying potential health impacts. A key part of this challenge was defining the system of interest: for instance, was the system the HIA process itself, the proposal, or the policy sector in which the policy was being introduced? More explicit guidance on how to identify the system of interest was therefore seen as important if these tools were to be useful in practice. At the final participant workshops, several participants reported having introduced systems tools to their own HIA practice as a result of the study: these comprised the perspective diagram, the context diagram, the critical systems heuristic tool, and participatory systems mapping. In our co-investigator workshops, we explored how applying systems tools may have made a difference to two completed HIA, with the involvement of the lead practitioner for each assessment. We selected the rich picture tool to apply to the scoping stage of the HIA of the affordable homes policy, and participatory systems mapping to apply to the appraisal stage of the HIA of Brexit in Wales. Some similar themes emerged to the participant workshops: in particular, while it was felt that many experienced HIA practitioners would naturally adopt a systems perspective, there was concern that the systems tools added additional complication to the HIA process without clear benefits. However, the co-investigators also [identified?] a number of other key lessons. First, we identified that applications of systems approaches were most likely to be beneficial when problem-led, rather than tool-led: i.e. starting with a defined problem or challenge for the HIA and applying a systems tool (as is or adapted) rather than starting with the tool and trying to identify a potential application. Second, we identified that some explicit discussion of systems approaches as part of HIA training might help less experienced practitioners to develop their understanding and practice. Finally, we identified a number of critical points during the HIA process where systems approaches were most likely to be useful: these are discussed further below under ‘Synthesis’. Combining results from all three strands of the project, we identified the following as the most fruitful future avenues for applying systems approaches to HIA practice. 1. HIA focus for systems tools: a consistent finding was the need for systems tools to be adapted to HIA if they were to be useful, and used, in practice. This was felt to be crucial given the time constraints and variable stakeholder buy-in which characterises most HIA practice. In particular, participants often returned to the need for tools to incorporate prompts which would allow participants to identify potential health consequences (including unintended ones), affected population groups, and opportunities to address these consequences. Changing the terminology used by tools to reduce jargon from systems science was also felt to be important, as was guidance on how to identify the system of interest. An alternative approach we identified was to incorporate systems prompts into HIA tools (such as checklists for identifying population groups or potential health impacts): for instance, suggestions to look for feedback loops. 2. Better understanding the system to inform boundary-setting: boundary setting at the scoping stage was identified as a consistent challenge in HIA, as well as an area where there was commonality with systems approaches, which typically place significant emphasis on clarifying the boundaries of the system of interest. The boundaries of the system were often a major focus of discussion when participants attempted to apply systems tools to specific vignettes, and scoping was among the most commonly identified stages at which tools were felt to be useful. A key application of systems tools may therefore be in supporting initial characterisation of the system in which the proposal will be implemented, at the outset of an HIA, in order to set appropriate boundaries in the scoping stage (for instance, those relating to the time frame or geographical reach of potential impacts). 3. Better understanding the system to inform appraisal of impacts: there was widespread acknowledgement across all three strands of the project that proposals subject to HIA do represent interventions in complex systems and that the processes that give rise to impacts display characteristics such as feedback, adaptation, and path-dependence. There is therefore a case for the use of systems approaches to enhance identification of potential impacts at the appraisal stage by incorporating these properties: for instance, by understanding feedback loops between different potential impacts and between the proposal and the wider system. However, based on our experience in the co-investigator workshops – through applying PSM to the appraisal stage of the scenario on Brexit in Wales – this is likely to require further adaptation of the tools as discussed above. 4. Application to the institutional and political context of HIA: a key theme across all three strands of work was that many of the challenges of HIA were institutional and political, rather than methodological. For instance, challenges identified include: gaining buy-in from stakeholders to support and participate in HIA; constraints on time, budget, and staff capacity for undertaking HIA; timing the HIA process optimally in order to influence decision-making; managing stakeholders’ responsibilities and expectations; and willingness to implement recommendations. Workshops with both participants and co-investigators identified that systems approaches could help provide a lens for analysis of the institutional and political context in which HIA takes place, in order to help enhance the uptake and impact of HIA. HIA is understood as a cross disciplinary and cross sectoral endeavour by practitioners that is highly sensitive to population groups, in particular vulnerable and underrepresented populations, and integrates a wide range of stakeholder views. In this project we started from this multidisciplinarity conception of HIA and investigated whether some more formalised systems science tools could usefully be integrated into HIA The findings of the scoping review suggest that while there is interest in, and appetite for, the application of systems approaches to impact assessment, this has to date been primarily limited to quantitative methods applied to narrowly-defined questions (particularly within environmental impact assessment) or general exhortations to apply a systems thinking perspective. There is much less of a tradition of use of participatory systems approaches across the full lifecycle of an impact assessment, particularly within health impact assessment. In the participant workshops we encountered more resistance to the integration of systems tools into HIA than anticipated. Concerns included over-complication of an existing well-established process for HIA without tangible benefit; additional demands on an already resource-constrained endeavour; challenges in integrating evidence with the systems tools tested; and inaccessible language. Some of these are recognised challenges within the wider field of systems science: for instance, systems methods commonly bring out aspects of systems which are not easily quantified through existing forms of evidence. This may explain why systems approaches have to date been more often used in environmental impact assessments where the scope is typically defined more narrowly and the questions arising are more conducive to specific quantitative methods. However, as some papers in the review identified, systems tools can still offer benefits in making non-quantifiable aspects of a system explicit and transparent. Reactions to the use of systems tools were greatly influenced by the resource constraints under which most HIA practitioners operate: this was identified as potentially the biggest hurdle for the integration of systems tools into HIA. Whilst none of the tools are particularly difficult to learn or apply, their application would require some investment to train up HIA (and potentially stakeholders) in their use. We concluded that in order to make this feasible and of value, the tools would have to be adapted considerably to the existing HIA process. Nonetheless, a number of potential strengths of applying systems tools, and potential opportunities to address established challenges of HIA practice, were identified. Among these were the potential for engaging and communicating with diverse stakeholders in conducting HIA and communicating findings. This could operate through a greater likelihood of securing buy-in from those familiar with systems tools already or through the inherently participatory nature of the tools tested, some of which (such as the critical systems heuristic tool) explicitly seek to elicit diverse stakeholder perspectives. Another was the capacity of systems tools to support boundary decisions in HIA, which are often one of the more challenging and potentially contentious aspects of the impact assessment process35–37. Such tools could therefore help enhance transparency, communication, and stakeholder buy-in for the HIA process. Importantly, the potential for systems approaches to contribute to the HIA process appears to depend heavily on the settings in which HIA is carried out, which varies greatly. We identified four areas where the application of systems approaches to HIA appears most promising and which merit further work: adapting systems tools to adopt a greater HIA focus; boundary-setting at the scoping stage; appraisal of impacts; and as a lens for understanding the institutional and political context of HIA. As an exploratory methodological study, this project raised as many questions as it answered. We propose to undertake further work on adapting and testing systems approaches to address the four applications highlighted above. We have identified a number of potential collaborative opportunities to do so: for instance, those participants who had started using systems tools in their own practice following the workshops and a local council who has approached us to contribute systems approaches to a forthcoming HIA as part of strategy development. The limitations of our study reflect its nature as a short exploratory methodological study with a limited number of participants involved. Despite our efforts to secure a diverse range of participants, our findings may not be generalisable across HIA practice. Our focus on specific tools that are used in policy research and policy making (drawing on the systems thinking toolkit for civil servants) led to a particular set of tools and a particular presentation of the tools, so our study cannot be considered comprehensive in terms of systems methods. The short timelines of the project and findings from the participant workshops regarding challenges implementing the tools also prevented us from testing the tools in live HIA as a pilot study: however, we hope to address this in future work. This is the first study systematically assessing potential value and practical implications of applying systems informed tools as part of HIA. While there is some conceptual overlap between HIA and systems approaches, and interest from practitioners, there are practical challenges in incorporating specific systems tools into HIA practice. We found that a direct integration of the systems tools we tested, as they currently stand, into the HIA process was neither desirable nor feasible, but identified four avenues for future work to support their application. The main contribution to knowledge of this study is understanding the difficulties of integrating systems approaches in existing practices. This might explain why, despite recurring calls for a complexity and systems orientation in public health, progress of integrating it into research and practice has been slow. Going forward, we will use our learning from this project as we develop further research, including the recognition of the use of accessible language when communicating systems methods and the necessity to adapt the methods to the systems properties relevant to HIA and the respective stages of HIA where they will be most useful. The participatory element of the study has helped us to establish a network of systems-curious HIA researchers and practitioners with whom to take forward the work. The University of Glasgow is committed to handling all personal information in line with the UK Data Protection Act (2018) and the General Data Protection Regulation (EU GDPR) 2016/679. Under the Data Protection legislation, the University of Glasgow is the Data Controller, and you can find out more about how we handle personal data, including how to exercise your individual rights and the contact details for our Data Protection Officer here: Ethical approval for the study was obtained from the University of Glasgow College of Social Sciences Research Ethics Committee for Non-Clinical Research involving Human Participants/Data (reference number 400220331). The study was registered with Research Registry (reference: researchregistry9504). Date of approval 19/07/2023. Written consent was obtained before the workshops from all participants. Open Science Framework: Extended data for ‘Developing methods for systems-informed health impact assessment (SYSTEM-HIA)’ https://doi.org/10.17605/OSF.IO/U76VW Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0) Additionally, we acknowledge the contribution of Valerie Wells to data curation, methodology, and resources for the scoping review. Faculty Opinions recommendedReferences - 1. Winkler MS, Viliani F, Knoblauch AM, et al.: Health Impact Assessment international best practice principles. Fargo, USA: International Association for Impact Assessment; 2021. Reference Source - 2. Rutter H, Savona N, Glonti K, et al.: The need for a complex systems model of evidence for public health. Lancet. 2017; 390(10112): 2602–4. PubMed Abstract | Publisher Full Text - 3. Diez Roux AV: Complex systems thinking and current impasses in health disparities research. Am J Public Health. 2011; 101(9): 1627–34. PubMed Abstract | Publisher Full Text | Free Full Text - 4. El-Sayed AM, Galea S: Systems science and population health. Oxford University Press; 2017. Publisher Full Text - 5. McGill E, Er V, Penney T, et al.: Evaluation of public health interventions from a complex systems perspective: a research methods review. Soc Sci Med. 2021; 272: 113697. PubMed Abstract | Publisher Full Text - 6. Trochim WM, Cabrera DA, Milstein B, et al.: Practical challenges of systems thinking and modeling in public health. Am J Public Health. 2006; 96(3): 538–46. PubMed Abstract | Publisher Full Text | Free Full Text - 7. Cole BL, Shimkhada R, Fielding JE, et al.: Methodologies for realizing the potential of Health Impact Assessment. Am J Prev Med. 2005; 28(4): 382–9. PubMed Abstract | Publisher Full Text - 8. Nieuwenhuijsen MJ, Khreis H, Verlinghieri E, et al.: Participatory quantitative Health Impact Assessment of urban and transport planning in cities: a review and research needs. Environ Int. 2017; 103: 61–72. PubMed Abstract | Publisher Full Text - 9. Boehnert J: The visual representation of complexity. CECAN, 2020. Reference Source - 10. Higgins M, Cain T, Lowther M, et al.: 50,000 affordable homes: a Health Impact Assessment. Edinburgh: Scottish Health and Inequality Impact Assessment Network and Scottish Public Health Network; 2017. Reference Source - 11. Green L, Edmonds N, Morgan L, et al.: The public health impact of brexit in wales: a Health Impact Assessment approach. Main findings. Cardiff: Public Health Wales NHS Trust; 2019. Reference Source - 12. Mesa-Frias M, Chalabi Z, Foss AM: Assessing framing assumptions in quantitative Health Impact Assessments: a housing intervention example. Environ Int. 2013; 59: 133–40. PubMed Abstract | Publisher Full Text - 13. Ehrlich A: Collective impacts: using systems thinking in project-level assessment. Impact Assess Proj A. 2022; 40(2): 129–45. Publisher Full Text - 14. Grace W, Pope J: A systems approach to sustainability assessment. In: Morrison-Saunders A, Pope J, Bond A, editors. Handbook of sustainability assessment. Research handbooks on impact assessment. Edward Elgar Publishing Ltd., 2015; 285–320. Publisher Full Text - 15. Audouin M, de Wet B: Sustainability thinking in environmental assessment. Impact Assess Proj A. 2012; 30(4): 264–74. Publisher Full Text - 16. Partidário MD: Strategic environmental assessment better practice guide - methodological guidance for strategic thinking in SEA. Lisbon: Portuguese Environment Agency; 2012. Reference Source - 17. Impact Assessment Agency of Canada: Guidance: considering the extent to which a project contributes to sustainability. Impact Assessment Agency of Canada; 2021. Reference Source - 18. Gunn J, Noble BF: Conceptual and methodological challenges to integrating SEA and cumulative effects assessment. Environ Impact Assess Rev. 2011; 31(2): 154–60. Publisher Full Text - 19. Jardine-Orr A, Anda M: Remote indigenous housing in australia - a social assessment. International Association for Impact Assessment Annual Conference. Vancouver, Canada, 2004. Reference Source - 20. Jones M: Can resilience thinking be integrated into the strategic environmental assessment process? Integr Environ Assess Manag. 2018; 14(5): 571–7. PubMed Abstract | Publisher Full Text - 21. Karami S, Karami E, Buys L, et al.: System dynamic simulation: a new method in Social Impact Assessment (SIA). Environ Impact Assess Rev. 2017; 62: 25–34. Publisher Full Text - 22. Martínez LF, Toro J, León CJ: A complex network approach to Environmental Impact Assessment. Impact Assess Proj A. 2019; 37(5): 407–20. Publisher Full Text - 23. Nieminen M, Hyytinen K: Future-oriented impact assessment: supporting strategic decision-making in complex socio-technical environments. Evaluation. 2015; 21(4): 448–61. Publisher Full Text - 24. Dianati K, Zimmermann N, Milner J, et al.: Household air pollution in Nairobi's slums: a long-term policy evaluation using participatory system dynamics. Sci Total Environ. 2019; 660: 1108–34. PubMed Abstract | Publisher Full Text | Free Full Text - 25. Ding ZK, Wang YF, Zou PXW: An agent based Environmental Impact Assessment of building Demolition Waste Management: conventional versus green management. J Clean Prod. 2016; 133: 1136–53. Publisher Full Text - 26. Gottschick M: Participatory Sustainability Impact Assessment: scientific policy advice as a social learning process. Syst Pract Action Res. 2008; 21(6): 479–95. Publisher Full Text - 27. Videira N, Antunes P, Santos R, et al.: A participatory modelling approach to support Integrated Sustainability Assessment processes. Syst Res Behav Sci. 2010; 27(4): 446–60. Publisher Full Text - 28. Zhao Y, Xu J, Xie XX, et al.: An integrated Environmental Impact Assessment of corn-based polyols compared with petroleum-based polyols production. J Clean Prod. 2014; 68: 272–8. Publisher Full Text - 29. Grinnell S: The social health and well-being impacts associated with hydraulic fracturing. Liverpool John Moores University; 2018. Reference Source - 30. Weaver P, Rotmans J: Integrated Sustainability Assessment: what is it, why do it and how? Int J Innovat Sustain Dev. 2006; 1. Reference Source - 31. Perdicoúlis A, Piper J: Network and system diagrams revisited: satisfying CEA requirements for causality analysis. Environ Impact Assess Rev. 2008; 28(7): 455–68. Publisher Full Text - 32. Bancroft D: International association for impact assessmentundated. [13/02/2024]. Reference Source - 33. Chen CH, Wu RS, Liu WL, et al.: Development of a methodology for strategic environmental assessment: application to the assessment of golf course installation policy in Taiwan. Environ Manage. 2009; 43(1): 166–88. PubMed Abstract | Publisher Full Text - 34. Chen CH, Liu WL, Liaw SL: Integrated dynamic policy management methodology and system for strategic environmental assessment of golf course installation policy in Taiwan. Environ Impact Assess Rev. 2011; 31(1): 66–76. Publisher Full Text - 35. Karstens SAM, Bots PWG, Slinger JH: Spatial boundary choice and the views of different actors. Environ Impact Assess Rev. 2007; 27(5): 386–407. Publisher Full Text - 36. Salomons GH, Hoberg G: Setting boundaries of participation in Environmental Impact Assessment. Environ Impact Assess Rev. 2014; 45: 67–75. Publisher Full Text - 37. Watterson A, Dinan W: Public health and unconventional oil and gas extraction including fracking: global lessons from a Scottish government review. Int J Environ Res Public Health. 2018; 15(4): 675. PubMed Abstract | Publisher Full Text | Free Full Text Author details Author details 1 MRC/CSO SPHSU, University of Glasgow, Glasgow, Scotland, G12 8TB, UK 2 Public Health Scotland, NHS Scotland, Glasgow, Scotland, UK 3 School of Health and Wellbeing, University of Glasgow, Glasgow, UK 4 Department of International Health, Care and Public Health Research Institute – CAPHRI, Maastricht University, Maastricht, The Netherlands 5 Public Health Wales, NHS Wales, Cardiff, UK 6 University of Liverpool Department of Geography and Planning, Liverpool, England, UK 7 School of Public Health, University College Cork, Cork, Ireland 2 Public Health Scotland, NHS Scotland, Glasgow, Scotland, UK 3 School of Health and Wellbeing, University of Glasgow, Glasgow, UK 4 Department of International Health, Care and Public Health Research Institute – CAPHRI, Maastricht University, Maastricht, The Netherlands 5 Public Health Wales, NHS Wales, Cardiff, UK 6 University of Liverpool Department of Geography and Planning, Liverpool, England, UK 7 School of Public Health, University College Cork, Cork, Ireland Elizabeth Inyang Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Emily Tweed Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roxana Pollack Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Data Curation, Formal Analysis, Investigation, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing Margaret Douglas Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Liz Green Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Petra Meier Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Monica O’Mullane Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Roles: Conceptualization, Funding Acquisition, Methodology, Writing – Review & Editing Corinna Elsenbroich Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing Competing interests Emily Tweed was a member of the NIHR Prioritisation Committee between 2022 and 2023. Grant information This project is funded by the National Institute for Health and Care Research (NIHR) (Grant Reference Number NIHR153474). This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Public Health Research programme or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Public Health Research programme or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright © Crown copyright, 2025 Inyang E et al.. This open access work is licensed under the Open Government Licence v3.0 metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article Inyang E, Tweed E, Pollack R et al. Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.13621.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 1 VERSION 1 PUBLISHED 11 Feb 2025 Views 0 How to cite this report: Dannenberg AL. Reviewer Report For: Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.14790.r39574) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39574 https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39574 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 26 Feb 2026 Approved with Reservations VIEWS 0 Overall, this is an interesting paper that examines the potential to use systems thinking related tools in the conduct of health impact assessments. The approaches to this research seem appropriate, including a literature review and detailed workshops. ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close The approaches to this research seem appropriate, including a literature review and detailed workshops. ... Continue reading Overall, this is an interesting paper that examines the potential to use systems thinking related tools in the conduct of health impact assessments. The approaches to this research seem appropriate, including a literature review and detailed workshops. The results lead to conclusions suggesting the approach might be useful but appears to have more barriers than benefits, so system tools are not now ready for use in conducting HIAs. Figure 1 – One concern relates to how the stages of HIA are listed: A. Scoping and “setting up a team” are typically described as one step B. The stages of HIA are missing a Reporting step that is commonly used in HIA in the U.S. and elsewhere – see National Research Council HIA report https://www.ncbi.nlm.nih.gov/books/NBK83546/. Without a reporting step being explicit, it is unclear how recommendations reach the audiences, such as decision makers and stakeholders, who could use those recommendations.

- Clarify what is meant by a public representative. Are these typically persons who have some public health experience or background? The text says they had participated in a prior HIA exercise.

– Describe which "selected websites" were searched to identify appropriate gray literature, rather than only listing them in the supplemental materials. It also mentions using a snowballing technique which seems appropriate. Table 1 provides more detail than necessary about how workshop participants were recruited. This information could be given more succinctly in a paragraph with the details moved to supplemental materials. Which of these recruitment methods were most successful in the final set of participants? Some participants may have been identified through several routes.

- What are the characteristics of the participants who were recruited – how many were from the UK and what other countries were represented, what professions did the final list of participants represent, and what level of experience in HIA did they have? Table 2 lists the system tools that were tested, but these tools are not explained in the body of the paper. While there is more information about each of these tools in the supplemental materials, they are central to the paper and more information about each tool should appear in the main paper, perhaps in a table. What are the characteristics of the systems tools, because some characteristics may make them more likely to be usable or not usable for HIA. For example, how much input or data are needed to use the various tools? How well did the workshop participants understand the tools? Which of the tools are designed for practical use and which ones may be more a diagram or a theory that is difficult to use in practice? Table 2 currently provides details about the steps in the workshops that could be summarized in a paragraph, with further details described in the supplemental materials. Table 3 - The vignettes used in the workshops seem quite appropriate. The descriptions of the vignettes would be more helpful if moved to the main text from the supplemental files. Post-workshop engagement: The fact that it was difficult to conduct a pilot may be an indicator that the system tools have limited usefulness in conducting HIAs.

- Scoping Review: From the 14 case studies in which systems tools were used in impact assessment, consider describing in more detail one or two of those case studies that seem like the best case scenario for such tools. The paper mentions that systems tools would need to be adapted to be more useful for HIA. Clarify what types of adaptation may be needed. The article overall seems to suggest that cost and complexity makes system tools less likely to be useful for HIA and the benefits of such tools remain unclear. The paper suggests some areas for further research, but given the barriers found and lack of clear benefits identified, it is unclear if this would be a productive area for further HIA research. The approaches to this research seem appropriate, including a literature review and detailed workshops. The results lead to conclusions suggesting the approach might be useful but appears to have more barriers than benefits, so system tools are not now ready for use in conducting HIAs. Figure 1 – One concern relates to how the stages of HIA are listed: A. Scoping and “setting up a team” are typically described as one step B. The stages of HIA are missing a Reporting step that is commonly used in HIA in the U.S. and elsewhere – see National Research Council HIA report https://www.ncbi.nlm.nih.gov/books/NBK83546/. Without a reporting step being explicit, it is unclear how recommendations reach the audiences, such as decision makers and stakeholders, who could use those recommendations.

- Clarify what is meant by a public representative. Are these typically persons who have some public health experience or background? The text says they had participated in a prior HIA exercise.

– Describe which "selected websites" were searched to identify appropriate gray literature, rather than only listing them in the supplemental materials. It also mentions using a snowballing technique which seems appropriate. Table 1 provides more detail than necessary about how workshop participants were recruited. This information could be given more succinctly in a paragraph with the details moved to supplemental materials. Which of these recruitment methods were most successful in the final set of participants? Some participants may have been identified through several routes.

- What are the characteristics of the participants who were recruited – how many were from the UK and what other countries were represented, what professions did the final list of participants represent, and what level of experience in HIA did they have? Table 2 lists the system tools that were tested, but these tools are not explained in the body of the paper. While there is more information about each of these tools in the supplemental materials, they are central to the paper and more information about each tool should appear in the main paper, perhaps in a table. What are the characteristics of the systems tools, because some characteristics may make them more likely to be usable or not usable for HIA. For example, how much input or data are needed to use the various tools? How well did the workshop participants understand the tools? Which of the tools are designed for practical use and which ones may be more a diagram or a theory that is difficult to use in practice? Table 2 currently provides details about the steps in the workshops that could be summarized in a paragraph, with further details described in the supplemental materials. Table 3 - The vignettes used in the workshops seem quite appropriate. The descriptions of the vignettes would be more helpful if moved to the main text from the supplemental files. Post-workshop engagement: The fact that it was difficult to conduct a pilot may be an indicator that the system tools have limited usefulness in conducting HIAs.

- Scoping Review: From the 14 case studies in which systems tools were used in impact assessment, consider describing in more detail one or two of those case studies that seem like the best case scenario for such tools. The paper mentions that systems tools would need to be adapted to be more useful for HIA. Clarify what types of adaptation may be needed. The article overall seems to suggest that cost and complexity makes system tools less likely to be useful for HIA and the benefits of such tools remain unclear. The paper suggests some areas for further research, but given the barriers found and lack of clear benefits identified, it is unclear if this would be a productive area for further HIA research. - Is the work clearly and accurately presented and does it cite the current literature? Yes - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? Partly - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: health impact assessment, health and built environment, public health, environmental health, urban planning, land use and transportation, climate change CITE HOW TO CITE THIS REPORT Dannenberg AL. Reviewer Report For: Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.14790.r39574) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39574 https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39574 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Rojas-Rueda D. Reviewer Report For: Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.14790.r39444) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39444 https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39444 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 19 Feb 2026 Approved with Reservations VIEWS 0 General Comments Thanks for the opportunity to review this manuscript, which describes a mixed-methods study combining a scoping review with stakeholder and co-investigator workshops to explore whether and how systems approaches could be integrated into Health Impact Assessment ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close Thanks for the opportunity to review this manuscript, which describes a mixed-methods study combining a scoping review with stakeholder and co-investigator workshops to explore whether and how systems approaches could be integrated into Health Impact Assessment ... Continue reading General Comments Thanks for the opportunity to review this manuscript, which describes a mixed-methods study combining a scoping review with stakeholder and co-investigator workshops to explore whether and how systems approaches could be integrated into Health Impact Assessment (HIA) practice. The paper addresses a genuinely under-explored question and makes a useful contribution by surfacing the practical barriers to integrating systems tools into an already resource-constrained practice, which is arguably more informative than a study that simply advocated for integration without testing feasibility. The participatory design, inclusion of public representatives, and transparency about the failure of the original pilot plan are commendable. However, the manuscript has several methodological reporting gaps, particularly around the qualitative analysis, that limit the ability to assess the rigor of the findings. The health equity dimension, though acknowledged as central to HIA, receives surprisingly thin treatment in the analysis itself. The paper would benefit from clearer reporting against established guidelines (PRISMA-ScR for the scoping review; COREQ or SRQR for the qualitative workshops) and more explicit reflexive engagement with the researchers' own positionality relative to systems science. Major Comments This paper addresses a timely and original question about whether systems approaches can be productively integrated into HIA practice, and its finding that direct integration is neither straightforward nor always welcome is an honest and useful contribution to the field. The most pressing issues to address are the absence of reporting guideline compliance (PRISMA-ScR, COREQ), the insufficiently described qualitative analysis methods, and the lack of a reflexivity statement. Strengthening these elements, along with a more intentional equity analysis and consistent terminology, would bring the paper to a standard where it could be approved without reservations. I look forward to reviewing a revised version. Thanks for the opportunity to review this manuscript, which describes a mixed-methods study combining a scoping review with stakeholder and co-investigator workshops to explore whether and how systems approaches could be integrated into Health Impact Assessment (HIA) practice. The paper addresses a genuinely under-explored question and makes a useful contribution by surfacing the practical barriers to integrating systems tools into an already resource-constrained practice, which is arguably more informative than a study that simply advocated for integration without testing feasibility. The participatory design, inclusion of public representatives, and transparency about the failure of the original pilot plan are commendable. However, the manuscript has several methodological reporting gaps, particularly around the qualitative analysis, that limit the ability to assess the rigor of the findings. The health equity dimension, though acknowledged as central to HIA, receives surprisingly thin treatment in the analysis itself. The paper would benefit from clearer reporting against established guidelines (PRISMA-ScR for the scoping review; COREQ or SRQR for the qualitative workshops) and more explicit reflexive engagement with the researchers' own positionality relative to systems science. Major Comments - Absence of reporting guideline compliance. The scoping review component should be reported against PRISMA-ScR (Tricco et al., 2018, Annals of Internal Medicine), and the qualitative workshop component should follow COREQ (Tong et al., 2007, International Journal for Quality in Health Care) or SRQR (O'Brien et al., 2014, Academic Medicine). Neither guideline is mentioned. While NIHR Open Research encourages adherence to appropriate reporting standards, the absence of any named guideline makes it difficult to assess completeness. For example, PRISMA-ScR requires explicit reporting of the rationale for not conducting quality assessment and a completed checklist as supplementary material. COREQ requires information on researcher credentials, relationship with participants, and coding trees. The authors should state which guidelines were followed and provide completed checklists, or explain their omission. - Insufficient description of qualitative analysis methods. The paper states that workshop transcripts, Miro boards, Padlet outputs, and facilitator reflections were analyzed using "thematic analysis in NVivo," but provides no further detail. This is insufficient for replication or quality assessment. Key missing information includes: whether the analysis was inductive, deductive, or hybrid; who developed the initial coding framework and how many researchers coded; whether any inter-coder reliability or reflexive checking was undertaken; how themes were defined, reviewed, and refined; and which tradition of thematic analysis was followed (e.g., Braun and Clarke reflexive thematic analysis versus codebook thematic analysis). The brief mention of NVivo as a software tool does not substitute for methodological transparency. The authors should describe their analytic approach in sufficient detail that another team could replicate the process. - Lack of reflexivity statement. Several members of the research team have backgrounds in systems science, and the study was designed with the premise that systems approaches might be valuable for HIA. This prior orientation could shape how workshop discussions were facilitated, how tools were presented, and how qualitative data were interpreted. The paper includes no reflexivity statement addressing the researchers' positionality. This is particularly relevant given that the findings include both enthusiasm and resistance from participants, and the synthesis leans toward identifying "promising avenues" rather than concluding that integration is not worthwhile. A brief statement acknowledging how the team's expertise and assumptions were managed during facilitation and analysis would strengthen credibility. - Data availability statement is inconsistent with the study. The paper states "no data are associated with this article," yet the study generated substantial qualitative data including workshop transcripts, Miro boards, Padlet entries, and facilitator reflections. While participant consent may limit sharing of raw transcripts, the current statement is misleading. The authors should clarify what data exist, explain why they cannot be shared (e.g., due to consent agreements or identifiability), and consider whether anonymized coding frameworks, theme tables, or summary data could be deposited. NIHR Open Research's own data policies encourage maximum transparency, and this statement as written does not meet that expectation. - Scoping review search strategy not reproducible from the article. The search terms and full strategy are not presented in the manuscript itself, and as Reviewer 1 noted, the referenced "Appendix 1" within the OSF documentation could not be located. Even if the full strategy is available in a supplementary repository, the article should include at least a summary of key search terms and Boolean logic for each database to allow readers to assess the scope and sensitivity of the search without navigating external links. The authors should also confirm that the OSF materials are complete and the appendix is accessible. - Underdeveloped health equity analysis. The paper acknowledges that equity is a core value of HIA and that the ability of systems tools to capture differential impacts was discussed in workshops. However, the analysis does not systematically examine how the tested systems tools performed in identifying or addressing health inequities, nor whether certain tools were better at surfacing equity considerations than others. Given that HIA's equity commitment is what distinguishes it from other forms of impact assessment, a more intentional analysis of this dimension would substantially strengthen the paper. Even a brief subsection in the results reporting which tools participants felt were most or least useful for equity analysis would be informative. - Workshop participant characteristics insufficiently reported. The paper states that 27 participants were recruited across six categories but provides no demographic information beyond professional categories and general recruitment routes. For a qualitative study, it is standard practice to report at minimum the professional roles, geographical locations, years of experience in HIA, and prior familiarity with systems approaches of participants. The paper mentions that participant familiarity with systems tools influenced their reactions, but without knowing the distribution of experience levels across the sample, readers cannot assess how much weight to give this finding. A participant characteristics table would address this. - Terminology inconsistency around "systems." As Reviewer 1 noted, the paper uses "systems thinking," "systems approaches," "systems methods," "systems tools," "systems concepts," "systems perspectives," and "systems techniques" without clearly distinguishing between them. This matters because the paper's own findings suggest that a systems "mindset" or "perspective" may be more readily adopted by HIA practitioners than specific systems "tools" or "methods." A brief paragraph in the introduction or methods defining how these terms are used in the paper, or adopting a consistent typology, would sharpen the argument. For instance, the distinction between systems thinking as a conceptual orientation versus systems tools as specific analytic instruments is central to the paper's conclusions but never explicitly defined. - Limited consideration of global and low-resource HIA contexts. The paper is largely situated within UK HIA practice, where established guidance, public health infrastructure, and practitioner networks exist. The discussion does not address whether findings are transferable to settings where HIA may be less formalized or differently resourced, such as low- and middle-income countries where HIA is increasingly applied in extractive industries, infrastructure, and development projects. Given the international scope of recruitment (participants were drawn from international networks including IAIA and SOPHIA), a brief discussion of how the systems-HIA integration question might differ across contexts would be valuable and responsive to the journal's NIHR-funded readership, which includes global health researchers. - The framing of "adapting systems tools to HIA" as a finding. The first of the four "promising avenues" identified in the synthesis is that systems tools need to be adapted to have an HIA focus. As Reviewer 1 observed, this reads more as a prerequisite for the other three findings than as a parallel finding at the same level. Presenting it as a co-equal area for future work alongside boundary-setting, appraisal, and institutional analysis creates a logical asymmetry. The authors should consider reframing this as a cross-cutting requirement that enables the other three applications, rather than as a standalone avenue. - Use of "vulnerable and underrepresented populations." In the Discussion section, the paper refers to "vulnerable and underrepresented populations." Current equity-centered terminology guidance recommends language that locates vulnerability in structural conditions rather than in populations themselves. Alternatives such as "populations made vulnerable by structural conditions" or "populations historically excluded from decision-making processes" more accurately reflect the social determinants framework that underpins HIA. This adjustment is consistent with the health equity values the paper itself identifies as central to HIA practice. - Figure 1 typographical error. In the "Scoping" row, the text reads "What are the the boundaries" with a duplicated article. This was flagged by Reviewer 1 and should be corrected. - Missing word on page 10. The sentence referring to the "ethical use of value" appears to be a typographical error for "ethical use of evidence," as also noted by Reviewer 1. - Incomplete sentence on page 10. The phrase "enabling integrated consideration (and mitigation) of cumulative impacts from 13, 17, 18, 22, 31" appears to have a missing word between "from" and the reference numbers. - Inconsistent reference to workshop numbering. The methods state that participants were expected to attend "at least the first three" workshops, but earlier text describes "four workshops held on two separate occasions" for a total of eight. On page 6, the text says "attending all three workshops" when four are described. The numbering should be made consistent throughout. - Plain Language Summary could be more concise. The Plain Language Summary is well-written but is somewhat long relative to the abstract. Given that this audience may benefit from brevity, consider tightening the summary to focus on what was found and what it means, rather than reiterating the three-phase design in detail. - Consider adding the One Health connection to the discussion. The relationship between HIA and One Health frameworks is an active area of development (e.g., Zinsstag and Winkler, 2024, discussed at the University of Basel), and systems approaches are central to One Health methodology. A brief acknowledgment that the systems-HIA integration question has parallels in the One Health adaptation of HIA could position the paper within this broader methodological conversation, as Reviewer 1 also suggested. This paper addresses a timely and original question about whether systems approaches can be productively integrated into HIA practice, and its finding that direct integration is neither straightforward nor always welcome is an honest and useful contribution to the field. The most pressing issues to address are the absence of reporting guideline compliance (PRISMA-ScR, COREQ), the insufficiently described qualitative analysis methods, and the lack of a reflexivity statement. Strengthening these elements, along with a more intentional equity analysis and consistent terminology, would bring the paper to a standard where it could be approved without reservations. I look forward to reviewing a revised version. - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? No - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? No - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: Environmental epidemiology, health impact assessment, environmental health, population risk assessment, health equity, environmental justice, planetary health, policy translation, community-based participatory research, complex systems in public health CITE HOW TO CITE THIS REPORT Rojas-Rueda D. Reviewer Report For: Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.14790.r39444) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39444 https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-39444 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Views 0 How to cite this report: Dardier G. Reviewer Report For: Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.14790.r34773) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-34773 https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-34773 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 14 May 2025 Approved with Reservations VIEWS 0 This paper presents an interesting work on an otherwise unexplored topic. Based on the results of a scoping review and stakeholders workshops, it explores the potential value and discusses the practical implications of mobilizing systems approaches and tools as part ... Continue reading 2. Lilly K, Kean B, Hallett J, Robinson S, et al.: Factors of the policy process influencing Health in All Policies in local government: A scoping review.Front Public Health. 2023; 11: 1010335 PubMed Abstract | Publisher Full Text 3. Jabot F, Romagon J, Dardier G: A New Framework for Monitoring and Evaluating Health Impact Assessment: Capitalising on a French Case Study with the Literature in Evaluation.Int J Environ Res Public Health. 2024; 21 (9). PubMed Abstract | Publisher Full Text 4. Craig P, Dieppe P, Macintyre S, Michie S, et al.: Developing and evaluating complex interventions: the new Medical Research Council guidance.BMJ. 2008; 337: a1655 PubMed Abstract | Publisher Full Text I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close This paper presents an interesting work on an otherwise unexplored topic. Based on the results of a scoping review and stakeholders workshops, it explores the potential value and discusses the practical implications of mobilizing systems approaches and tools as part of Health Impact Assessment (HIA). The contribution to knowledge of this study lies in elucidating the difficulties associated with integrating systems approaches into existing HIA practices. Additionally, it identifies three key areas where the application of systems approaches and tools to HIA shows significant promise and warrants further exploration: boundary-setting during the scoping stage, impact appraisal, and using systems thinking as a lens to comprehend the institutional and political context of HIA. Here are my section-by-section comments.

Could you clarify the boundaries of what you call – or not – “HIA” along your paper? In the first paragraph you present quite a broad definition of HIA, but you refer to the IAIA International Best Practice Principles (2021), that, in line with the Gothenburg Consensus, defines HIA as a multi-stages participatory process that involves numerous stakeholders, defends several values (equity, sustainability, comprehensive approach to health, etc.) and aims at supporting decision-making by generating evidence and making recommendations, not just a modelling exercise to quantify the impacts on health of a specific pollutant. However, in your scoping review, you use terms relevant to IA in general, hence opening your scoping review results to EIA, SEA, or quantitative HIA, but then you seem to refocus more on “Gothenburg-HIA” in your workshops and Discussion section. Could you please clarify and justify that changes in scope along the paper as it might be a little bit confusing (I’m not questioning that opening the reflexion to QHIA and EIA is otherwise enriching the paper). Figure 1. Please consider merging “scoping” and “setting up team” into just one step and adding an “evaluation” step at the end of the process. Could you harmonize the terms you associate with “systems…” along your paper? You do define the discipline of “systems thinking” very early in your work, but then you use systems approaches, methods, tools, concepts, perspectives, techniques, ... without always specifying what you put behind those words.

If technically and “legally” possible, please consider moving the PPI and EDI sections at the end of the “Methods” chapter, right before the “Ethics and study registration” section. As it stands, it hurts a little bit the flow of the reading, and it makes more sense to come once you’ve described your stakeholder workshops and advisory groups. From transparency and reproducibility reasons, it would be interesting to have, within the article, the list of the terms you used on the five bibliographic databases for you scoping review. Right now, when you access the OSF repository, you can find a document called “Systems-HIA Scoping Review Documentation” which stands that the terms can be found on “appendix 1”, but I was not able to find this appendix 1. Could you please specify how many individuals took part in the optional workshop n°4? Regarding the 10 core systems tools you tested during the stakeholder workshops, did they all come from the UK Government toolkit on Systems Thinking for Civil Servants? If no, please specify additional references. Could you please elaborate on how the themes you used for your Nvivo thematic analysis of the workshops material emerged?

To provide a comprehensive understanding of the scope and constraints of your scoping review, please consider inserting, at the end of the “scoping review” section, a paragraph regarding its limitations: this could for example address the fact that the review was conducted solely in English and was based on a single search instance, and question if these choices might have influenced or limited your results. Additionally, it is unclear whether the same exclusion criteria were applied during the initial title and abstract (T&A) screening as during the full-text screening. This ambiguity is particularly evident in Figure 2, which should be clarified to ensure transparency in the screening process. Including a vignette or a paragraph with an illustration to present one or two of the ten systems thinking tools you tested during workshops would be ideal. This could be enhanced with a couple of pictures from the workshops, providing a visual representation of the tools in action and making the section a little bit more engaging and informative.

Page 12, left column, last paragraph: could you add one or a couple of scientific references that support your argument that “some of these are recognised challenges within the wider field of systems science”? Page 12, right column, 4th paragraph: to me, “adapting systems tools to adopt a greater HIA focus” is not an “area where the application of systems approaches to HIA appears most promising”, it is a prerequisite (it is presented as such in the following paragraph). Or at least, it is not as the same level as the other three areas of application that you mention here. Two good points mentioned in your Results chapter, that you don’t pick up on in your Discussion chapter as it stands, but could add, in my opinion, depth to your work: 1) Page 10, right column, last paragraph, you mention that “those who worked in established public health teams […] were less keen to adopt tools and more likely to perceive them as overly complicated that independent practitioners” ==> What does it tell you about path dependency and the way policy innovation emerges and disseminates, and about the state of the HIA as a competitive, consultancy market? 2) In your Results section, you make a very compelling point about the interest to apply systems thinking approaches and tools to the institutional and political context of HIA, among three other potential areas of application. In the Discussion chapter you briefly mention that you “propose to undertake further work on adapting and testing systems approaches to address” those four applications ==> Could you please elaborate a little bit more on that, from a research and methodological guidance perspective? For example, on the institutional and political context of HIA, to what extent can systems thinking approaches and tools be a way to better apprehend the wider political and economic factors that influence HIA implementation and success, including commercial determinants of health? In other words, can it help advance our knowledge of the effects of corporate influence, power differentials and entrenched political dynamics on HIA implementation and efficiency, especially when applied to controversial projects (such as big-scale transport infrastructures or power plants)? See for example: Douglas MJ, et al., 2024 (Ref 1) Lilly K, et al., 2023 (Ref 2) Finally, here are some suggestions to further connect your discussion with the current HIA research agenda and some issues regarding its practices: 1) I think it would be interesting for you to expand your step-by-step approach by questioning how systems thinking approaches and tools might support HIA evaluation. Theory-Based evaluation already mobilizes theory of change and causal pathways with interactions between effects and feedback loops, with several examples of evaluation where HIA is considered as a complex intervention. See for example: Jabot F, et al., 2024 (Ref 3) Craig P, et al., 2008 (Ref 4) 2) Maybe you could question the potential for systems thinking approaches and tools to contribute to the adaptation of the HIA methodology to One Health (Winkler et al., 2025)? If you think relevant, please position your work in the current discussion regarding this issue. See for example: Zinsstag, J., and Winkler, M. (2024). Towards One Health Impact Assessment. One Health: Connecting Humans, Animals and the Environment. University of Basel. https://www.futurelearn.com/info/courses/one-health/0/steps/25513 Typos Page 4, Figure 1, in “2. Scoping”: remove the second “the” before “boundaries” Page 5, left column, 2nd paragraph: remove “to” between “prior to each event” and “and were also encouraged to make…” Page 6, left column, 2nd paragraph, third line: I think you meant “four workshops” when you said “to maximise the possibility of the same people attending all three workshops” Page 10, left column, 2nd paragraph: there seems to be a problem with the sentence “enabling integrated consideration (and mitigation) of cumulative impacts from 13, 17, 18, 22, 31” Page 10, left column, 4th paragraph, 4th line: there is a problem with the way “appraisal stage” appears in the article Page 10, right column, 4th paragraph, last line: I think you meant “ethical use of evidence” when you said “given that ethical use of value is a core value of HIA” Page 11, right column, 2nd paragraph, 1st line: there is a problem with the way “identified” appears in the article Page 12, right column, 5th paragraph, 8th line: please consider changing “to contribute systems approaches to a forthcoming HIA” by “to apply systems approaches to a forthcoming HIA” Here are my section-by-section comments.

Could you clarify the boundaries of what you call – or not – “HIA” along your paper? In the first paragraph you present quite a broad definition of HIA, but you refer to the IAIA International Best Practice Principles (2021), that, in line with the Gothenburg Consensus, defines HIA as a multi-stages participatory process that involves numerous stakeholders, defends several values (equity, sustainability, comprehensive approach to health, etc.) and aims at supporting decision-making by generating evidence and making recommendations, not just a modelling exercise to quantify the impacts on health of a specific pollutant. However, in your scoping review, you use terms relevant to IA in general, hence opening your scoping review results to EIA, SEA, or quantitative HIA, but then you seem to refocus more on “Gothenburg-HIA” in your workshops and Discussion section. Could you please clarify and justify that changes in scope along the paper as it might be a little bit confusing (I’m not questioning that opening the reflexion to QHIA and EIA is otherwise enriching the paper). Figure 1. Please consider merging “scoping” and “setting up team” into just one step and adding an “evaluation” step at the end of the process. Could you harmonize the terms you associate with “systems…” along your paper? You do define the discipline of “systems thinking” very early in your work, but then you use systems approaches, methods, tools, concepts, perspectives, techniques, ... without always specifying what you put behind those words.

If technically and “legally” possible, please consider moving the PPI and EDI sections at the end of the “Methods” chapter, right before the “Ethics and study registration” section. As it stands, it hurts a little bit the flow of the reading, and it makes more sense to come once you’ve described your stakeholder workshops and advisory groups. From transparency and reproducibility reasons, it would be interesting to have, within the article, the list of the terms you used on the five bibliographic databases for you scoping review. Right now, when you access the OSF repository, you can find a document called “Systems-HIA Scoping Review Documentation” which stands that the terms can be found on “appendix 1”, but I was not able to find this appendix 1. Could you please specify how many individuals took part in the optional workshop n°4? Regarding the 10 core systems tools you tested during the stakeholder workshops, did they all come from the UK Government toolkit on Systems Thinking for Civil Servants? If no, please specify additional references. Could you please elaborate on how the themes you used for your Nvivo thematic analysis of the workshops material emerged?

To provide a comprehensive understanding of the scope and constraints of your scoping review, please consider inserting, at the end of the “scoping review” section, a paragraph regarding its limitations: this could for example address the fact that the review was conducted solely in English and was based on a single search instance, and question if these choices might have influenced or limited your results. Additionally, it is unclear whether the same exclusion criteria were applied during the initial title and abstract (T&A) screening as during the full-text screening. This ambiguity is particularly evident in Figure 2, which should be clarified to ensure transparency in the screening process. Including a vignette or a paragraph with an illustration to present one or two of the ten systems thinking tools you tested during workshops would be ideal. This could be enhanced with a couple of pictures from the workshops, providing a visual representation of the tools in action and making the section a little bit more engaging and informative.

Page 12, left column, last paragraph: could you add one or a couple of scientific references that support your argument that “some of these are recognised challenges within the wider field of systems science”? Page 12, right column, 4th paragraph: to me, “adapting systems tools to adopt a greater HIA focus” is not an “area where the application of systems approaches to HIA appears most promising”, it is a prerequisite (it is presented as such in the following paragraph). Or at least, it is not as the same level as the other three areas of application that you mention here. Two good points mentioned in your Results chapter, that you don’t pick up on in your Discussion chapter as it stands, but could add, in my opinion, depth to your work: 1) Page 10, right column, last paragraph, you mention that “those who worked in established public health teams […] were less keen to adopt tools and more likely to perceive them as overly complicated that independent practitioners” ==> What does it tell you about path dependency and the way policy innovation emerges and disseminates, and about the state of the HIA as a competitive, consultancy market? 2) In your Results section, you make a very compelling point about the interest to apply systems thinking approaches and tools to the institutional and political context of HIA, among three other potential areas of application. In the Discussion chapter you briefly mention that you “propose to undertake further work on adapting and testing systems approaches to address” those four applications ==> Could you please elaborate a little bit more on that, from a research and methodological guidance perspective? For example, on the institutional and political context of HIA, to what extent can systems thinking approaches and tools be a way to better apprehend the wider political and economic factors that influence HIA implementation and success, including commercial determinants of health? In other words, can it help advance our knowledge of the effects of corporate influence, power differentials and entrenched political dynamics on HIA implementation and efficiency, especially when applied to controversial projects (such as big-scale transport infrastructures or power plants)? See for example: Douglas MJ, et al., 2024 (Ref 1) Lilly K, et al., 2023 (Ref 2) Finally, here are some suggestions to further connect your discussion with the current HIA research agenda and some issues regarding its practices: 1) I think it would be interesting for you to expand your step-by-step approach by questioning how systems thinking approaches and tools might support HIA evaluation. Theory-Based evaluation already mobilizes theory of change and causal pathways with interactions between effects and feedback loops, with several examples of evaluation where HIA is considered as a complex intervention. See for example: Jabot F, et al., 2024 (Ref 3) Craig P, et al., 2008 (Ref 4) 2) Maybe you could question the potential for systems thinking approaches and tools to contribute to the adaptation of the HIA methodology to One Health (Winkler et al., 2025)? If you think relevant, please position your work in the current discussion regarding this issue. See for example: Zinsstag, J., and Winkler, M. (2024). Towards One Health Impact Assessment. One Health: Connecting Humans, Animals and the Environment. University of Basel. https://www.futurelearn.com/info/courses/one-health/0/steps/25513 Typos Page 4, Figure 1, in “2. Scoping”: remove the second “the” before “boundaries” Page 5, left column, 2nd paragraph: remove “to” between “prior to each event” and “and were also encouraged to make…” Page 6, left column, 2nd paragraph, third line: I think you meant “four workshops” when you said “to maximise the possibility of the same people attending all three workshops” Page 10, left column, 2nd paragraph: there seems to be a problem with the sentence “enabling integrated consideration (and mitigation) of cumulative impacts from 13, 17, 18, 22, 31” Page 10, left column, 4th paragraph, 4th line: there is a problem with the way “appraisal stage” appears in the article Page 10, right column, 4th paragraph, last line: I think you meant “ethical use of evidence” when you said “given that ethical use of value is a core value of HIA” Page 11, right column, 2nd paragraph, 1st line: there is a problem with the way “identified” appears in the article Page 12, right column, 5th paragraph, 8th line: please consider changing “to contribute systems approaches to a forthcoming HIA” by “to apply systems approaches to a forthcoming HIA” - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? Yes - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Not applicable - Are all the source data underlying the results available to ensure full reproducibility? Yes - Are the conclusions drawn adequately supported by the results? Yes

1. Douglas MJ, Foster C, McDermott R, Bunse L, et al.: Can health impact assessment help tackle commercial determinants of health and support a wellbeing economy?. Health Promot Int. 2024; 39 (6). PubMed Abstract | Publisher Full Text2. Lilly K, Kean B, Hallett J, Robinson S, et al.: Factors of the policy process influencing Health in All Policies in local government: A scoping review.Front Public Health. 2023; 11: 1010335 PubMed Abstract | Publisher Full Text 3. Jabot F, Romagon J, Dardier G: A New Framework for Monitoring and Evaluating Health Impact Assessment: Capitalising on a French Case Study with the Literature in Evaluation.Int J Environ Res Public Health. 2024; 21 (9). PubMed Abstract | Publisher Full Text 4. Craig P, Dieppe P, Macintyre S, Michie S, et al.: Developing and evaluating complex interventions: the new Medical Research Council guidance.BMJ. 2008; 337: a1655 PubMed Abstract | Publisher Full Text Competing Interests: No competing interests were disclosed. Reviewer Expertise: Political science and public health ("health in all policies" practical implementation at the local level, with a specific focus on the power dynamics, policy instruments - like HIA - and emerging professions that regulate cross-sectoral relationships) CITE HOW TO CITE THIS REPORT Dardier G. Reviewer Report For: Developing methods for systems-informed Health Impact Assessment (System-HIA) [version 1; peer review: 3 approved with reservations]. NIHR Open Res 2025, 5:11 (https://doi.org/10.3310/nihropenres.14790.r34773) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-34773 https://openresearch.nihr.ac.uk/articles/5-11/v1#referee-response-34773 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | ||| |---|---|---|---| | 1 | 2 | 3 | | | Version 1 11 Feb 25 | read | read | read | Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...