The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland

Electronic screening, paper screening, electronic health records, maternity guideline implementation ALL Metrics - Views Downloads How to cite this article Viner A, Deeks O, Nayyar P et al. The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.13791.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. Export Citation Sciwheel EndNote Ref. Manager Bibtex ProCite Sente Select a format first ▬ ✚ Research Article [version 1; peer review: 2 approved with reservations] Alex Viner1, Oscar Deeks2, Pamela Nayyar https://orcid.org/0000-0003-3572-7886 2, [...] Jennifer Allison3, Sarah Murray4, Katherine Ainslie5, Neil Cockburn2, Richard Lilford https://orcid.org/0000-0002-0634-984X 2, Brian Magowan5Alex Viner1, Oscar Deeks2, [...] Pamela Nayyar https://orcid.org/0000-0003-3572-7886 2, Jennifer Allison3, Sarah Murray4, Katherine Ainslie5, Neil Cockburn2, Richard Lilford https://orcid.org/0000-0002-0634-984X 2, Brian Magowan5 PUBLISHED 23 Apr 2025 Author details Author details 1 Queen Elizabeth University Hospital, Glasgow, Scotland, UK 2 School of Health Sciences, University of Birmingham, Birmingham, England, UK 3 Victoria Hospital, Kirkcaldy, Scotland, UK 4 Centre for Reproductive Health, The University of Edinburgh, Edinburgh, Scotland, UK 5 NHS Borders, Melrose, Scotland, UK 2 School of Health Sciences, University of Birmingham, Birmingham, England, UK 3 Victoria Hospital, Kirkcaldy, Scotland, UK 4 Centre for Reproductive Health, The University of Edinburgh, Edinburgh, Scotland, UK 5 NHS Borders, Melrose, Scotland, UK Alex Viner Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing Oscar Deeks Roles: Formal Analysis Roles: Formal Analysis Pamela Nayyar Roles: Visualization Roles: Visualization Jennifer Allison Roles: Investigation, Writing – Original Draft Preparation Roles: Investigation, Writing – Original Draft Preparation Sarah Murray Roles: Formal Analysis, Writing – Original Draft Preparation Roles: Formal Analysis, Writing – Original Draft Preparation Katherine Ainslie Roles: Investigation Roles: Investigation Neil Cockburn Roles: Validation Roles: Validation Richard Lilford Roles: Formal Analysis, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Formal Analysis, Writing – Original Draft Preparation, Writing – Review & Editing Brian Magowan Roles: Conceptualization, Data Curation, Funding Acquisition, Writing – Review & Editing Roles: Conceptualization, Data Curation, Funding Acquisition, Writing – Review & Editing OPEN PEER REVIEW REVIEWER STATUS It is easy to overlook risk factors that require specific healthcare actions. This is particularly true in maternity care, which deals with a natural process where risk might be distinguished from normality at many points in the care pathway. In this paper, we describe the effects of clinical decision support, first in the form of paper checklists and then in the form of an electronic checklist to screen for risks of (1) venous thromboembolism (VTE), (2) intrauterine growth restriction (IUGR), (3) high body mass index (BMI), and (4) gestational diabetes mellitus (GDM). Here, we track the effects of different screening algorithms introduced on different media (paper versus computer). We screened sequential maternity records at three time points: baseline, following the introduction of a paper checklist, and following the introduction of the electronic system. First, we examined (at each time-point) the proportion of pregnancies appropriately screened at each time point. Second, we examined the proportion of correct actions taken following a positive screening result. The study was conducted at a District General Hospital in Scotland between 2011 and 2015, which covered the introduction of the above system to screen patients and suggest appropriate management for positive cases. We found that the introduction of a paper checklist was associated with an increased proportion of pregnancies appropriately screened and correct actions taken contingent on positive screening. These trends continued after the introduction of the electronic prompts. Compliance with maternity guideline recommendations for VTE, high BMI, high risk of fetal growth restriction, and GDM improved over time with the introduction of paper and electronic prompts. Electronic maternity screening is significantly superior to no screening or paper-based screening for VTE, BMI and GDM. Nil additional unpublished data from the study are available. Maternity care identifies risks in an essentially normal population. However, it is easy to overlook the risk factors among the masses of information collated. To mitigate this problem, we implemented a checklist system. These checklists were first implemented in paper form and then in electronic form. These checklists covered (1) the risk of venous thrombosis and embolism, risk of stillbirth, and risks related to body mass index and gestational diabetes. We sampled three sets of case notes at the end of calendar years 2011, 2014, and 2015, corresponding to points prior to paper checklists, after implementation of paper checklists, and after implementation of the electronic system. Each set of case notes was examined to determine the rates of adherence to the established standards of care related to the above risks. Electronic screening, paper screening, electronic health records, maternity guideline implementation Corresponding Author(s) Brian Magowan ([email protected]) Grant information: This project is funded by the National Institute for Health and Care Research (NIHR) under its [NIHR Applied Research Collaboration West Midlands (ARCWM) and Midlands Patient Safety Research Collaboration (PSRC) (Grant Reference Number NIHR200165)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2025 Viner A et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Viner A, Deeks O, Nayyar P et al. The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.13791.1) First published: 23 Apr 2025, 5:37 (https://doi.org/10.3310/nihropenres.13791.1) Latest published: 16 Jul 2025, 5:37 (https://doi.org/10.3310/nihropenres.13791.2) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. There is a newer version of this article available. of this article available. Clinical decision support has a long history. MacDonald published a paper in the New England Journal of Medicine on decision support to prompt clinicians as long ago as 19761. His paper addressed what he called the “non-perfectibility” of humans. Since then, numerous methods have been developed to provide clinical support. Computerised decision support for medical prescribing is an archetypal and well-studied example of such decision support2. We shall refer to these decision support systems that identify a risk and then suggest an action to mitigate the risk as ‘prompts’. Despite their widespread use in medicine and prescribing, systems to prompt clinicians to take action have not been widely used in maternity care. This is in contrast to systems to inform and support decision making once a risk has been identified, for example, to help people choose between vaginal and caesarean birth plans or whether or not to undergo invasive prenatal testing for congenital conditions3. A randomized trial in 1992 showed that use of structured data and prompts could improve adherence to established principles of safe/effective antenatal care4. Only seven studies, all in Low or Middle-income countries, have sought to replicate these findings over the past thirty-one years (see discussion). In this study, we evaluated a program to implement clinical prompts in a District General Hospital (Borders General Hospital) in Scotland. Clinicians were prompted to act where necessary, in accordance with four sets of evidence-based national guidelines. The prompts were designed to (1) collect appropriate data, (2) alert clinicians to any risk, and (3) suggest contingent action when the risk is present. The study period was 2011 to 2015. During this period, changes were made in the “message and the medium” – different messages/prompts (e.g. for venous thrombosis and diabetes) were introduced at different times on different media (paper or computer). Initially, the prompts were paper-based in the form of a paper checklist that the clinician was required to complete for all the patients. At a later stage, the paper checklist was supplemented by an electronic form called the Safe Assessment Form (SAFER checklist) to Evaluate Risks. This phasing of prompts enabled us to track compliance with standards of care according to the particular standard and type of medium used to present the prompt to the clinician. More specifically, we tracked the extent to which clinicians complied with criteria for safe care over three time periods according to whether or not the particular prompt had been implemented and according to the method of presentation. After completion, the paper checklist or electronic print-outs were incorporated into clinical notes. We sampled three sets of case notes at the end of the calendar years 2011, 2014, and 2015, corresponding to points prior to paper checklists, after implementation of the paper checklists, and after implementation of the electronic (SAFER) system. Each set of case-notes was examined to determine the rates of adherence to established standards of care – thus we carried out three cross-sectional sets of observations or ‘audits.’ Standards We established four sets of standards, each addressing one of the four clinical areas described below. Each standard contained two components. (1) Risk assessment component and (2) contingent action component. The risk-assessment should be carried out on all patients and the ‘contingent action’ should be carried out in those cases where it is indicted on the basis of the risk-assessment – in other words contingent on the case being high risk. Risk assessment was based on a checklist of questions and observation of body mass index (BMI). The system algorithm then issues prompts based on certain data, either singly or in combination. A corollary of this sequence (assessment and action suggestion) is that, while all cases sampled would be eligible for the assessment/screening step, only a subset (those who screen positive) would be eligible for the contingent action step, a point to which we shall return. The four standards, all based on national guidelines, were:- 1) Venous thromboembolism (VTE), based on the Royal College of Obstetricians and Gynaecologists (RCOG) guidelines5. VTE is the leading cause of direct maternal death in the UK6. There are three time points at which a VTE prompt is appropriate – antenatally (at the initial ‘booking’ visit or, when a patient is admitted) and postnatally. However, these were not introduced simultaneously, as described below. 2) Intrauterine growth restriction (IUGR). There are many factors that increase the risk of intrauterine growth restriction, which is a precursor to poor perinatal outcome, and contingent actions can be taken to reduce this risk7. This standard is also underpinned by RCOG guidance8. There are many possible actions that might be prompted when the risk of IUGR is detected, but here we concentrate on serial growth scans because of consensus on this point. 3) Body Mass Index (BMI). Again, this standard is covered by an RCOG guideline, itself based on the need to reduce obesity-associated outcomes including gestational diabetes, pre-eclampsia, perinatal death and miscarriage9. There are three actions that follow positive screening for BMI-behavioural change support: performance of an oral glucose tolerance test (OGTT), and prescription of certain antibiotics. 4) Screening for gestational diabetes (GDM) – a topic related to obesity and again supported by guidance and evidence10,11 that adverse pregnancy can be avoided12,13. We identified the performance of a Glucose Tolerance Test as a definitive test as the appropriate action for women screening positive. Phasing of intervention The VTE prompt is invoked in three stages in the patient pathway (above). The postnatal VTE prompt was implemented many years prior to the initial observation at the end of 2011. Thus, there was no pre-implementation period for this component of the VTE prompt. Antenatal VTE prompts, however, were implemented soon after 2011 and before the second observation period in 2014. The IUGR prompt was implemented after 2011 (in 2013) and before 2014. However, the BMI and GDM prompts were only implemented in 2015. The phasing of interventions (prompts) and observation periods (audits) are shown in Figure 1. We also provide a summary of the prompt media used across each observation point in Table 1. | 2011 | 2014 | 2015 | | |---|---|---|---| | Antenatal VTE | No formal screening | Paper checklist | SAFER | | Postnatal VTE | Paper checklist | Paper checklist | SAFER | | IUGR | NA* | Paper checklist | SAFER | | BMI | No formal screening | No formal screening | SAFER | | Gestational Diabetes | No formal screening | No formal screening | SAFER | Data collection points and missing data The observations available for analysis were as follows: 1) Postnatal VTE. Collected at baseline and midline but implemented before the 2011 baseline. 2) Antenatal VTE. Collected at baseline, midline and endline. 3) IUGR. Not collected at baseline (the intervention was not implemented until 2013 and, unfortunately, data were missing for the effect of moving from no prompts to paper prompts. Collected at midline and endline. 4) BMI and GDM. Collected at baseline, midline and endline. Observations Audits were carried out in the form of cross-sectional observations at the above three time points. At each observation time point, we sampled 100 consecutive case notes from the singleton pregnancies. Completed checklists or electronic printouts were included in these case notes. These documents and corresponding case notes were scrutinized to observe whether the required data had been collected and whether any contingent actions prompted by the system had been carried out. For example, in a set of 100 case notes, we first observed whether the risk factors for fetal risk were collected. Then, if a prompt was issued on the basis of this information, we recorded if that contingent action (in this scenario, case-serial scans) was taken. The denominator for this contingent action would be the number of cases where an action was required according to “opportunity for error” principles14. We represent this dual quality assessment process in Figure 2 in relation to the IUGR risk factor. The sample size of 100 in each group provides 90% power to identify a 20% difference in compliance with a screening standard across any two time periods. Precision was lower with respect to contingent actions. To investigate the relationship between the different media used for screening algorithms in maternity care and the prompt completion proportion, we used logistic regression modelling. We used descriptive statistics to summarize the proportions of the prompts completed successfully for each clinical area, split into each time point, where the denominators are the total number of participants at each time point. We summarized two different proportions for each clinical area: the proportion of correctly completed risk assessments, and the proportion of contingent actions completed successfully. All analyses were performed using R 4.4.1. Two logistic mixed-effects regression models were used to estimate the association between the medium of the prompt and how successfully it was completed. One model was used to examine the proportion of successfully completed risk assessments and the other to examine the proportion of successfully completed contingent actions. We pooled the clinical areas to estimate the mean difference in the proportion of risk assessments and contingent actions completed successfully across patients and clinical areas. The dependent variable was coded as 1 if the relevant part of the prompt was completed successfully and 0 if it was not. This model aims to find a more succinct estimate of whether there is an association between the medium of the prompt and the chance of the prompt being completed successfully. We included the following explanatory variables: type of prompt (the explanatory variable of interest), clinical area (VTE, IUGR, BMI, GDM), and year (2011, 2014, 2015). There is collinearity between the year 2015 and the medium type of a SAFER prompt, as in 2015, SAFER prompts were implemented across all clinical areas. We also included a mixed-effect random variable in one of our models to account for the possibility that each person could have multiple records within each clinical area related to different contingent actions. Using the values in Table 2, we adjusted for the average age and BMI each year. We also tested the collinearity of the variables included in the models (Table 3), where the generalized VIF values were < 3, showing low to moderate collinearity. | Cohorts | 2011 | 2014 | 2015 | |---|---|---|---| | Number of patients (n) | 100 | 100 | 100 | | Mean Age in years (SD) | 28.4 (6.3) | 27.2 (6.3) | 28.3 (5.5) | | Mean BMI (SD) | 26.6 (6.1) | 25.6 (6.5) | 27.3 (7.2) | Among the 300 participants in the study, the mean age was 28.0, and the mean BMI was found to be 26.5. These values were similar across the three study epochs (Table 2). As stated, to standardize, we simplified the questions to the same two questions for each clinical area: “Was the correct risk assessment made?” and “Was the correct contingent action applied when appropriate?”. The question related to risk assessment contained a value for all 100 participants over each of the three time points. For this value to be positive for each participant, the risk assessment must have been correct. If either of these criteria was not met, it was recorded as not having been completed when calculating the proportion. The question relating to contingent actions only contained values where contingent actions were needed, with the proportion being the total number of contingent actions that were completed correctly for each clinical area out of the total number of actions that should have been completed. Note that there are some areas where participants can have multiple treatments, causing the total number of contingent actions to be larger than the number of eligible people. The results for all the four conditions are presented in Table 4. Antenatal VTE prophylaxis There was a significant improvement from 2014 to 2015 when there was a change from using a paper checklist to an electronic SAFER prompt. In 2014 34.5% of the risk assessments were completed correctly, compared to 96.5% of SAFER prompts completed correctly in 2015. However, there appeared to be little difference in the proportion of correct contingent actions applied when the SAFER prompt was introduced. Postnatal VTE prophylaxis However, the implementation of postnatal VTE preceded the first observation period (Figure 1). There appears to be a ceiling effect with how often postnatal VTE risk assessments are completed correctly, as 93%, 97%, and 98% of the risk assessments were completed correctly in 2011, 2014, and 2015, respectively. The same is the case for treatment of postnatal VTE, where 96.8%, 100%, and 96.5% of the indicated treatments were completed successfully. This contrasts with the pattern for antenatal VTE above, where high compliance was delayed until after its guideline came into operation. IUGR There were no guidelines in 2011 regarding IUGR risk assessments nor were data collected at this time point. In 2014 and 2015, 93% and 97% of IUGR risk assessments, respectively, were completed correctly. There is some suggestion of an association between improvement with respect to this prompt type when considering whether the correct contingent actions were applied, where 70/81 (86.4%) contingent actions were applied successfully in 2014 and 81/87 (93.1%) in 2015 when the SAFER prompt was implemented. BMI When observing both the proportion of BMI risk assessments completed correctly and the proportion of contingent actions applied correctly, there appears to be an increase from 2011 to 2014, although there are no formal systems (checklists or SAFER electronic systems) used in either year. Contingent action, for example, increased from 31% to 85%. This may suggest that there is a temporal association, net of any intervention, towards improved surveillance over those years. There appears to be no significant difference between 2014 and 2015 once SAFER prompts had been implemented, with differences of 0 and 2 percentage points for assessment and contingent action, respectively. GDM Similar to BMI, no checklist intervention was implemented before the SAFER intervention between 2014 and 2015. Again, the proportion of GDM risk assessments that were completed correctly was very high in all of our samples. Although the proportion of correctly completed risk assessments is lower post-SAFER implementation in comparison to 2014 before prompts were implemented (100% (2014) and 99% (2015), the difference in these proportions is too small to support any conclusions. This is another example of the ceiling effect that appears when looking at how often risk assessments are correctly completed. We also observed no improvement with respect to the proportion of contingent actions completed correctly when comparing 2014 to 2015; however, there was a large increase from 2011 to 2014, reinforcing the suggestion that there may be an association between the time in which the risk assessment was used and an improvement in completing the risk assessments successfully, that is, a temporal trend. Risk assessment model Using the coefficients from Table 5, we can calculate the difference in the absolute probabilities of risk assessments being completed correctly between different media of prompt. We begin by examining the differences between non-formal screening methods and paper checklists. There was an absolute difference in probabilities of 0.078 (7.8%) between risk assessments being completed correctly using non-formal versus paper prompts. After this, we looked at the difference between the proportion of completed paper checklists and SAFER prompts, of which there was an absolute difference of 0.006 (<1%). This value is too small to be significant given our sample size, and is affected by the ceiling effect stated earlier. The use of SAFER prompts versus non-formal screening techniques was associated with a higher proportion of correctly completed risk assessments. There is an absolute difference in probabilities of 0.08 (8%) between risk assessments being completed correctly using either non-formal prompts or SAFER prompts. Contingent action model The first comparison of contingent actions was made between non-formal screening techniques and paper checklists using the values from Table 6. There was an absolute difference in probabilities of 0.05 (5%) between the probability that a contingent action was completed correctly using paper-checklists versus using non-formal screening techniques. When comparing the proportion of successfully completed contingent actions between the prompt medium of paper and SAFER prompts, we obtained a log odds estimate of 0.2058, which corresponds to a difference in probabilities of -0.007 (-<1%) between paper and SAFER prompts. This interpretation suggests that there is a slightly smaller chance that a SAFER prompt will have a successfully completed contingent action; however, this value is too small to be significant. As shown in the results, there is a very large increase in the absolute difference between informal screening techniques and the other two prompt media. After adjusting for confounders in our model, the use of SAFER prompts over non-formal screening techniques was associated with a higher proportion of successfully completed contingent actions. There is an absolute difference in probabilities of 0.10 (10%) when comparing the probability of a contingent action being completed successfully between non-formal screening techniques and SAFER prompts. One limitation of these models is that because all SAFER prompts were implemented in 2015, we were unable to account for the confounding effect of time; therefore, the odds ratios and absolute differences in probability include any confounding effect of time from 2011 to 2015. Even with this in mind, we are still seeing an increase in the proportion of prompts being completed correctly, which is a positive sign that risk screening in maternity care is improving, even if we cannot isolate the cause of this specifically. A clear trend of improvement is seen across epochs: the proportions screened and proportions where the correct action was taken increased progressively from one epoch to the other. Only in the case of postnatal VTE is there no trend; this results from high baseline performance, leaving almost no headroom for further improvement. A checklist for postnatal VTE assessment preceded that of 2011; hence, our first set of observations. Thereafter, an antenatal VTE assessment was implemented, with subsequent improvement in assessment and contingent action. This pattern is certainly consistent with an intervention effect, but other explanations are plausible. One such explanation is the so-called rising tide phenomenon15 where concern across a social system prompts general improvement and specific interventions. The headroom for improvement from the latter is reduced because of spontaneous improvements across the system. There is little doubt that real improvement occurred over time; precision is low with respect to contingent actions, but the pattern in the data is highly consistent. The question is whether these changes were causal or resulted from other factors that created a secular trend. The fact that changes in the medium and message did not correspond to the dataset provides some clues. The improvement in adherence with respect to GDM and BMI over both earlier epochs, despite not using paper checklists, provides some evidence for a secular trend. On the other hand, as stated, the VTE data point towards a causal explanation since antenatal and intrapartum performance improved only after the checklists were introduced. The rapidity of change - over only one year - for standards where ceiling effects were not in evidence may provide at least some evidence that the effects were (at least partially) causal. However, the data presented here have limitations in the sense that no data were collected for IUGR in 2011. The 1992 randomized trial of prompts quoted in the introduction found that both paper and computer prompts caused a sharp increase in compliance with quality standards compared to no prompt at all. The results regarding antenatal VTE were consistent with these earlier findings. While our study has significant limitations, it is one of only a few studies evaluating prompts in antenatal care. A recent systematic review summarised the world’s literature on decision support in maternity care16. This literature described evaluation of many different types of decision support, covering decision aids for patient choice, prescribing systems, risk scores and decision prompts (covered here). The decision prompts covered postnatal and intrapartum care as well as single and multiple issues. We found seven studies (all from low- and middle-income countries) that evaluated multiple antenatal prompts, analogous to the study cited here (Table 7). The findings, taken in the round, confirm our findings that antenatal decision support in the form of prompts is associated with improved compliance with criteria of effective antenatal care. The prompt systems supplemented pre-existing paper records. A much neater solution is to integrate prompts into an electronic record system. Since our study, electronic maternity records have been widely adopted in the UK. Thus, there is a need to evaluate the performance over the years following our observations, during which electronic records, some of which incorporate prompts, have been introduced. To gain an insight into the causal relationship between the improvement of completion and correctness of risk assessments and the contingent actions that follow, there should be a more in-depth analysis of a larger dataset that is able to account for all the confounding factors. | Authors | Intervention | Study Type | Location | Findings | |---|---|---|---|---| | Lilford, et al. 19924 | Unstructured notes, structured notes and computer system | Individual patient 3-way RCT | England | Structured histories and antenatal prompts improved adherence to care processes. | | Horner, et al. 201319 | eHealth decision support system – compliance to care protocols | Before-and-after | South Africa | eHealth improved compliance with standards over a checklist system. | | Venkateswaran, et al. 202220 | Digital clinical support vs. paper records | Cluster RCT | Palestine | Decision support superior for most process outcomes. | | McNabb, et al. 201521 | Use of an app with decision support | Before-and-after | Nigeria | Most process outcomes improved. | | Maïga, et al. 202322 | Digital clinical support | Cross-sectional, post-intervention, observational study | Burkina Faso | No difference in quality-of-care scores, but counselling and malaria prophylaxis improved in intervention clinics. | | Benski, et al. 201723 | Mobile health system | Cross-sectional pilot study | Madagascar | The mHealth system studied is a feasible method for collecting large amounts of patient information. | | Saronga, et al. 201724 | Electronic clinical decision support system | Quantitative pre- and post- intervention study | Tanzania | Marginal improvement in individual process quality variables (e.g. history taking). No significant improvement on overall process quality of care. | | Ahmad, et al. 201625 | Educating on correct procedure for documenting VTE assessments | Before-and-after | England | Increase in completion of VTE assessments on EPR for the duration of the study. | The introduction of maternity records is a significant implementation that has taken place in the absence of any evaluation of which we are aware. Therefore, it is critical to evaluate these new systems. While the data presented here suggest that incorporating the above prompts in new electronic maternity systems is a good idea, there is no certainty that they were optimally configured for maximal impact. Electronic notes provide the basis for examining different formats and a wider set of prompts than we have examined, including blood test results (e.g., rhesus disease/anemia), medical disorders, medicine prescriptions, and mental health problems. There is a real risk of information overload with modern electronic systems17,18 and it will be important to study the effect of further formal systems and design systems to suit human users, not the other way around. Our retrospective study provides evidence on the value of prompts, particularly electronic prompts. To date, our study represents the only formal evidence relating to prompts incorporated in modern maternity electronic notes. The need for ethical approval and consent were not required. The local audit committee, NHS Borders Clinical Governance and Quality ruled on 13/01/2017 (application reference number 2016-743) that no formal ethics approval was required. There were no participants involved as the study was an audit to evaluate service improvement. We comply with guidance in paragraph 79 of the General Medical Council’s Confidentiality: good practice in handling patient information (https://www.gmc-uk.org/professional-standards/the-professional-standards/confidentiality/ethical-and-legal-duties-of-confidentiality) which states that “Anonymised information will usually be sufficient for purposes other than the direct care of the patient and you must use it in preference to identifiable information wherever possible.” The information we use is totally anonymised. We also comply with paragraph 81 of the guidance, “The Information Commissioner’s Office anonymisation code of practice (ICO code) considers data to be anonymised if it does not itself identify any individual, and if it is unlikely to allow any individual to be identified through its combination with other data.” The data contains no information that would allow it to be linked back to patient notes. Furthermore, we also comply with paragraph 14g of the General Medical Council’s Confidentiality: good practice in handling patient information (https://www.gmc-uk.org/professional-standards/the-professional-standards/confidentiality/ethical-and-legal-duties-of-confidentiality). Fighshare: SAFER study_audit case notes, DOI: https://doi.org/10.6084/m9.figshare.28483838.v226. The project contains the following underlying data: Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0). Faculty Opinions recommendedReferences - 1. McDonald CJ: Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976; 295(24): 1351–5. PubMed Abstract | Publisher Full Text - 2. Trotter NE, Slight SP, Karimi R, et al.: The effect of digital antimicrobial stewardship programmes on antimicrobial usage, length of stay, mortality and cost. Inform Med Unlocked. 2023; 37: 101183. Publisher Full Text - 3. Dugas M, Shorten A, Dube E, et al.: Decision tools to support women’s decision making in pregnancy and birth: a systematic review and meta-analysis. Soc Sci Med. 2012; 74(12): 1968–1978. PubMed Abstract | Publisher Full Text - 4. Lilford RJ, Kelly M, Baines A, et al.: Effect of using protocols on medical care: randomised trial of three methods of taking an antenatal history. BMJ. 1992; 305(6863): 1181–4. PubMed Abstract | Publisher Full Text | Free Full Text - 5. Royal College of Obstetricians and Gynaecologists: Reducing the risk of venous thromboembolism during pregnancy and the puerperium. Green-top Guideline No.37a. London: RCOG, 2015. Reference Source - 6. Knight M, Nair M, Tuffnell D, (Eds.), et al.: Saving Lives, Improving Mothers’ Care. Lessons learned to inform maternity care from the UK and Ireland confidential enquiries into maternal deaths and morbidity 2013–15. Oxford: National Perinatal Epidemiology Unit, University of Oxford, 2017. Reference Source - 7. Gardosi J, Giddings S, Clifford S, et al.: Association between stillbirth rates in England and uptake of accreditation training in customized fetal growth assessment. BMJ Open. 2013; 3(12): e003942. PubMed Abstract | Publisher Full Text | Free Full Text - 8. Royal College of Obstetricians and Gynaecologists: Small-for-gestational-age fetus, investigation and management. Green-top Guideline No.31. London: RCOG, 2015. Reference Source - 9. Royal College of Obstetricians and Gynaecologists: Management of women with obesity in pregnancy. London: CMACE/RCOG; 2010. Reference Source - 10. Scottish Intercollegiate Guidelines Network (SIGN): Management of diabetes: a national clinical guideline. Edinburgh: SIGN; 2010. Reference Source - 11. Royal College of Obstetricians and Gynaecologists: Diagnosis and treatment of gestational diabetes. Scientific Impact Paper No.23. London: RCOG; 2011. Reference Source - 12. Crowther CA, Hiller J, Moss JR, et al.: Effect of treatment of gestational diabetes mellitus on pregnancy outcomes. N Engl J Med. 2005; 352(24): 2477–2486.PubMed Abstract | Publisher Full Text - 13. Landon MB, Spong CY, Thom E, et al.: A multicentre, randomized trial of treatment for mild gestational diabetes. N Engl J Med. 2009; 361(14): 1339–1348. PubMed Abstract | Publisher Full Text | Free Full Text - 14. Brown C, Hofer T, Johal A, et al.: An epistemology of patient safety research: a framework for study design and interpretation. Part 2. study design. Qual Saf Health Care. 2008; 17(3): 163–169. PubMed Abstract | Publisher Full Text - 15. Chen YF, Hemming K, Stevens AJ, et al.: Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015; 25(5): 303–10. PubMed Abstract | Publisher Full Text | Free Full Text - 16. Cockburn N, Osborne C, Withana S, et al.: Clinical decision support systems for maternity care: a systematic review and meta-analysis. EClinicalMedicine. 2024; 76: 102822. PubMed Abstract | Publisher Full Text | Free Full Text - 17. Lilford RJ: The information paradox at the heart of non-directive counselling. NIHR ARC West Midlands & NIHR Midlands PSRC News Blog. 2024; 6(4): 1–3. Reference Source - 18. Singh H, Spitzmueller C, Petersen NJ, et al.: Information overload and missed test results in electronic health record-based settings. JAMA Intern Med. 2013; 173(8): 702–4. PubMed Abstract | Publisher Full Text | Free Full Text - 19. Horner V, Rautenbach P, Mbananga N, et al.: An e-health decision support system for improving compliance of health workers to the maternity care protocols in South Africa. Appl Clin Inform. 2013; 4(1): 25–36. PubMed Abstract | Publisher Full Text | Free Full Text - 20. Venkateswaran M, Ghanem B, Abbas E, et al.: A digital health registry with clinical decision support for improving quality of antenatal care in Palestine (eRegQual): a pragmatic, cluster-randomised, controlled, superiority trial. Lancet Digit Health. 2022; 4(2): e126–e36. PubMed Abstract | Publisher Full Text | Free Full Text - 21. McNabb M, Chukwu E, Ojo O, et al.: Assessment of the quality of antenatal care services provided by health workers using a mobile phone decision support application in northern Nigeria: a pre/post-intervention study. PLoS One. 2015; 10(5): e0123940. PubMed Abstract | Publisher Full Text | Free Full Text - 22. Maïga A, Ogyu A, Millogo RM, et al.: Use of a digital job-aid in improving antenatal clinical protocols and quality of care in rural primary-level health facilities in Burkina Faso: a quasi-experimental evaluation. BMJ Open. 2023; 13(9): e074770. PubMed Abstract | Publisher Full Text | Free Full Text - 23. Benski AC, Stancanelli G, Scaringella S, et al.: Usability and feasibility of a mobile health system to provide comprehensive antenatal care in low-income countries: PANDA mHealth pilot study in Madagascar. J Telemed Telecare. 2017; 23(5): 536–543. PubMed Abstract | Publisher Full Text - 24. Saronga HP, Duysburgh E, Massawe S, et al.: Cost-effectiveness of an electronic clinical decision support system for improving quality of antenatal and childbirth care in rural Tanzania: an intervention study. BMC Health Serv Res. 2017; 17(1): 537. PubMed Abstract | Publisher Full Text | Free Full Text - 25. Ahmad AN, Byrne ML, Imambaccus N, et al.: Venous thromboembolism capture on electronic systems in obstetrics patients at St Thomas’ Hospital. BMJ Qual Improv Rep. 2016; 5(1): u212405.w5122. PubMed Abstract | Publisher Full Text | Free Full Text - 26. Magowan B, Nayyar: SAFER study_audit case notes. figshare. Dataset. 2025. http://www.doi.org/10.6084/m9.figshare.28483838.v2 Author details Author details 1 Queen Elizabeth University Hospital, Glasgow, Scotland, UK 2 School of Health Sciences, University of Birmingham, Birmingham, England, UK 3 Victoria Hospital, Kirkcaldy, Scotland, UK 4 Centre for Reproductive Health, The University of Edinburgh, Edinburgh, Scotland, UK 5 NHS Borders, Melrose, Scotland, UK 2 School of Health Sciences, University of Birmingham, Birmingham, England, UK 3 Victoria Hospital, Kirkcaldy, Scotland, UK 4 Centre for Reproductive Health, The University of Edinburgh, Edinburgh, Scotland, UK 5 NHS Borders, Melrose, Scotland, UK Alex Viner Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing Oscar Deeks Roles: Formal Analysis Roles: Formal Analysis Pamela Nayyar Roles: Visualization Roles: Visualization Jennifer Allison Roles: Investigation, Writing – Original Draft Preparation Roles: Investigation, Writing – Original Draft Preparation Sarah Murray Roles: Formal Analysis, Writing – Original Draft Preparation Roles: Formal Analysis, Writing – Original Draft Preparation Katherine Ainslie Roles: Investigation Roles: Investigation Neil Cockburn Roles: Validation Roles: Validation Richard Lilford Roles: Formal Analysis, Writing – Original Draft Preparation, Writing – Review & Editing Roles: Formal Analysis, Writing – Original Draft Preparation, Writing – Review & Editing Brian Magowan Roles: Conceptualization, Data Curation, Funding Acquisition, Writing – Review & Editing Roles: Conceptualization, Data Curation, Funding Acquisition, Writing – Review & Editing Competing interests No competing interests were disclosed. Grant information This project is funded by the National Institute for Health and Care Research (NIHR) under its [NIHR Applied Research Collaboration West Midlands (ARCWM) and Midlands Patient Safety Research Collaboration (PSRC) (Grant Reference Number NIHR200165)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Article Versions (2) Copyright © 2025 Viner A et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. metrics VIEWS $counts.viewCount downloads Citations CITE how to cite this article Viner A, Deeks O, Nayyar P et al. The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.13791.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article. track receive updates on this article Track an article to receive email alerts on any updates to this article. Current Reviewer Status: ? Key to Reviewer Statuses VIEW HIDE ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions Version 1 VERSION 1 PUBLISHED 23 Apr 2025 Views 0 How to cite this report: Ngwenya MW. Reviewer Report For: The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.14980.r35641) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35641 https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35641 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 09 Jun 2025 Mxolisi W Ngwenya, University of Limpopo, Mankweng, Limpopo, South Africa Approved with Reservations VIEWS 0 Thank you for the opportunity to review the paper entitled, “The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland” The present a novel ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close Thank you for the opportunity to review the paper entitled, “The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland” The present a novel approach towards early detection of high-risk conditions in pregnancy. The study findings are compelling and are of significance in advancing maternal health in clinical practice. However, I have a few comments and suggestions for the authors.

Abstract - The abstract is well-designed and highlight some of the significant information to the readers. However, to enhance its readability I suggest that in the background should highlight the briefly indicate a research gap to why the study was conducted, particularly in Scotland. In methods, I suggest the authors highlight the research methodology followed (If it is quasi-experimental, please indicate). - Drawing from the abstract conclusion, the authors indicated that the use of the risk assessment is associated with positive outcomes. I suggest they also indicate the implications of this tool (digital/paper-based) in clinical practice and policies. - The introduction and background, mostly focuses on describing the current study process and implementation of the SAFER tool. I suggest the authors provide more detailed information of the importance of the clinical decision tools particularly in maternity units. Moreover, the authors could further describe in detail the potential benefits of clinical decision support tools on pregnancy outcomes based on the existing literature. - In addition, since the tool was then supplemented to be digital thereafter. I strongly suggest the authors add literature to why they opted to supplement it into an electronic form. For instance, the authors include the benefits and positive implications of digital health in the current era. - I suggest that the authors should add the research design followed to enhance the study credibility and reproducibility. - I suggest the authors also indicate on how they sampled the 300 cases, which sampling technique did they follow. This would the credibility of the study and enhance its understandability to the readers. - The authors indicated that they delved into four standards namely, BMI, VTE, IUGR and DM. They devised a set of questions to screen those conditions. I suggest they indicate the process of how the set of questions were developed and validated (i.e. if experts were involved in the content and face validity of the devised questions). if the tool was adapted the authors could indicate so. - In addition, although the study is already implemented. I am just wondering if the authors have a particular reason to why hypertensive disorders in pregnancy were excluded, since it also one of the leading causes of maternal mortality worldwide. - I suggest the authors provide a detailed inclusion and exclusion criteria to which pregnant women were eligible to this intervention. This would enhance the study quality and methodological rigor. - Kindly indicate the time frame if the files were from year-round or only a couple months in each year. (I.e. The data was collected from Jan-Dec in 2011, 2014 and 2015). This could also be well reflected in figure 1. - In data collection, kindly indicate whether the screening was normally done on hospital visits by all pregnant women or on admission or if this tool was incorporated in a routine practice within the hospital. Kindly provide a detailed methods to enhance its methodological rigor. - Kindly indicate on how the duplication of cases was addressed. For instance, if some of the patients were re-admitted. - Please include the demographic data of the participants throughout the years 2011, 2014 and 2015. This will also assist to identify the trends on the demographics associated with the high-risk conditions. (Age, parity, Gravidity, and educational level, etc) This could also reflect the effectiveness of the tool in identifying demographic factors predisposing those pregnant women at risk at developing the identified conditions. - One of the study objectives as highlighted by the authors was to ‘describe the effects of clinical decision support…’ However, the discussion lacks implication of the use of this particular tool in clinical practice. I suggest the authors indicate to whether the use of this tool enhance patient outcomes and improved quality of care. This could be also linked to the high compliance rate with the SAFER in 2015. Thereafter the authors could add more policy and clinical practice implications of introducing this particular tool in the maternity units. - Similar to the abstract, I suggest that the authors indicate the implications of this tool (digital/paper-based) in clinical practice and policies in relevance to their findings. - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? No - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Yes - Are all the source data underlying the results available to ensure full reproducibility? Partly - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: Maternal and digital health CITE HOW TO CITE THIS REPORT Ngwenya MW. Reviewer Report For: The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.14980.r35641) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35641 https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35641 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. - Author Response 10 Sep 2025Richard Lilford, School of Health Sciences, University of Birmingham, Birmingham, UK10 Sep 2025Author ResponseWe thank the reviewer for their thorough consideration of our paper.

We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned ... Continue reading We thank the reviewer for their thorough consideration of our paper.

We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned the research the gap and the relationship between taking action and outcome in the Introduction. Introduction- Further detail on decision tools and relation to outcomes added. - Potential advantages of digital over paper checklists and prompts added. Methods- Design added. - We explicate the design by adding that the consecutive sampling was sequential from on a pre-defined date. - We add that the checklist was implemented ‘verbatim’ form the published checklist with no adaptation, to the contingent action recommended by the Boolean (if/then) algorithm (as laid out in Figure 2). - We did not include hypertension because this would have required a more complex algorithm on which there is less consensus. We have not altered the text because hypertension is one of many potential triggers for a prompt. - We have not produced exclusion criteria since there was only one already listed as our inclusion criteria. - We had stated that we sampled at the end of each calendar year and now add “November and December”. - We now state “booking clinic” – which also answers the next question. Results- Since the cases were consecutive, they represent the users of the service. Our focus was clinical behaviour in terms of adherence to checklist and guidelines for contingent action.

and Conclusion We now specify the relationship between observed change in process (clinical behaviour) and outcome.We thank the reviewer for their thorough consideration of our paper.Competing Interests: None Close

We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned the research the gap and the relationship between taking action and outcome in the Introduction. Introduction- Further detail on decision tools and relation to outcomes added. - Potential advantages of digital over paper checklists and prompts added. Methods- Design added. - We explicate the design by adding that the consecutive sampling was sequential from on a pre-defined date. - We add that the checklist was implemented ‘verbatim’ form the published checklist with no adaptation, to the contingent action recommended by the Boolean (if/then) algorithm (as laid out in Figure 2). - We did not include hypertension because this would have required a more complex algorithm on which there is less consensus. We have not altered the text because hypertension is one of many potential triggers for a prompt. - We have not produced exclusion criteria since there was only one already listed as our inclusion criteria. - We had stated that we sampled at the end of each calendar year and now add “November and December”. - We now state “booking clinic” – which also answers the next question. Results- Since the cases were consecutive, they represent the users of the service. Our focus was clinical behaviour in terms of adherence to checklist and guidelines for contingent action.

and Conclusion We now specify the relationship between observed change in process (clinical behaviour) and outcome. COMMENTS ON THIS REPORT - Author Response 10 Sep 2025Richard Lilford, School of Health Sciences, University of Birmingham, Birmingham, UK10 Sep 2025Author ResponseWe thank the reviewer for their thorough consideration of our paper.

We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned ... Continue reading We thank the reviewer for their thorough consideration of our paper.

We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned the research the gap and the relationship between taking action and outcome in the Introduction. Introduction- Further detail on decision tools and relation to outcomes added. - Potential advantages of digital over paper checklists and prompts added. Methods- Design added. - We explicate the design by adding that the consecutive sampling was sequential from on a pre-defined date. - We add that the checklist was implemented ‘verbatim’ form the published checklist with no adaptation, to the contingent action recommended by the Boolean (if/then) algorithm (as laid out in Figure 2). - We did not include hypertension because this would have required a more complex algorithm on which there is less consensus. We have not altered the text because hypertension is one of many potential triggers for a prompt. - We have not produced exclusion criteria since there was only one already listed as our inclusion criteria. - We had stated that we sampled at the end of each calendar year and now add “November and December”. - We now state “booking clinic” – which also answers the next question. Results- Since the cases were consecutive, they represent the users of the service. Our focus was clinical behaviour in terms of adherence to checklist and guidelines for contingent action.

and Conclusion We now specify the relationship between observed change in process (clinical behaviour) and outcome.We thank the reviewer for their thorough consideration of our paper.Competing Interests: None Close

We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned the research the gap and the relationship between taking action and outcome in the Introduction. Introduction- Further detail on decision tools and relation to outcomes added. - Potential advantages of digital over paper checklists and prompts added. Methods- Design added. - We explicate the design by adding that the consecutive sampling was sequential from on a pre-defined date. - We add that the checklist was implemented ‘verbatim’ form the published checklist with no adaptation, to the contingent action recommended by the Boolean (if/then) algorithm (as laid out in Figure 2). - We did not include hypertension because this would have required a more complex algorithm on which there is less consensus. We have not altered the text because hypertension is one of many potential triggers for a prompt. - We have not produced exclusion criteria since there was only one already listed as our inclusion criteria. - We had stated that we sampled at the end of each calendar year and now add “November and December”. - We now state “booking clinic” – which also answers the next question. Results- Since the cases were consecutive, they represent the users of the service. Our focus was clinical behaviour in terms of adherence to checklist and guidelines for contingent action.

and Conclusion We now specify the relationship between observed change in process (clinical behaviour) and outcome. Views 0 How to cite this report: Ondersma S. Reviewer Report For: The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.14980.r35401) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35401 https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35401 NOTE: it is important to ensure the information in square brackets after the title is included in this citation. Reviewer Report 24 May 2025 Steven Ondersma, Charles Stewart Mott Department of Public Health, Michigan State University, Flint, Michigan, USA Approved with Reservations VIEWS 0 Clinician-directed prompts for risk screening, followed by care recommendations for positive screens, are a well-known approach to facilitating adherence to clinical practice guidelines. This interesting manuscript uses a retrospective review to examine changes in screening rates and subsequent recommended ... Continue reading I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Close Clinician-directed prompts for risk screening, followed by care recommendations for positive screens, are a well-known approach to facilitating adherence to clinical practice guidelines. This interesting manuscript uses a retrospective review to examine changes in screening rates and subsequent recommended action for four specific pregnancy-related risks following implementation of prompts. In a sample of 300 case records, the authors find clear increases in adherence across the three observation points (2011, 2014, and 2015) in 2 of the 4 areas, and high levels of adherence at all observed points for the others. The authors appropriately consider challenges to causal inference but also suggest that the findings support the use of prompts, particularly in electronic format. The authors are quite correct that such techniques are an important area of study, and they make creative use of historical medical records. The measurement of appropriate actions with screen-positive patients, and not just rates of screening, is especially valuable. Unfortunately, multiple factors negatively affect the rigor and ultimately the impact of this effort. - The authors reference the clinical decision support and prompt literatures and suggest that little research has been done regarding prompts in antenatal care in the past several decades, particularly in high-income countries. More relevant research may be available in studies of quality improvement, practice guideline adherence, and Electronic Health Record (EHR)-based prompts. - The timing of abstracted cases does not correspond with the introduction of the prompts into regular clinical practice; case notes were sampled at the end of three separate calendar years sometime after the changes had been introduced. This lack of resolution complicates causal inference, particularly compared to alternative methods such as continuous observation of cases before and after the change was introduced. This is particularly true given references to changes in the message over time (these changes and their timing are not specified) and the likely evolution of EHR risk screening and best practice prompts over time. - More detail is needed regarding how much reviewer judgment was needed when abstracting case notes. For example, did the case notes have clear checkboxes that drove ratings, or did reviewers interpret actions based on written notes? And if the latter, did the investigators use a clear manual and determine inter-rater reliability? - Were records selected consecutively based on dates, or using some other criteria? - Similarly, how were the four risk factors selected? Were other prompts introduced during the study period that were not selected for study? - The manuscript notes that paper checklists were supplemented by the SAFER electronic checklist. Does this mean that some elements of the risk screening in question continued to be done on paper? - EHRs regularly require screening for a wide range of potential risks and provide clinical decision/best practice alerts following a positive screen. Can the authors comment on the extent to which the high rates of adherence observed might be explained by increased use of required fields and best practice alerts in EHRs used in Scotland during the study period? Or the extent to which evidence-based prompts are already entering widespread use via EHRs? - The abstract should include the sample size and statistics to support the key findings. Please also edit the tweetable abstract to be more measured. - Is the work clearly and accurately presented and does it cite the current literature? Partly - Is the study design appropriate and is the work technically sound? Partly - Are sufficient details of methods and analysis provided to allow replication by others? Partly - If applicable, is the statistical analysis and its interpretation appropriate? Partly - Are all the source data underlying the results available to ensure full reproducibility? Yes - Are the conclusions drawn adequately supported by the results? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: Risk screening and technology-based screening and brief intervention for pregnancy-related risks, particularly substance use. CITE HOW TO CITE THIS REPORT Ondersma S. Reviewer Report For: The Safe Assessment Form to Evaluate Risks (‘SAFER’) chart – a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2025, 5:37 (https://doi.org/10.3310/nihropenres.14980.r35401) The direct URL for this report is: https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35401 https://openresearch.nihr.ac.uk/articles/5-37/v1#referee-response-35401 NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article. - Author Response 10 Sep 2025Richard Lilford, School of Health Sciences, University of Birmingham, Birmingham, UK10 Sep 2025Author ResponseWe thank the reviewer for their attentive reading of our paper and for insightful suggestions. - We may have missed other literature. Our search would have included the quality - We may have missed other literature. Our search would have included the quality literature covering antenatal care. It was done independently of a recent literature review cited in our article (Cockburn et al. 2024) and our search included all articles on prompts/action suggestions cited in this paper. - Sampling at the end of calendar years versus continuous observation. Doing so would have introduced (additional) sources of bias. Had we done so, we would have needed an adjustment period for the clinicians to get used to each new prompt system as it was introduced. More observations would have been required. - Role of reviewer judgement. This relates to the " observations” subheading under methods. As stated, we retrieved the case notes. The case notes included the paper checklists that we introduced in the earlier epoch and the electronic print outs from the later epoch. We now make that clearer. We then go on to specify that the checklists and electronic printouts were scrutinised to determine whether the data was collected. Then the case notes themselves were examined to see whether the contingent actions were followed. We have rewritten this part to make this clearer. Thank you. We then add “Thus the data collected were based on clear boxes on the paper or electronic checklist, while the contingent actions were extracted from the case notes.” Then in the ‘limitations’ section of the discussion we add “The elicitation of data collected by clinical staff was based on clear check boxes. However, the contingent actions had to be ascertained from case notes. While this exercise is not complex, we cannot exclude the possibility that some of the actions were overlooked (leading to underestimation of effects) and we did not have resources to duplicate observations. However, the finding of ceiling effects for some observations suggests that few, if any, of the contingent actions were missed”. Thank you for this observation. - Case notes selection. Again, thank you. We have clarified that they were consecutive. - Selection of prompts. We now add “The clinical service decided to implement four sets of standards. They were selected because they all had national endorsement (see below). We therefore examined these standards.” - Paper checklists were “replaced” by electronic checklists. Thank you - we should have been clearer. - High rate of adherence explained by increased use of required fields: evidence- based prompts on EHRs. We had made it clear that the systems described here were forerunners of electronic notes - first paragraph of ‘relevance’ section in the discussion. We had also discussed a more general trend under ‘causal inference.’ - We have added sample size to the abstract. We have rewritten the ‘Results’ section to include the statistical findings. We agree that the tweetable abstract was too positive and have modified it. We thank the reviewer for their attentive reading of our paper and for insightful suggestions.Competing Interests: None Close- We may have missed other literature. Our search would have included the quality literature covering antenatal care. It was done independently of a recent literature review cited in our article (Cockburn et al. 2024) and our search included all articles on prompts/action suggestions cited in this paper. - Sampling at the end of calendar years versus continuous observation. Doing so would have introduced (additional) sources of bias. Had we done so, we would have needed an adjustment period for the clinicians to get used to each new prompt system as it was introduced. More observations would have been required. - Role of reviewer judgement. This relates to the " observations” subheading under methods. As stated, we retrieved the case notes. The case notes included the paper checklists that we introduced in the earlier epoch and the electronic print outs from the later epoch. We now make that clearer. We then go on to specify that the checklists and electronic printouts were scrutinised to determine whether the data was collected. Then the case notes themselves were examined to see whether the contingent actions were followed. We have rewritten this part to make this clearer. Thank you. We then add “Thus the data collected were based on clear boxes on the paper or electronic checklist, while the contingent actions were extracted from the case notes.” Then in the ‘limitations’ section of the discussion we add “The elicitation of data collected by clinical staff was based on clear check boxes. However, the contingent actions had to be ascertained from case notes. While this exercise is not complex, we cannot exclude the possibility that some of the actions were overlooked (leading to underestimation of effects) and we did not have resources to duplicate observations. However, the finding of ceiling effects for some observations suggests that few, if any, of the contingent actions were missed”. Thank you for this observation. - Case notes selection. Again, thank you. We have clarified that they were consecutive. - Selection of prompts. We now add “The clinical service decided to implement four sets of standards. They were selected because they all had national endorsement (see below). We therefore examined these standards.” - Paper checklists were “replaced” by electronic checklists. Thank you - we should have been clearer. - High rate of adherence explained by increased use of required fields: evidence- based prompts on EHRs. We had made it clear that the systems described here were forerunners of electronic notes - first paragraph of ‘relevance’ section in the discussion. We had also discussed a more general trend under ‘causal inference.’ - We have added sample size to the abstract. We have rewritten the ‘Results’ section to include the statistical findings. We agree that the tweetable abstract was too positive and have modified it. COMMENTS ON THIS REPORT - Author Response 10 Sep 2025Richard Lilford, School of Health Sciences, University of Birmingham, Birmingham, UK10 Sep 2025Author ResponseWe thank the reviewer for their attentive reading of our paper and for insightful suggestions. - We may have missed other literature. Our search would have included the quality - We may have missed other literature. Our search would have included the quality literature covering antenatal care. It was done independently of a recent literature review cited in our article (Cockburn et al. 2024) and our search included all articles on prompts/action suggestions cited in this paper. - Sampling at the end of calendar years versus continuous observation. Doing so would have introduced (additional) sources of bias. Had we done so, we would have needed an adjustment period for the clinicians to get used to each new prompt system as it was introduced. More observations would have been required. - Role of reviewer judgement. This relates to the " observations” subheading under methods. As stated, we retrieved the case notes. The case notes included the paper checklists that we introduced in the earlier epoch and the electronic print outs from the later epoch. We now make that clearer. We then go on to specify that the checklists and electronic printouts were scrutinised to determine whether the data was collected. Then the case notes themselves were examined to see whether the contingent actions were followed. We have rewritten this part to make this clearer. Thank you. We then add “Thus the data collected were based on clear boxes on the paper or electronic checklist, while the contingent actions were extracted from the case notes.” Then in the ‘limitations’ section of the discussion we add “The elicitation of data collected by clinical staff was based on clear check boxes. However, the contingent actions had to be ascertained from case notes. While this exercise is not complex, we cannot exclude the possibility that some of the actions were overlooked (leading to underestimation of effects) and we did not have resources to duplicate observations. However, the finding of ceiling effects for some observations suggests that few, if any, of the contingent actions were missed”. Thank you for this observation. - Case notes selection. Again, thank you. We have clarified that they were consecutive. - Selection of prompts. We now add “The clinical service decided to implement four sets of standards. They were selected because they all had national endorsement (see below). We therefore examined these standards.” - Paper checklists were “replaced” by electronic checklists. Thank you - we should have been clearer. - High rate of adherence explained by increased use of required fields: evidence- based prompts on EHRs. We had made it clear that the systems described here were forerunners of electronic notes - first paragraph of ‘relevance’ section in the discussion. We had also discussed a more general trend under ‘causal inference.’ - We have added sample size to the abstract. We have rewritten the ‘Results’ section to include the statistical findings. We agree that the tweetable abstract was too positive and have modified it. We thank the reviewer for their attentive reading of our paper and for insightful suggestions.Competing Interests: None Close- We may have missed other literature. Our search would have included the quality literature covering antenatal care. It was done independently of a recent literature review cited in our article (Cockburn et al. 2024) and our search included all articles on prompts/action suggestions cited in this paper. - Sampling at the end of calendar years versus continuous observation. Doing so would have introduced (additional) sources of bias. Had we done so, we would have needed an adjustment period for the clinicians to get used to each new prompt system as it was introduced. More observations would have been required. - Role of reviewer judgement. This relates to the " observations” subheading under methods. As stated, we retrieved the case notes. The case notes included the paper checklists that we introduced in the earlier epoch and the electronic print outs from the later epoch. We now make that clearer. We then go on to specify that the checklists and electronic printouts were scrutinised to determine whether the data was collected. Then the case notes themselves were examined to see whether the contingent actions were followed. We have rewritten this part to make this clearer. Thank you. We then add “Thus the data collected were based on clear boxes on the paper or electronic checklist, while the contingent actions were extracted from the case notes.” Then in the ‘limitations’ section of the discussion we add “The elicitation of data collected by clinical staff was based on clear check boxes. However, the contingent actions had to be ascertained from case notes. While this exercise is not complex, we cannot exclude the possibility that some of the actions were overlooked (leading to underestimation of effects) and we did not have resources to duplicate observations. However, the finding of ceiling effects for some observations suggests that few, if any, of the contingent actions were missed”. Thank you for this observation. - Case notes selection. Again, thank you. We have clarified that they were consecutive. - Selection of prompts. We now add “The clinical service decided to implement four sets of standards. They were selected because they all had national endorsement (see below). We therefore examined these standards.” - Paper checklists were “replaced” by electronic checklists. Thank you - we should have been clearer. - High rate of adherence explained by increased use of required fields: evidence- based prompts on EHRs. We had made it clear that the systems described here were forerunners of electronic notes - first paragraph of ‘relevance’ section in the discussion. We had also discussed a more general trend under ‘causal inference.’ - We have added sample size to the abstract. We have rewritten the ‘Results’ section to include the statistical findings. We agree that the tweetable abstract was too positive and have modified it. Alongside their report, reviewers assign a status to the article: - Approved - Approved with reservations - Not approved | Invited Reviewers | |||| |---|---|---|---|---| | 1 | 2 | 3 | 4 | | | Version 2 (revision) 16 Jul 25 | read | read | || | Version 1 23 Apr 25 | read | read | - Steven Ondersma, Michigan State University, Flint, USA - Mxolisi W Ngwenya, University of Limpopo, Mankweng, South Africa Sign up for content alerts You are now signed up to receive this alert Alongside their report, reviewers assign a status to the article: Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit. Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list: Examples of 'Non-Financial Competing Interests' - Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper. - You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors. - You are a close professional associate of any of the authors (e.g. scientific mentor, recent student). - You work at the same institute as any of the authors. - You hope/expect to benefit (e.g. favour or employment) as a result of your submission. - You are an Editor for the journal in which the article is published. Examples of 'Financial Competing Interests' - You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements. - You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors. - You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on. Sign up for content alerts and receive a weekly or monthly email with all newly published articles Register with NIHR Open Research Already registered? Sign in close Error If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research. We'll keep you updated on any major new updates to NIHR Open Research Sign In If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password. Email us for further assistance. The email address should be the one you originally registered with F1000. Email address not valid, please try again You registered with F1000 via Google, so we cannot reset your password. To sign in, please click here. If you still need help with your Google account password, please click here. You registered with F1000 via Facebook, so we cannot reset your password. To sign in, please click here. If you still need help with your Facebook account password, please click here. Code not correct, please try again Server error, please try again. If your email address is registered with us, we will email you instructions to reset your password. If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance. Please wait...