A mixed-method study exploring experiences, perceptions, and acceptability of using a Safe Delivery mHealth Application in two district hospitals in Rwanda

A mixed-method study exploring experiences, perceptions, and acceptability of using a Safe Delivery mHealth Application in two district hospitals in Rwanda | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A mixed-method study exploring experiences, perceptions, and acceptability of using a Safe Delivery mHealth Application in two district hospitals in Rwanda Aurore Nishimwe, Daphney Nozizwe Conco, Marc Nyssen, Latifat Ibisomi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1260261/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Background: Innovative use of mobile health (mHealth) technology in timely management of childbirth complications is a promising strategy, but its evidence base is limited. The Safe Delivery mHealth Application (SDA) is one of the recent mhealth applications (loaded in smartphones) which is a clinical decision support and training tool for basic emergency obstetric and newborn care (BEmONC). This paper describes, the health providers’ experiences, perceptions, and acceptability of using the SDA, as well as the perceptions of key stakeholders. Methods: A mixed-methods approach was utilized. Quantitative methods consisted of a self-reported acceptability survey, administered to 54 nurses and midwives, including questions on their usage and perceptions of the SDA. Descriptive statistics were employed to analyze the survey data. Qualitative methods included two focus group discussions with 24 nurses and midwives, and six key informant interviews with stakeholders (maternity matrons, responsible for maternal and child health, and district hospital managers). Thematic analysis was performed and selected quotations used to illustrate themes. The study took place in two district hospitals in Rwanda. Results: Quantitative results found that 31 (57.4%) participants used the SDA four to six times per week. Many participants felt more confident (53.7%) and better at their job (40.7%) since having the SDA. Likert scale survey responses (1-5, 1 =Strongly Disagree, 5 =Strongly Agree) indicated general agreement that SDA is easy to use (Mean =4.46), is an effective decision support tool (4.63), and training tool (4.65). Qualitative results included themes on perceived usefulness; professional growth acquired through the use of the SDA; SDA, an empowering, intuitive, and user-friendly technology; desired SDA features and functions; benefits of SDA as perceived by key informants, and future use of the SDA. Conclusion: The nurses and midwives perceive the SDA as having improved their ability to manage childbirth complications. Key stakeholders also perceive the SDA as a useful tool with a reasonable cost and recommend its implementation in routine practices. This study deepens the understanding of the potential benefits of mHealth such as the SDA in low-income settings, like Rwanda. It also provides more evidence on the impact of mHealth in assuring quality BEmONC. BEmONC nurses and midwives mHealth Safe Delivery Application Rwanda Figures Figure 1 Background Globally, maternal and newborn deaths remain a public health concern despite several efforts that have been/are being made to reduce the deaths. It is estimated that 295,000 maternal deaths, 2 million stillbirths and 2.5 million early newborn deaths occur annually in the world [ 1 , 2 ]. More than 90% of these deaths happen in low- and middle-income countries, like Rwanda [ 3 ]. Most of these deaths are attributable to preventable causes such as post-partum haemorrhage (PPH), newborn asphyxia, pre-eclampsia, obstructed labor, infections, and hypertensive disorders [ 4 , 5 ]. The WHO, UNICEF, and UNFPA have identified essential interventions known as Basic Emergency Obstetric and Newborn Care (BEmONC) for prevention and treatment of the preventable causes of maternal and newborn deaths [ 6 , 7 ]. The BEmONC package includes the management of PPH, the removal of a retained placenta, the management of maternal sepsis, the management of hypertensive disorders, the management of prolonged labor, the management of complications of abortion, and neonatal resuscitation (NR) [ 6 ]. These services can be provided by nurses and midwives. There is evidence that BEmONC is a key intervention to reduce death and disability of mothers and newborns [ 7 ]. However, the available literature remains characterized by gaps in assuring the quality of BEmONC. The quality of BEmONC services is important to improving maternal and newborn health [ 8 ]. The research evidence suggests that barriers to instituting quality BEmONC are complex and are often linked to limited knowledge and skills among health care providers, lack of access and adherence to evidence-based clinical guidelines, insufficient monitoring to inform appropriate responses, and lack of some essential materials [ 9 ]. Besides, research recommends the use of mobile health (mHealth) to improve health systems, as a solution that could potentially support in addressing some of the health systems challenges and improve service delivery [ 10 , 11 ]. The safe delivery mHealth application (SDA) is a novel mHealth application which is a decision support and training tool for BEmONC that was developed by the Maternity Foundation [ 12 ]. The main purpose of the SDA is to make a healthcare provider better prepared to prevent, detect, and manage life threatening complications to mothers and newborn via offering evidence-based BEmONC [ 12 ]. The SDA contains animated videos with instructions on management of PPH and NR. The SDA offers the opportunity to watch the entire instructional animated video (8–15 min long) as well as choosing to see a specific procedural step of each video (1-3 min long). Besides, the SDA contains a catalogue of action cards with essential recommendations to support clinical decision making in an emergency situation when the user does not have time to review a video. The SDA also include a comprehensive drug list which contains useful information on the various essential drugs needed to perform PPH management and NR. Further, the SDA describes certain essential procedures step-by-step and required equipments. In addition, the SDA has a self-explanatory learning platform ‘My Learning’ that provides learning and assessment strategies based on key elements within the clinical content found in the SDA. To stimulate use, the SDA sends weekly notifications with quiz questions and a direct link to the films where the information to answer the question is found. The clinical content of the SDA was developed following international WHO guidelines but focusing on key essential lifesaving interventions of BEmONC [ 13 ]. The SDA can be downloaded free of charge for iPhone at: https://itunes.apple.com/dk/app/safe-delivery/id985603707?mt=8 and for Android at: https://play.google.com/store/apps/details?id=dk.maternity.safedelivery&hl=en The present paper, was derived from a broader study about the implementation of the SDA in handling two elements of the BEmONC: PPH and NR in two district hospitals in Rwanda. Such focus was informed by evidence that indicated PPH and newborn asphyxia as the priority challenges in Rwanda [ 14 , 15 ]. The SDA was piloted in two district hospitals in Rwanda. This pilot implementation process was divided into three phases (pre-intervention, intervention, and post-intervention). The first phase (pre-intervention) involved the presentation of the research to different stakeholders and the baseline study (surveys, records review, and focus group discussions). The second phase (intervention) included the capacity building in the usage of the SDA among nurses and midwives; the SDA provision and piloting; and the SDA implementation for a period of six months. The third phase (post-intervention) included the implementation evaluation and encompassed the endline study (surveys, records reviews, focus group discussions, and key informant interviews). More details are provided in the protocol paper for this research [ 16 ]. The present paper explores the end-users’ experiences, perceptions and acceptability of the SDA and key stakeholders’ perspectives. Methods Study design This paper describes part of a larger study about the implementation of the SDA in BEmONC services in two district hospitals in Rwanda. The larger study methodology and the evaluation of initial outcomes have been reported elsewhere [ 16 – 18 ]. The part of the study presented in this paper, used mixed research methods to explore end-users’ experiences, perceptions, and acceptability, and explore the key stakeholders’ perceptions of the SDA. Study setting The study took place in two district hospitals in Rwanda: Masaka hospital in Kigali, an urban province; and Nyamata hospital located in the eastern rural province [ 19 ].The two hospitals were selected out of 12 district hospitals in the two provinces because both had a high number of deliveries per year [ 20 ] and had been offering BEmONC services for more than five years. Study Participants The study included 60 participants, the nurses and midwives (n=54) and stakeholders (n=6). Nurses and midwives who deal with childbirth in the maternity departments of the two study hospitals were invited to participate in the study and the inclusion criteria were as follows: having work experience of at least six months in obstetric care; being full-time employed in the selected hospitals; and willing to participate in the study. The selected stakeholders (key informants in the management position at the district hospital level) included maternity matrons (n=2), managers responsible for maternal and child health (n=2), and district hospital directors (n=2). Data collection and analysis Quantitative data The quantitative part of the research entailed asking all nurses and midwives who were part of implementation of the SDA to complete a self-administered questionnaire (Additional file 1) evaluating usage of the SDA, as well as their perception and acceptability. The survey was conducted at endpoint once users had used the SDA for six months. Responses were analysed descriptively using Stata version 16 (StataCorp LLC). Qualitative data Two methods were used to collect qualitative data - focus group discussions (FGDs) and key informant interviews (KIIs). Focus groups discussions explored in depth the participants’ experiences and views on the functionality of the SDA. Two FGDs were conducted, one in each of the study hospitals. In each hospital, 12 participants were recruited and total number of FGD participants is 24. The participants included in the FGDs were selected based on their interest and availability. We sought perspective diversity in terms of age, gender, and experience [ 21 ] and these criteria were considered in both groups. The Key Informant Interviews were conducted with six participants (three from each hospital), who were selected because of their leadership/management portfolios in the participating hospitals. The purpose was to get their views on the feasibility of adopting the SDA for wider implementation with the focus on its costs, possible scalability and sustainability. The interviews with key stakeholders started with an introduction of the SDA to the key informant including an explanation of the features and modules of the SDA, and a demonstration of how the SDA functions using a smartphone. After the introduction of the SDA, the key informant was invited to ask questions and comment on the SDA. Once their concerns were addressed, the researcher presented the estimated total cost that was used for piloting the SDA in two district hospitals (Masaka and Nyamata) in Rwanda. The estimated total cost was 10.076.410 Rwf ( $ 10000) including the technology cost =4.504.000 Rwf ( $ 4470.56), the training cost =2.083.410 Rwf ( $ 2067.94) and the intervention cost =3.489.000 Rwf ( $ 3463.09). Then, participants were asked to provide their perceptions of the SDA and their opinions on its cost, scalability, and sustainability. The interview guides (Additional file 2) were developed in English, translated to Kinyarwanda, and translated back to English in a blinded manner to ensure accuracy and equivalence [ 22 ]. To ensure the quality and adequacy of the interview guides, they were pretested with two participants at a different district hospital. The interviews were conducted face to face in March 2020 and were held in private rooms at both Masaka and Nyamata district hospitals to ensure confidentiality. Interviews were conducted in the language of the participants’ preference, which was Kinyarwanda. The first author conducted the interviews with assistance of an experienced research assistant who assisted in taking notes and supervising the tape recording. FGDs lasted between 60 and 90 minutes and KII interviews between 30 and 45 minutes. All FGDs and KIIs were digitally recorded and transcribed in Kinyarwanda by two research assistants. Transcriptions were done following the true verbatim method to accurately capture meanings, perceptions, and context [ 23 ]. Then, transcripts were translated from Kinyarwanda to English by a professional translator. In order to ensure that everything was translated as correctly as possible, all transcriptions were compared with the audio files and field notes. The data were analyzed using thematic analysis with a hybrid approach (inductive and deductive) [ 22 ]. Two authors (AN and DNC) first read all transcripts independently, then jointly developed a codebook. Disagreements were resolved by discussion. The final list of codes was applied to all transcripts using Nvivo 11 Plus software. Coded output was then read, merged into major themes, and illustrative quotations were identified. The consolidated criteria for reporting qualitative research [ 24 ] guided reporting for the qualitative data (Additional file 3). Findings from both quantitative and qualitative data were integrated during the data-interpretation stage. The goal was to obtain different but complementary data that validate the overall results about the SDA acceptability. Ethical considerations Ethical approvals were obtained from the Human Research Ethics Committee of the University of the Witwatersrand (M190258) and the University of Rwanda, College of Medicine and Health Sciences’ Institutional Review Board (No. 377/CMHS IRB/2018). All participants gave written informed consent and consents for recording the interviews prior to their participation. Participants were assured that their identities would remain anonymous in all reporting of the study and that their personal information would be kept confidential. All methods in this study were carried out in accordance with relevant guidelines and regulations in the Ethical Declarations. Results Characteristics of quantitative study participants A total of 54 nurses and midwives completed the acceptability survey. The majority of participants were midwives (n=33). More than half were from Masaka district hospital (n = 30, 56%), the majority were female (n = 33, 61%), their average age was 33 years (SD = 7.1). The level of education was predominantly the advanced diploma (A1) in midwifery with 27 of 54 (50%) participants having advanced diploma (A1 level) in midwifery. The least represented level of education was the secondary school level (A2) in nursing with only one participant with A2 level in nursing. A detailed description of the survey respondents is provided in Table 1 . Table 1 Demographic characteristics of the acceptability survey respondents ( N=54) n (%) Hospital Affiliation Masaka District Hospital 30 (55.6) Nyamata District Hospital 24 (44.4) Sex Male 21 (38.9) Female 33 (61.1) Education Level Midwife A0 6 (11.1) Midwife A1 27 (50) Nurse A0 5 (9.3) Nurse A1 15 (27.8) Nurse A2 1 (1.8) Age, years, Mean (SD) 33 (7.12) Abbreviations: %: Weighted percent, A0 = bachelor’s degree, A1 = Advanced Diploma, A2 = Secondary school level. The nurses’ and midwives’ experiences of the SDA during the study implementation. The utilization of the SDA by nurses and midwives The majority of participants (n=31) reported using the SDA four to six times per week. Features of the SDA that were most consulted include; the action cards (n=15), my learning (n=14) and the drug list (n=10). A half of the participants used the SDA to revise their knowledge. Other participants used the SDA in emergency situations (n=14), in a normal work situation (n=5) and to discuss with co-workers (n=8). Nurses and midwives predominantly accessed the SDA on both their own smartphones and on the smartphones provided by the researcher (n=28). A large number of participants (n=35) often used the SDA with a co-worker. Participants reported having access to most of the equipments and drugs seen in the SDA (n=40). More details on the SDA use and evaluation by end-users are shown in Table 2 . Table 2 SDA access and evaluation by nurses and midwives ( N=54) n (%) Frequency of SDA use in a week I used it almost every day 15 (27.8) I used it 4-6 times 31 (57.4) I used it 1-3 times 8 (14.8) I didn’t use it at all 0 (0) Features of the SDA most consulted Videos 9 (16.7) Action cards 15 (27.8) Drug list 10 (18.5) Procedures 6 (11.1) My Learning 14 (25.9) Most often quoted reasons for using the SDA To revise my knowledge 27 (50) During an emergency 14 (25.9) In a normal work situation 5 (9.3) To discuss with co-workers 8 (14.8) The device often used to access the SDA Own phone 7 (13) The phone provided by the researcher 19 (35.2) Both 28 (51.8) With who do you often use the SDA? By myself 19 (35.2) With a co-worker 35 (64.8) Access to the equipment and/or drugs seen in the SDA. All 14 (25.9) Most 40 (74.1) None 0 (0) Abbreviations: Wt.% : Weighted percent; SDA: Safe Delivery mHealth Application When rating their experiences of using the SDA particularly, half of respondents (n=27) indicated that it was very easy to the SDA, however one participant reported that it was very difficult to use a smartphone. SDA features that were most appreciated included: the action cards (33%), the drug list (22%), and my learning (19%). They also reported feeling more confident (53.7%) and becoming better (40.7%) at doing their jobs at work. Few participants (n=5) reported what they liked least about the SDA, three participants stated that they did not understand instructions, and the other two said that it was difficult to navigate the smartphone. More details on the experiences with the SDA and its added value are displayed in Table 3 . Table 3 Experiences with the SDA and its added value ( N=54) n (%) Experience with the smartphone and the SDA Very easy 27 (50) Somewhat easy 23 (42.6) Somewhat difficult 3 (5.6) Very difficult 1 (1.8) Features of the SDA most appreciated Action cards 18 (33.3) Videos 10 (18.5) Drug list 5 (9.3) Procedures 12 (22.2) My Learning 9 (16.7) Added value from the SDA I have become better at doing my job. 22 (40.7) I feel more confident at work. 29 (53.7) Fewer mortalities/fatal cases 3 (5.6) I did not notice any change. 0 (0) What do you like least about the SDA? I do not understand the instructions. 3 (5.6) I think it is difficult to navigate the smartphone. 2 (3.7) The app does not add to my existing knowledge. 0 (0) None applicable. 49 (90.7) Abbreviations: Wt.% : Weighted percent; SDA: Safe Delivery mHealth Application Acceptability of the SDA as rated by nurses and midwives. Nurses and midwives were asked to rate their acceptance of the SDA in a Likert scale. They were asked whether they "strongly disagree", "disagree", were "undecided", "agree", or "strongly agree" to a series of statements relating to the SDA acceptability. Responses were assigned point values as follows; strongly disagree =1, disagree =2, undecided =3, agree =4, strongly agree =5. Likert scale survey results showed that the SDA was: well understood (Mean=4.00); easy (4.46) and comfortable (4.31) to use; an effective decision support (4.63) and training (4.65) tool; and useful in improving efficiency in BEmONC (4.59). Most participants reported that they intended to continue using the SDA in daily practices (4.68). On the other hand, a lower rating was given to using the SDA when seeing notifications (2.48), implying that most participants did not only use the SDA when they saw notifications. The notifications are push messages that appear occasionally on the phones of the SDA users to remind them to revisit the SDA. The messages mainly guide the users to interesting parts that they may want to explore in the SDA. Overall participants reported good acceptability of the SDA. Likert scale responses are highlighted in Table 4 . Table 4 Likert scale survey responses of nurses and midwives about SDA acceptance Statement N Mean (SD) [Range] I have a good understanding of what the SDA is 54 4.00 (0.71) [ 3 – 5 ] The SDA is easy to use 54 4.46 (0.50) [ 4 – 5 ] I am comfortable using the SDA 54 4.31 (0.57) [ 4 – 5 ] The SDA is an effective decision support tool for BEmONC 54 4.63 (0.69) [ 4 – 5 ] The SDA is an effective training tool for BEmONC 54 4.65 (0.48) [ 4 – 5 ] The SDA is useful in improving efficiency in BEmONC 54 4.59 (0.56) [ 3 – 5 ] I intend to continue using the SDA in my daily practices 54 4.68 (0.50) [ 4 – 5 ] I use the SDA when I see notifications 54 2.48 (0.40) [ 1 – 3 ] Note: Likert scale response options were as follows: 1Strongly Disagree, 2Disagree, 3Neutral, 4Agree, and 5Strongly Agree Perceptions about the usefulness and acceptability of the SDA The results of the interviews with end-users of the SDA emerged into four main themes: (1) the perceived usefulness of the SDA; (2) professional growth acquired through the use of the SDA; (3) SDA, an empowering, intuitive and user-friendly technology; and (4) the desired SDA features and functions. The results of the interviews with key stakeholders revealed two main themes (1) benefits of the SDA as perceived by the key informants, and (2) future use of the SDA. These identified themes are shown in Figure 1 . More details on the main themes and sub-themes are presented in the discussion of results, and are illustrated by verbatim quotations from the two FGDs and six KIIs. Interviews with end-users of the SDA Perceived usefulness of the SDA Participants described the SDA as a useful, intuitive, empowering and practical tool that supported their daily practices. They described how it provided important information and guidelines on management of obstetric emergencies, and thus, increases their knowledge and skills on how to manage normal childbirth as well as birth complications. “This application came as a golden opportunity for us …normally, it is very hard to go to a library all the time to read; but the library, I am hereby talking about is that SDA application we have received…It provides very useful information and guidelines that upgrade our knowledge and skills in delivery care” Midwife, FGD1. In addition, nurses and midwives reported using the SDA both as a decision support tool while managing birth complications including PPH or NR, and as a debriefing tool for reflections on performed care to identify if essential procedures had been missed. “When you have an emergency case to manage, you don’t have difficulties figuring out how to handle it…all the steps are clearly outlined in this application and so well summarized that you immediately find what you need whenever you refer to it…The SDA support our clinical decisions in emergency situations…Sometimes, when I go home from a night duty, on the bus, I consult the SDA to see if I have performed well during the night” Nurse, FGD2. Participants also described the SDA as a practical and easy to use tool that helps strengthen their theoretical knowledge. They articulated how the SDA helped them remember the management of rare clinical cases. Procedures for those rare cases are easy to forget as they rarely had to apply the theory learnt from school into practice. “Before SDA, you could sometimes start wondering what management was appropriate for a case which is not so frequent after you had forgotten…Now, we refer to the SDA when there is a doubt about anything… With this new application, we have become all fresh and we can easily remember the essential procedures step by step as per international standards” Midwife, FGD1. Other participants expressed appreciation of the SDA as a training tool, highlighting the ingenuity of the visuals when dealing with life threatening complications of mothers and their newborns. They considered the SDA to be a kind of simulated clinical training that continuously enhances their knowledge and practical skills, contributing to their professional growth. They also reported improved performance when managing childbirth complications. Such improvement was attributed to continuous learning, facilitated by the SDA and the fact that learning could take place anywhere. This application has been an important learning tool…I consider it as a training manual with all clinical guidelines accompanied by video materials… we learn a lot from the SDA in addition to what we learned at school…You can learn from this application from anywhere: at the hospital, on the bus, at home or anywhere. So, having your telephone means being able to learn more anywhere.” Midwife, FGD2. Then again, participants revealed that they used to consult other internet sources for learning like youtube. They found such sources not so easy to use, as it was hard to get concise updated information and guidance in a timely manner, the way the SDA does. “We used to visit youtube to learn something about the services we deliver, but most materials in there are not so clear as in this app. When you consult the SDA, you find that everything is easy to understand. You quickly get information on which medication is good for which case…” Midwife, FGD1. “Some of us are mentors in health centers and we have been consulting this application while preparing training materials” Midwife, FGD1. Some found the SDA notifications useful as reminders for them to continue learning. The notifications that appeared intermittently on their phones, reminded them to revisit the SDA that also pointed them to other interesting parts of the SDA that they may want to explore. “We have posters displayed in maternity telling us what to always bear in mind. Some of the information on them is “remember to close the door to keep the newborn warm” … I compare this with the SDA notifications that I frequently get on my smartphone…these are very nice reminders that I have to keep learning” Midwife, FGD2. “I would come back to what a colleague said about SDA notifications. This is a way of alerting you and I like that… There exists a wrong belief that nurses don’t like reading. Maybe, it is true but this application is not too demanding. It doesn’t consume your time as if you go to a library… This app is a good source and we need to regularly use it” Nurse, FGD1. Professional growth acquired through the use of the SDA The perceived usefulness of the SDA described by the participants was associated with substantial improvements in their childbirth care practices. Nurses and midwives narrated experiencing an increase in their knowledge, skills, and confidence after using the SDA and how such professional growth has been reflected in the quality service they offer. “The reason number one I like this app is that it upgrades my knowledge and skills, and makes me confident than before…it is a kind of continuous professional development…It is so good in a way that if you fail to understand the text when reading, you have an option to search for more explanation using videos. It is really very useful…” Nurse, FGD2. “The biggest difference for me when I compare now and the past experience is service delivery which has so positively improved. The special aspect of the application is that the content therein has everything we theoretically studied at school” Midwife, FGD2. The study participants indicated that the SDA has helped them to recollect and revise their skills, which have made them more capable at managing childbirth complications in accordance with the recommended essential procedures. One of the significant changes expressed by participants was that they had learned the lifesaving steps of how to stop a bleeding by applying bimanual compression. “This application has been helping us refresh our memories and equipping us with techniques that we use to handle obstetric emergencies.… My colleagues have talked about bimanual compression…You may even not know anything about it … Then, when you try to familiarize with it on the SDA, you get to know about it and perform it correctly” Midwife, FGD1. Other participants explained how the SDA supported them during practice. They described the before SDA childbirth care experience and compared it with after SDA experience by highlighting the specific instances in which the SDA supported them. Previously, in situations where nurses and midwives were struggling to assist a mother or a newborn, they would immediately call upon a medical doctor to come and help. And, sometimes the doctor assigned to maternity department was busy with caesarean surgery which would delay his assistance to attend to the complicated case. But, now with the support of the SDA, nurses and midwives have been able to provide essential assistance to mothers and newborns while waiting for the doctors. “I agree that BEmONC services were also delivered before we started using SDA but the situation was not the same. For example, when there was a complication with a patient, the only thing we would do was to ask a colleague or to call a medical doctor. This was time consuming and would delay service delivery and could complicate the situation even cause maternal deaths... However, when there is a problem nowadays, you just take your phone, go to the application and get a quick solution. As a result, our patients get quick and more efficient service” Midwife, FGD2. Both nurses and midwives revealed that they used to fear occurrence of birth complications, because of the limited skills and experience on how to manage such events. They noted that the SDA had reduced this fear and had led them to have more confidence in their work. “In using this app, you are always aware of what might happen in a given case. So, when you have a real case, you don’t panic much…For instance, when it comes to PPH management, with the SDA you can easily figure out the steps to follow ” Nurse, FGD1. “This application can help you when you are stressed. You may have had a lot of cases to manage in a day and end up in confusion somewhere. When this happens, you take your phone and see what should be done to handle a particular case with confidence.” Midwife, FGD2. Some participants described how solving problems using the SDA motivated and enabled them to be more efficient in providing care at work. “when you have a problem and you find a way to solve it, that is self-explanatory; you feel motivated automatically. As we get support from the application, our job becomes easier and more efficient” Midwife, FGD1. Further, nurses and midwives explained that they used the SDA either at home to update knowledge or while they were at work, either before a delivery when the woman is in labor or when complication arises in the course of a delivery. Some participants described using the SDA to guide co-workers when they work in pairs in order to follow the necessary steps during a delivery. In emergency situations, the SDA was often used as a decision support tool to see how to manage e.g. PPH and neonatal resuscitation or as a reminder of drug dosages and administration. Participants also acknowledged having access to most of the equipments and drugs seen in the SDA. “We have most of the materials we need to perform well. But, the SDA has been there to support us in upgrading our knowledge wherever we are - at home, before assisting a delivery or during an emergency…You may forget something like medication dosage, even though you learned it at school… but since we got this app, it helps us to remember the dosages in emergencies, we use it as a job aid in a simple way” Nurse, FGD2. SDA an empowering, intuitive and user friendly technology The study participants also voiced that the SDA is easy to use, intuitive, user friendly, and enabled them to provide first aid to patients before calling upon doctors. This application is special in that it is so focused and shows you clear sequences…For example, you can easily find what specifically refers to steps for removing a retained placenta when you have such a case. We really like it because it is easy to use” Midwife, FGD2. “I would say something additional. With this application, we have learnt how at least one can provide first aid for a mother or do resuscitation for a newborn… That is so important because you don’t need to call upon a doctor for everything” Midwife, FGD2. Likewise, participants indicated that the SDA has helped them to get a common understanding with medical doctors about managing birth complications. They explained on how consulting the SDA helped them to have informed recommendations and they were happy to contribute with confidence to care plans and sometimes supporting doctors. “I would say that when I have a special case, I first consult the application, see how I can myself manage it before I talk to a medical doctor. Even when I have to talk to the doctor, at least I provide an informed suggestion and contribute to further treatment steps. We are supporting our doctors much more.” Midwife, FGD1. Then again, nurses and midwives described how the SDA has helped them to get updated information and to have a common understanding among themselves during practice. “I can easily remember what I studied in school and when I tell my colleague X, he sometimes tells me a different story. This is due to the fact that he studied so recently, let’s say three years ago, while I studied the similar subject 12 years ago…There is a need for all of us to have the same updates. This application is a solution because it keeps getting updated” Nurse, FGD1. When asked if they will continue using the SDA in their future practice. Both nurses and midwives stated that due to the SDA benefits, it is their own advantage to continue using it in their routine practice for efficient provision of care. “I would say that we have been lucky to be the champions of this application. I agree that we have a lot of work to do in maternity but we can’t forget that we also need updates in our career…We will always need to refer to the app in our routine practices and find some time to learn new things from the SDA. We shall keep using it and it is for our own interest” Midwife, FGD2. “This application is user friendly, affordable, covers a lot of areas, and does not take your internet megabytes when using… It is not like a book or a computer that would be heavy for you to carry…It is in a mobile phone that you always carry in your hands…You can check up anything, anywhere and anytime.… Unless something unexpected happens, I don’t see how people can stop using the SDA” Midwife, FGD1. When asked how they perceive the SDA sustainability in district hospitals in Rwanda, all nurses and midwives indicated that the SDA will be likely sustained. “This application will be maintained because it is an important tool for us to deliver good service to the patients. When you use it properly, you deal with various cases with confidence and you perform your duties with success” Midwife, FGD1. “I am also confident that SDA will be sustained for two main reasons. The first reason is that it is easy to use and provides text content accompanied by video materials. Secondly, SDA is so relevant in that it is equipped with modules that reflect our day to day clinical practices” Midwife, FGD2. Furthermore, participants requested the SDA scalability to the whole country of Rwanda in consideration of the present country’s agenda to support the adoption of technology in healthcare service delivery. “Personally, I would say that this is the most appealing application on my phone…When we are discussing with colleagues on social media (WhatsApp), there may be some questions raised in relation to our career. When there is a confusion, I immediately take my phone, get reliable information from the SDA and in a fast way I share with them… Is it possible to implement this app in all hospitals in Rwanda? …Since most people have smartphones, this should not be a problem, also our country is currently promoting application of technology in healthcare. I think this would make a lot of difference” Midwife, FGD1. “I have to tell you that this is a very good application, those using the application have already started seeing the benefits and they gain a lot from it. I wish it could have been shared with other hospitals too. It should be scaled up to national level. I wish to see all midwives in Rwanda using this app… It is easy to use, once you have it installed, you don’t need much guidance on how to use it, then we will be able to save more maternal and newborn lives” Midwife, FGD2. “I remember I was in Musanze some time back and when I talked about this application to my friends, they were surprised. They started asking me how they can access it too. I just showed them and they became very interested. I am inviting everyone to have this spirit of sharing so that it reaches more people” Nurse, FGD1. Other participants recommended that the SDA should be part of learning materials used at University. “Ever since I started using the SDA, there has been no single case that I have failed to get information about in this application. I think this application should be part of tools that midwifery students use at University before they graduate. If this is done, using it at workplace would be easier. I am confident that if you once saw something at school and you need to use it in your job, it becomes easily implementable” Midwife, FGD1. Desired SDA features and functions The study participants were asked if there might be any other information or functionalities they would want to be added to the SDA for improving the application. Most participants stated that the SDA is useful as it is. However, some participants suggested features and functions they wish to be added to the SDA. Firstly, participants requested if the SDA could have a version for desktop and laptop computers so that it could be installed in computers available at their respective hospitals or if they could access the SDA using a desktop/laptop browser. “I wonder if this application can’t be installed in our office computers because sometimes one’s phone might not be available maybe because it is not charged, it is stolen or broken or the owner simply doesn’t have it with him/her. Having the application available in various devices would be much better and accessing it might be easier for everyone” Midwife, FGD1. Other participants expressed their need for locally appropriate clinical decision support tools and emphasized their wish to have a version of the SDA available in the local language ‘Kinyarwanda’. “I would like to add that it would be great to have this application in Kinyarwanda like I saw it having other language versions including Swahili, Hindi…. I guess this is a process, but we will be grateful if this is expedited. That will really be helpful. Thanks” Nurse, FGD2. In addition, nurses and midwives requested additional animated videos to be added to the SDA. These include; videos on antenatal care in line with the new WHO antenatal care models with a minimum of eight contacts as well as guidance on the identification of risk factors for birth complications before delivery. ‘‘I think this application can be very helpful if they add some videos on antenatal care or how one could identify a woman in labor who would probably get a complication” Midwife, FGD2. Also important was the suggestion that the SDA application could have a discussion forum window or platform for the users across different hospitals and countries to share experiences about managing obstetric emergencies. “I wish that this application can have a kind of worldwide discussion forum where we can put comments and share experiences and views. For example, a midwife from X hospital in Rwanda can discuss and share her experience with midwives in other hospitals across multiple countries…Sharing like this can upgrade levels of knowledge for everyone. I think this would be very beneficial” Midwife, FGD1. Interviews with key stakeholders The analysis revealed two main themes: (1) benefits of the SDA as perceived by the key informants, and (2) future use of the SDA. These identified themes are shown in Figure 1 . The details on the main themes and sub-themes are presented, and are illustrated by quotations from the six KIIs. Benefits of the SDA as perceived by the key informants The key informants described the SDA as a useful tool and recommended its implementation in routine practices, specifically for the purpose of continuous learning without interrupting health services. Such benefit was depicted as fitting for the work environment in the context of Rwanda where numbers of healthcare providers were insufficient. Clinical staff require periodic training to stay on track with the knowledge and skills necessary for their practice. For their annual practice licence renewal, they need evidence of continuous professional development in addition to corresponding academic credits. SDA was therefore seen as a valuable solution as it offers on-the-job learning opportunities, assessment strategies, and possibilities of getting certificates. “This application is useful as it allows on-job continuous learning. We have been facing challenges for getting continuous learning opportunities for all staffs, now the SDA comes with this advantage. I recommend that it should be incorporated in routine practices” Respondent, KII 3. While the SDA was recommended for staff, because it captures the essential topics for BEmONC, key informants felt that this should in addition to the other sources of information including formal books and Ministry of Health guidelines that gives a broader view of obstetric care. “The SDA is good particularly for BEmONC but the SDA alone cannot solve all maternity care problems. Our staff need to use this application together with other sources of reliable information like the guidelines from the Ministry of Health” Respondent, KII 1. Another benefit highlighted was that the SDA is applicable in clinical practices by providing guidance on essential procedures that are needed to save maternal and newborn lives. “…We need safety in all we do, so this application is welcomed. I was happy to note that it provides concise information that will help nurses and midwives to take good decisions when dealing with childbirth complications” Respondent, KII 6. Future use of the SDA When KII participants were asked their opinions about the SDA cost, scalability, and sustainability, opinions on the future use of the SDA in Rwanda were apparent, along with the aspect of technology application in healthcare and its support in continuous learning. Participants perceived the estimated cost of piloting the SDA as affordable compared to physical training opportunities or online learning resources. They recommended the SDA implementation in consideration of its benefits, particularly the mLearning opportunity. “The cost is reasonable compared to other online learning tools…Also physical training has a double cost of time and resources. This application can train many people at the same time depending on individuals’ needs…I think this mLearning approach should be promoted in Rwanda as the country is looking to making healthcare services electronic-based…This application will be one of the solutions” Respondent, KII 5. One participant was concerned about the technology cost. But, he indicated that this cost could be alleviated by the fact that many healthcare providers own a smartphone. He suggested that they could use their own smartphone for future use of the SDA. “The technology cost is a bit high but as many people have smartphones and access to good internet, this might not be a problem. They could download the SDA on their own smartphones” Respondent, KII 6. In addition, participants expressed their wish to see the SDA implemented at a large scale, in all hospitals in Rwanda. However, they recommended that the SDA scalability needs to be preceded by a proper assessment and approvals by all required bodies. “Technology is in everything we do in healthcare and this application is useful…For future implementation, you need more sensitization and to involve all key stakeholders like MOH (Ministry of Health), RAM (Rwanda association of midwives), association of gynecologists and pediatricians…” Respondent, KII 2. “I think after in-depth assessment and approval by MOH (Ministry of Health), the whole country (all hospitals) could use this application and it could help us to promote continuous learning while at work” Respondent, KII 3. Another participant highlighted the need for possible monitoring and follow-up programs to evaluate if the acquired knowledge and skills are translated into real clinical practices. He also suggested that the monitoring programs could be connected with the existing annual performance contracts and evaluation. “…Some mechanisms of monitoring could be put in place to assess if what has been learnt in the SDA is being practiced…Why not link this monitoring with imihigo (performance contracts)? People should think about it” Respondent, KII 4. Then again, participants indicated that the SDA could benefit a large variety of healthcare cadres like medical doctors. “…Not only nurses and midwives would benefit from the application, medical doctors can use it …Everyone needs to know how it works and its importance. Respondent, KII 2. Moreover, KII participants perceived the SDA as an easy to use tool with well explained topics. They indicated that the SDA could be sustained because it supports healthcare providers to manage childbirth complications. They also acknowledged the fact that the SDA provides useful information needed in routine clinical practices. “I think this application will be sustained…When you see the topics and how they are explained…This information is what our staff need every day for managing birth complications” Respondent, KII 1. Furthermore, participants suggested additional topics other than topics of BEmONC that could be added to the SDA to make it more useful and sustainable. “We have seen some additional topic like COVID 19 and infection prevention in this application. Other topics that are generally relevant for healthcare providers could be added…I am thinking of topics like: medical bio safety and bio security, patient centered care and quality improvement, ethics in clinical practice…all these topics could be tied to maternity care” Respondent, KII 3. Discussion The present study findings indicate that the SDA was acceptable to both the end-users and the key informants. End-users, nurses and midwives, used the SDA frequently, and viewed it as a useful tool that contributes to their professional growth and improve their practice. They frequently used it for education to revise and improve knowledge and skills, and as a decision support tool when immediate guidance was required to manage birth-related complications. Users felt that the SDA has improved their confidence and ability to manage childbirth complications, which they were previously insecure about. These findings are supported by a pre-post intervention study conducted with the same population, which found that the SDA in Rwanda led to a significant increase in nurses and midwives’ knowledge and skills scores for PPH and NR management six months after SDA introduction [18]. Other researchers similarly indicated that the SDA, when used as a training tool, has shown significant increases in knowledge and skills of different healthcare professionals [25,26]. The perceived usefulness of the SDA seems to be related to its functionalities when applied to the clinical context of low resource settings [12]. This study emphasized the fact that the SDA offers continuous training opportunity without interrupting health services which is very beneficial due to the shortage of staffs. These findings mirror those of other studies that mHealth applications are increasingly being used in LMICs to improve health professionals’ access to evidenced and up to date information and training. In maternal and newborn healthcare services, mHealth application are being used as both training tools [27] and decision support tools [28]. The adoption of these applications in resources poor settings depend on their functions that need to be tailored to contexts of interest [29]. Other studies have similarly shown that mLearning are potential solutions to support continuous learning among health professionals in LMICs [30,31]. Also, in the current study, the SDA was reported to be practical by helping nurses and midwives translate theoretical knowledge into practice with the support of visual instructions in the animated videos. Then again, with mLearning, the user has the opportunity to study/read multiple times and master the subject. A study in Kenya compared hands-on and video training for PPH management and reported that video training appeared to be as effective as hands-on training [32]. Thus, mHealth applications providing visual instructions via animated videos such as the SDA are potential solutions to improve knowledge and skills of nurses and midwives. In addition, nurses and midwives had a positive experience with the SDA, they felt confident and better at their job when using the SDA, and reported it as an easy to use, intuitive and empowering tool. Participants were motivated to use the SDA in providing better services and suggested additional features and functions. These findings are in agreement with other studies which reported that the use of mHealth application is perceived as an opportunity for improving the quality of healthcare services with an effect on staff confidence [33], motivation [34], and self-efficacy [35]. Likewise, a realistic review about mHealth and the performance of maternal health care workers in low- and middle-income countries, revealed that health workers can be empowered to adopt and utilize mHealth in contexts where it is aligned to their needs, workload, training, and skills [36]. The review also noted that mHealth can empower health workers with skills and confidence when it is perceived as useful and easy to use [36]. Consequently, the SDA like other mHealth applications is considered an empowering tool that has a positive impact on healthcare delivery processes. Moreover, in this study, the SDA was highly acceptable by nurses and midwives who admitted to continue using the SDA in their daily clinical practices. This level of user acceptance combined with the documented impact of the SDA is interesting considering the existing gap of ensuring access and adherence to BEmONC evidence-based clinical guidelines in low- and middle-income countries. Several studies have found a low adherence to evidence-based standards for maternal and newborn care particularly in low resource settings [37,38]. Other studies have suggested the use of mHealth to improve the access and adherence to evidence-based guidelines [28,39,40]. Therefore, mHealth tools such as the SDA, which ensures access to the most recent BEmONC evidence-based clinical guidelines, and is demonstrably well accepted by its end-users, may help to improve the adoption of guidelines. The advantages of the SDA are that it is self-explanatory, available in many languages, is open source and free for download, and, once installed on the smartphone, does not require internet to function. Furthermore, the SDA was appreciated by key informants in the management position at the district hospital level. On the other hand, those key stakeholders acknowledged the benefits of the SDA and were willing to advocate for its scalability for future use in Rwanda. They noted that the cost of piloting the SDA in two district hospitals was reasonable and predicted the possible sustainability of the SDA. However, they recommended a large scale research on the SDA and involvement of the required bodies. Likewise, Eze et al [41] used a stakeholder perspective to analyze existing research on the mHealth process in developing countries, and reported that interactions between system developers and stakeholder groups are important for sustainability of mHealth [41]. Another research documented that in order to realize the potential benefits of mHealth, it is important to identify the relevant stakeholders and how they might be affected [42]. Other studies similarly found that the complexity of introducing mHealth into health care calls for strategies encouraging collaboration between multiple stakeholders to enhance successful implementation [43–46]. The implications of this study are that the SDA was accepted by both end-users and key stakeholders due to its functions tailored to the Rwandan context. Among the beneficial functionalities of the SDA, there is the capacity of the SDA to function as a training tool and a decision support tool. This increases the confidence and the ability of nurses and midwives in providing quality care during obstetric and neonatal emergencies as well as improved routine obstetric and newborn care. The acceptability and the positive perceptions of the SDA in two district hospitals reflect a possible adoption of the SDA even on a large scale in Rwanda like in other low-resources settings. Evolving efforts for quality maternal and newborn care in Rwanda and similar contexts should consider the integration of SDA as an approach for training and as a clinical decision support tool for BEmONC in order to reduce maternal and newborn mortality. Future work could consider large scale studies of the SDA with maternal and newborn outcomes as primary outcomes. Strength and limitations of the study The study is limited by the small sample size, which reflects the fact that the SDA was implemented only in two district hospitals due to funding limitations. Therefore, the findings of the current study cannot be generalized. Nevertheless, the evidence that the SDA was acceptable and perceived useful by both end-users and key stakeholders at the piloting settings sets the stage for expanding coverage and further testing of the SDA intervention to other hospitals in Rwanda. Additionally, the accuracy of our findings is increased by supporting the acceptability survey with focus group discussions to provide context to and clarify reasons behind some participant responses. Conclusion The nurses and midwives perceived the SDA as an easy to use, useful, intuitive and empowering tool, which helps them update and revise their knowledge and skills, improves their confidence and makes them feel better to deliver quality obstetric care. Key stakeholders also perceived the SDA as a relevant and useful tool with a reasonable cost and recommended its implementation in routine practices. Overall, both end-users and key informants think that the SDA is beneficial and could be sustained. End-users suggested additional features that would improve the usability of the SDA. These included: a computer/ laptop version of the SDA, a SDA version in Kinyarwanda language, some additional videos, and a SDA discussion forum. Key informants also suggested additional topics that could be added to the SDA such as; medical bio safety and bio security, patient centered care and quality improvement, and ethics in clinical practice. The positive perceptions of the SDA in Rwanda suggest that mLearning and mHealth decision support tools such as the SDA may be a potential way to improve knowledge and skills of nurses and midwives. The mHealth applications like the SDA have the potential to further improve patient care, as they avail evidence-based clinical guidelines and can be easily adapted to contextual needs. Thus, credible, evidence-based, affordable mHealth applications like the SDA are needed to support clinical decision making and provide a platform for updates and continuous learning for nurses and midwives in Rwanda and other limited resource settings. The present study also adds to existing evidence about the benefits of mHealth applications in assuring quality BEmONC. Abbreviations BEmONC: Basic Emergency Obstetric and Newborn Care; FGD: Focus Group Discussion; KII: Key informant interview; mHealth: mobile health; mLearning: mobile learning; LMICs: Low- and middle-income countries; NR: Neonatal Resuscitation; PPH: Post-Partum Hemorrhage; RAM: Rwanda Association of midwives; UNICEF: United Nations International Children's Emergency Fund; UNFPA: United Nations Population Fund, WHO: World Health Organization. Declarations Authors’ contributions AN, DNC, MN and, LI were involved in the conception and design of the study. AN drafted the manuscript. DNC and LI assisted with data analysis and interpretation. DNC, MN, and LI reviewed the manuscript. All authors read and approved the final manuscript. Acknowledgments We would like to thank The Ministry of Health, Rwanda, and The Maternity Foundation for supporting this research. We also acknowledge the Consortium for Advanced Research and Training (CARTA) for funding this research. Funding This research was supported by the Consortium for Advanced Research Training in Africa (CARTA). CARTA is jointly led by the African Population and Health Research Center and the University of the Witwatersrand and funded by the Carnegie Corporation of New York (Grant No--B 8606.R02), Sida (Grant No:54100113), the DELTAS Africa Initiative (Grant No: 107768/Z/15/Z) and Deutscher Akademischer Austauschdienst (DAAD). The DELTAS Africa Initiative is an independent funding scheme of the African Academy of Sciences (AAS)’s Alliance for Accelerating Excellence in Science in Africa (AESA) and supported by the New Partnership for Africa’s Development Planning and Coordinating Agency (NEPAD Agency) with funding from the Wellcome Trust (UK) and the UK government. The statements made and views expressed are solely the responsibility of the Fellow". The funding body/bodies played no role in the design of the study and data collection process, analysis, interpretation of data, and in writing the manuscript. Competing interests None declared. Availability of data and materials The datasets generated for this study will be made available from the corresponding author on a reasonable request. Consent for publication Not applicable. Ethics approval and consent to participate Ethical approvals were obtained from the Human Research Ethics Committee of the University of the Witwatersrand (M190258) and the University of Rwanda, College of Medicine and Health Sciences’ Institutional Review Board (No. 377/CMHS IRB/2018). All participants gave written informed consent and consents for recording the interviews prior to their participation. Participants were assured that their identities would remain anonymous in all reporting of the study and that their personal information would be kept confidential. References Organization WH. Trends in maternal mortality 2000 to 2017: estimates by WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division. 2019; Available from: https://apps.who.int/iris/handle/10665/327595 Hug L, Alexander M, You D, Alkema L, for Child UNIG. National, regional, and global levels and trends in neonatal mortality between 1990 and 2017, with scenario-based projections to 2030: a systematic analysis. Lancet Glob Heal. 2019;7(6):e710–20. Ameh C, Msuya S, Hofman J, Raven J, Mathai M, van den Broek N. Status of Emergency Obstetric Care in Six Developing Countries Five Years before the MDG Targets for Maternal and Newborn Health. Frasch MG, editor. PLoS One [Internet]. 2012 Dec 6 [cited 2017 Apr 10];7(12):e49938. Available from: http://dx.plos.org/10.1371/journal.pone.0049938 Bhutta ZA, Das JK, Bahl R, Lawn JE, Salam RA, Paul VK, et al. Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost? Lancet. 2014;384(9940). Otolorin E, Gomez P, Currie S, Thapa K. Essential basic and emergency obstetric and newborn care: From education and training to service delivery and quality of care. Int J Gynecol Obstet [Internet]. 2015 [cited 2017 Apr 5];130:S46–53. Available from: http://www.sciencedirect.com/science/article/pii/S0020729215001368 UNFPA. Setting standards for emergency obstetric and newborn care | UNFPA - United Nations Population Fund [Internet]. 2014 [cited 2017 Aug 23]. Available from: http://www.unfpa.org/resources/setting-standards-emergency-obstetric-and-newborn-care Tull K. Inequity and Leave No One Behind: Approaches to Maternal and Newborn Healthcare. 2019 Feb 27 [cited 2019 Apr 22]; Available from: https://opendocs.ids.ac.uk/opendocs/handle/123456789/14408 Joannah H RR. What is quality in maternal and neonatal health care? Midwifery [Internet]. 2012 [cited 2017 Oct 24];28(5):e676–83. Available from: http://www.sciencedirect.com/science/article/pii/S0266613811001367 Sharma G, Mathai M, Dickson KE, Weeks A, Hofmeyr GJ, Lavender T, et al. Quality care during labour and birth: a multi- country analysis of health system bottlenecks and potential solutions. BMC Pregnancy Childbirth [Internet]. 2015 [cited 2017 Apr 7];15(2):S2. Available from: http://www.biomedcentral.com/1471-2393/15 /S2/S2 Needs S, Health C, Philbrick WC. mHealth and MNCH: State of the Evidence. Mhealth Alliance, Un Found [Internet]. 2013;(January):1–45. Available from: http://www.mhealthknowledge.org/sites/default/files/15_un_007_evidencegapreport_digital_aaa.pdf Feroz A, Perveen S, Aftab W. Role of mHealth applications for improving antenatal and postnatal care in low and middle income countries: A systematic review. BMC Health Serv Res. 2017;17(1). Maternity Foundation M for mothers. What is the SDA app - Maternity Foundation [Internet]. 2014 [cited 2018 Jul 26]. Available from: https://www.maternity.dk/about-the-app/what-is-the-app/ Website MF. Our Work - Maternity Foundation [Internet]. [cited 2020 Jul 23]. Available from: https://www.maternity.dk/our-work/ Sayinzoga F, Bijlmakers L, van Dillen J, Mivumbi V, Ngabo F, van der Velden K. Maternal death audit in Rwanda 2009-2013: a nationwide facility-based retrospective cohort study. BMJ Open [Internet]. 2016 [cited 2016 Jul 24];6(1):e009734. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26801466 Kasine Y, Babenko-Mould Y, Regan S. Translating continuing professional development education to nursing practice in Rwanda: Enhancing maternal and newborn health. Int J Africa Nurs Sci [Internet]. 2018 Jan 1 [cited 2018 Jul 20];8:75–81. Available from: https://www.sciencedirect.com/science/article/pii/S221413911730063X Nishimwe A, Nyssen M, Ibisomi L, Nozizwe Conco D. Clinical decision making in basic emergency obstetric and newborn care among nurses and midwives: the role of the safe delivery mhealth application_pre-post-intervention study (research protocol). Informatics Heal Soc Care. 2021;1–11. Nishimwe A, Conco DN, Nyssen M, Ibisomi L. Context specific realities and experiences of nurses and midwives in basic emergency obstetric and newborn care services in two district hospitals in Rwanda: a qualitative study. BMC Nurs [Internet]. 2022;21(1):9. Available from: https://doi.org/10.1186/s12912-021-00793-y Nishimwe A, Ibisomi L, Nyssen M, Conco DN. The effect of an mLearning application on nurses’ and midwives’ knowledge and skills for the management of postpartum hemorrhage and neonatal resuscitation: pre–post intervention study. Hum Resour Health. 2021;19(1):1–10. MOH RHP. REPUBLIC OF RWANDA. 2015 [cited 2017 Aug 7]; Available from: http://www.moh.gov.rw/fileadmin/templates/policies/Health_Sector_Policy___19th_January_2015.pdf Rwanda NI of S of, Planning/Rwanda M of F and E, Health/Rwanda M of, International I. Rwanda Demographic and Health Survey 2014-15 [Internet]. 2016 [cited 2019 Nov 14]. Available from: https://dhsprogram.com/publications/publication-fr316-dhs-final-reports.cfm Patton MQ. Two decades of developments in qualitative inquiry: A personal, experiential perspective. Qual Soc Work. 2002;1(3):261–83. http://www.ncbi.nlm.nih.gov/pubmed/25326092 Poland BD. Transcription quality as an aspect of rigor in qualitative research. Qual Inq. 1995;1(3):290–310. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Heal care. 2007;19(6):349–57. Lund S, Boas IM, Bedesa T, Fekede W, Nielsen HS, Sørensen BL. Association Between the Safe Delivery App and Quality of Care and Perinatal Survival in Ethiopia: A Randomized Clinical Trial. JAMA Pediatr [Internet]. 2016;170(8):765–71. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27322089 Bolan NE, Sthreshley L, Ngoy B, Ledy F, Ntayingi M, Makasy D, et al. mLearning in the Democratic Republic of the Congo: A Mixed-Methods Feasibility and Pilot Cluster Randomized Trial Using the Safe Delivery App. Glob Heal Sci Pract [Internet]. 2018 Dec 27 [cited 2019 Apr 22];6(4):693–710. Available from: http://www.ncbi.nlm.nih.gov/pubmed/30591577 Michel-Schuldt M, Dayon MB, Klar RT, Subah M, King-Lincoln E, Kpangbala-Flomo C, et al. Continuous professional development of Liberia’s midwifery workforce—A coordinated multi-stakeholder approach. Midwifery. 2018;62:77–80. Adepoju I-OO, Albersen BJA, De Brouwere V, van Roosmalen J, Zweekhorst M. mHealth for Clinical Decision-Making in Sub-Saharan Africa: A Scoping Review. JMIR mHealth uHealth [Internet]. 2017 Mar 23 [cited 2019 Sep 15];5(3):e38. Available from: http://mhealth.jmir.org/2017/3/e38/ Ahmed MAA, Gagnon M-P, Hamelin-Brabant L, Mbemba GIC, Alami H. A mixed methods systematic review of success factors of mhealth and telehealth for maternal health in Sub-Saharan Africa. Mhealth. 2017;3. Brusamento S, Kyaw BM, Whiting P, Li L, Car LT. Digital health professions education in the field of pediatrics: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(9):e14231. Dunleavy G, Nikolaou CK, Nifakos S, Atun R, Law GCY, Car LT. Mobile digital education for health professions: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(2):e12937. Nilsson C, Sørensen BL, Sørensen JL. Comparing hands-on and video training for postpartum hemorrhage management. Acta Obstet Gynecol Scand. 2014;93(5):517–20. Taekman JM, Foureman MF, Bulamba F, Steele M, Comstock E, Kintu A, et al. A novel multiplayer screen-based simulation experience for African learners improved confidence in management of postpartum hemorrhage. Front public Heal. 2017;5:248. Zakane SA, Gustafsson LL, Tomson G, Loukanova S, Sié A, Nasiell J, et al. Guidelines for maternal and neonatal “point of care”: needs of and attitudes towards a computerized clinical decision support system in rural Burkina Faso. Int J Med Inform. 2014;83(6):459–69. Walker DM, Holme F, Zelek ST, Olvera-García M, Montoya-Rodríguez A, Fritz J, et al. A process evaluation of PRONTO simulation training for obstetric and neonatal emergency response teams in Guatemala. BMC Med Educ. 2015;15(1):117. Abejirinde IOO, Ilozumba O, Marchal B, Zweekhorst M, Dieleman M. Mobile health and the performance of maternal health care workers in low- and middle-income countries: A realist review. Vol. 21, International Journal of Care Coordination. 2018. Miller S, Abalos E, Chamillard M, Ciapponi A, Colaci D, Comandé D, et al. Beyond too little, too late and too much, too soon: a pathway towards evidence-based, respectful maternity care worldwide. Lancet. 2016;388(10056):2176–92. Ritchie LMP, Khan S, Moore JE, Timmings C, van Lettow M, Vogel JP, et al. Low-and middle-income countries face many common barriers to implementation of maternal health evidence products. J Clin Epidemiol. 2016;76:229–37. Runnacles J, Roueché A, Lachman P. The right care, every time: improving adherence to evidence-based guidelines. Arch Dis Childhood-Education Pract. 2018;103(1):27–33. Chawla D, Thukral A, Kumar P, Deorari A. Harnessing mobile technology to deliver evidence-based maternal-infant care. In: Seminars in Fetal and Neonatal Medicine. Elsevier; 2021. p. 101206. Eze E, Gleasure R, Heavin C. Reviewing mHealth in developing countries: A stakeholder perspective. Procedia Comput Sci. 2016;100:1024–32. Petersen C, Adams SA, DeMuro PR. mHealth: don’t forget all the stakeholders in the business case. Med 20. 2015;4(2). Maar MA, Yeates K, Perkins N, Boesch L, Hua-Stewart D, Liu P, et al. A framework for the study of complex mHealth interventions in diverse cultural settings. JMIR mHealth uHealth. 2017;5(4):e47. Bally ELS, Cesuroglu T. Toward integration of mHealth in primary care in the Netherlands: a qualitative analysis of stakeholder perspectives. Front public Heal. 2020;7:407. Franz-Vasdeki J, Pratt BA, Newsome M, Germann S. Taking mHealth solutions to scale: enabling environments and successful implementation. J Mob Technol Med. 2015;4(1):35–8. Dharmayat KI. Sustainability of ‘mHealth’interventions in sub-Saharan Africa: a stakeholder analysis of an electronic community case management project in Malawi. Malawi Med J. 2019;31(3):177–83. Additional Declarations No competing interests reported. Supplementary Files Additionalfile1AcceptabilitySurveyquestionnaire.pdf Additionalfile2InterviewGuides.pdf Additionalfile3COREQChecklist.pdf Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Major revision 08 Jun, 2022 Reviews received at journal 06 Jun, 2022 Reviews received at journal 16 Apr, 2022 Reviews received at journal 11 Apr, 2022 Reviewers agreed at journal 11 Apr, 2022 Reviewers agreed at journal 11 Apr, 2022 Reviewers invited by journal 11 Apr, 2022 Editor assigned by journal 06 Apr, 2022 Editor invited by journal 01 Feb, 2022 Submission checks completed at journal 24 Jan, 2022 First submitted to journal 14 Jan, 2022 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1260261","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":78874223,"identity":"9b6be893-0a22-4bbf-ac28-12aad17595c5","order_by":0,"name":"Aurore Nishimwe","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYBADOQhlw8BDtBZjIGZsYEgjQUtiA1QLYWDOfsbsw8c9Nun97WefP/iRcE+Gv/0A4+OKX7i1WPbkGM+c8Swtd8aZdMPGnoRiHokzCcyGZ/twazE4kGPMzHPgcO4GhjTGBt4fCTwGDAlsko09eLScfwPS8j/dgP8ZY+OfBKAW/gcEtNwA23IgwUAijbGZB6RFAmhLww98Wp4VM844kGw448YzxtkyQC0SNx42GzY24HNY8maGDwfs5Pn70xg+vklIsOfvTz74sOEPbi3YADB+GNtI0wICJNoyCkbBKBgFwxoAAKhEUJdd+iDVAAAAAElFTkSuQmCC","orcid":"","institution":"University of the Witwatersrand","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Aurore","middleName":"","lastName":"Nishimwe","suffix":""},{"id":78874224,"identity":"97504c4c-f1ad-4fd8-abea-c6f859d6c953","order_by":1,"name":"Daphney Nozizwe Conco","email":"","orcid":"","institution":"University of the Witwatersrand","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Daphney","middleName":"Nozizwe","lastName":"Conco","suffix":""},{"id":78874225,"identity":"39bbf756-a5bd-4297-8f7b-5203fe8338b4","order_by":2,"name":"Marc Nyssen","email":"","orcid":"","institution":"Vrije Universiteit Brussel","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Marc","middleName":"","lastName":"Nyssen","suffix":""},{"id":78874226,"identity":"9a8f74ef-ac38-4fd0-8a6f-a425026a5fa1","order_by":3,"name":"Latifat Ibisomi","email":"","orcid":"","institution":"University of the Witwatersrand","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Latifat","middleName":"","lastName":"Ibisomi","suffix":""}],"badges":[],"createdAt":"2022-01-14 10:14:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1260261/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1260261/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":17715526,"identity":"61aa14b8-3db7-482b-af74-a2097bb8ab79","added_by":"auto","created_at":"2022-01-27 21:09:54","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":125061,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eThematic framework\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-1260261/v1/52f57d3765ecd62f1437e87b.png"},{"id":17715583,"identity":"897c9c68-bf10-4db4-9a42-3c0de3b0026a","added_by":"auto","created_at":"2022-01-27 21:12:58","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":861624,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1260261/v1/52cba3f1-5a0b-4cfd-9930-b4e086290372.pdf"},{"id":17715529,"identity":"1d8ea717-ac9d-402a-8f4a-d782b0372983","added_by":"auto","created_at":"2022-01-27 21:09:55","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":588907,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1AcceptabilitySurveyquestionnaire.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1260261/v1/5fbaa9c61ac96b9d41eb02cb.pdf"},{"id":17715582,"identity":"b51e992f-6d46-4375-b784-ff6da08dc7eb","added_by":"auto","created_at":"2022-01-27 21:12:55","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":126985,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile2InterviewGuides.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1260261/v1/e87b70f7130ce470befd509d.pdf"},{"id":17715527,"identity":"77d4cb8f-8254-4545-adb4-d5c5bb113f69","added_by":"auto","created_at":"2022-01-27 21:09:55","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":374306,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile3COREQChecklist.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1260261/v1/8f8f7686535e606e61bc55ec.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A mixed-method study exploring experiences, perceptions, and acceptability of using a Safe Delivery mHealth Application in two district hospitals in Rwanda","fulltext":[{"header":"Background","content":"\u003cp\u003eGlobally, maternal and newborn deaths remain a public health concern despite several efforts that have been/are being made to reduce the deaths. It is estimated that 295,000 maternal deaths, 2 million stillbirths and 2.5 million early newborn deaths occur annually in the world [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. More than 90% of these deaths happen in low- and middle-income countries, like Rwanda [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Most of these deaths are attributable to preventable causes such as post-partum haemorrhage (PPH), newborn asphyxia, pre-eclampsia, obstructed labor, infections, and hypertensive disorders [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. The WHO, UNICEF, and UNFPA have identified essential interventions known as Basic Emergency Obstetric and Newborn Care (BEmONC) for prevention and treatment of the preventable causes of maternal and newborn deaths [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. The BEmONC package includes the management of PPH, the removal of a retained placenta, the management of maternal sepsis, the management of hypertensive disorders, the management of prolonged labor, the management of complications of abortion, and neonatal resuscitation (NR) [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. These services can be provided by nurses and midwives. There is evidence that BEmONC is a key intervention to reduce death and disability of mothers and newborns [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. However, the available literature remains characterized by gaps in assuring the quality of BEmONC.\u003c/p\u003e \u003cp\u003eThe quality of BEmONC services is important to improving maternal and newborn health [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. The research evidence suggests that barriers to instituting quality BEmONC are complex and are often linked to limited knowledge and skills among health care providers, lack of access and adherence to evidence-based clinical guidelines, insufficient monitoring to inform appropriate responses, and lack of some essential materials [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. Besides, research recommends the use of mobile health (mHealth) to improve health systems, as a solution that could potentially support in addressing some of the health systems challenges and improve service delivery [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe safe delivery mHealth application (SDA) is a novel mHealth application which is a decision support and training tool for BEmONC that was developed by the Maternity Foundation [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. The main purpose of the SDA is to make a healthcare provider better prepared to prevent, detect, and manage life threatening complications to mothers and newborn via offering evidence-based BEmONC [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. The SDA contains animated videos with instructions on management of PPH and NR. The SDA offers the opportunity to watch the entire instructional animated video (8\u0026ndash;15 min long) as well as choosing to see a specific procedural step of each video (1-3 min long). Besides, the SDA contains a catalogue of action cards with essential recommendations to support clinical decision making in an emergency situation when the user does not have time to review a video. The SDA also include a comprehensive drug list which contains useful information on the various essential drugs needed to perform PPH management and NR. Further, the SDA describes certain essential procedures step-by-step and required equipments. In addition, the SDA has a self-explanatory learning platform \u0026lsquo;My Learning\u0026rsquo; that provides learning and assessment strategies based on key elements within the clinical content found in the SDA. To stimulate use, the SDA sends weekly notifications with quiz questions and a direct link to the films where the information to answer the question is found. The clinical content of the SDA was developed following international WHO guidelines but focusing on key essential lifesaving interventions of BEmONC [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe SDA can be downloaded free of charge for iPhone at: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://itunes.apple.com/dk/app/safe-delivery/id985603707?mt=8\u003c/span\u003e\u003c/span\u003e and for Android at: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://play.google.com/store/apps/details?id=dk.maternity.safedelivery\u0026amp;hl=en\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003eThe present paper, was derived from a broader study about the implementation of the SDA in handling two elements of the BEmONC: PPH and NR in two district hospitals in Rwanda. Such focus was informed by evidence that indicated PPH and newborn asphyxia as the priority challenges in Rwanda [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. The SDA was piloted in two district hospitals in Rwanda. This pilot implementation process was divided into three phases (pre-intervention, intervention, and post-intervention). The first phase (pre-intervention) involved the presentation of the research to different stakeholders and the baseline study (surveys, records review, and focus group discussions). The second phase (intervention) included the capacity building in the usage of the SDA among nurses and midwives; the SDA provision and piloting; and the SDA implementation for a period of six months. The third phase (post-intervention) included the implementation evaluation and encompassed the endline study (surveys, records reviews, focus group discussions, and key informant interviews). More details are provided in the protocol paper for this research [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. The present paper explores the end-users\u0026rsquo; experiences, perceptions and acceptability of the SDA and key stakeholders\u0026rsquo; perspectives.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv class=\"Section2\" id=\"Sec3\"\u003e\n \u003ch2\u003eStudy design\u003c/h2\u003e\n \u003cp\u003eThis paper describes part of a larger study about the implementation of the SDA in BEmONC services in two district hospitals in Rwanda. The larger study methodology and the evaluation of initial outcomes have been reported elsewhere [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e]. The part of the study presented in this paper, used mixed research methods to explore end-users\u0026rsquo; experiences, perceptions, and acceptability, and explore the key stakeholders\u0026rsquo; perceptions of the SDA.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec4\"\u003e\n \u003ch2\u003eStudy setting\u003c/h2\u003e\n \u003cp\u003eThe study took place in two district hospitals in Rwanda: Masaka hospital in Kigali, an urban province; and Nyamata hospital located in the eastern rural province [\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e].The two hospitals were selected out of 12 district hospitals in the two provinces because both had a high number of deliveries per year [\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e] and had been offering BEmONC services for more than five years.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec5\"\u003e\n \u003ch2\u003eStudy Participants\u003c/h2\u003e\n \u003cp\u003eThe study included 60 participants, the nurses and midwives (n=54) and stakeholders (n=6). Nurses and midwives who deal with childbirth in the maternity departments of the two study hospitals were invited to participate in the study and the inclusion criteria were as follows: having work experience of at least six months in obstetric care; being full-time employed in the selected hospitals; and willing to participate in the study. The selected stakeholders (key informants in the management position at the district hospital level) included maternity matrons (n=2), managers responsible for maternal and child health (n=2), and district hospital directors (n=2).\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec6\"\u003e\n \u003ch2\u003eData collection and analysis\u003c/h2\u003e\n \u003cdiv class=\"Section3\" id=\"Sec7\"\u003e\n \u003ch2\u003eQuantitative data\u003c/h2\u003e\n \u003cp\u003eThe quantitative part of the research entailed asking all nurses and midwives who were part of implementation of the SDA to complete a self-administered questionnaire (Additional file 1) evaluating usage of the SDA, as well as their perception and acceptability. The survey was conducted at endpoint once users had used the SDA for six months. Responses were analysed descriptively using Stata version 16 (StataCorp LLC).\u003c/p\u003e\n \u003c/div\u003e\n \u003cdiv class=\"Section3\" id=\"Sec8\"\u003e\n \u003ch2\u003eQualitative data\u003c/h2\u003e\n \u003cp\u003eTwo methods were used to collect qualitative data - focus group discussions (FGDs) and key informant interviews (KIIs).\u003c/p\u003e\n \u003cp\u003eFocus groups discussions explored in depth the participants\u0026rsquo; experiences and views on the functionality of the SDA. Two FGDs were conducted, one in each of the study hospitals. In each hospital, 12 participants were recruited and total number of FGD participants is 24. The participants included in the FGDs were selected based on their interest and availability. We sought perspective diversity in terms of age, gender, and experience [\u003cspan class=\"CitationRef\"\u003e21\u003c/span\u003e] and these criteria were considered in both groups.\u003c/p\u003e\n \u003cp\u003eThe Key Informant Interviews were conducted with six participants (three from each hospital), who were selected because of their leadership/management portfolios in the participating hospitals. The purpose was to get their views on the feasibility of adopting the SDA for wider implementation with the focus on its costs, possible scalability and sustainability. The interviews with key stakeholders started with an introduction of the SDA to the key informant including an explanation of the features and modules of the SDA, and a demonstration of how the SDA functions using a smartphone. After the introduction of the SDA, the key informant was invited to ask questions and comment on the SDA. Once their concerns were addressed, the researcher presented the estimated total cost that was used for piloting the SDA in two district hospitals (Masaka and Nyamata) in Rwanda. The estimated total cost was 10.076.410 Rwf (\u003cspan\u003e$\u003c/span\u003e10000) including the technology cost =4.504.000 Rwf (\u003cspan\u003e$\u003c/span\u003e4470.56), the training cost =2.083.410 Rwf (\u003cspan\u003e$\u003c/span\u003e2067.94) and the intervention cost =3.489.000 Rwf (\u003cspan\u003e$\u003c/span\u003e3463.09). Then, participants were asked to provide their perceptions of the SDA and their opinions on its cost, scalability, and sustainability.\u003c/p\u003e\n \u003cp\u003eThe interview guides (Additional file 2) were developed in English, translated to Kinyarwanda, and translated back to English in a blinded manner to ensure accuracy and equivalence [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e]. To ensure the quality and adequacy of the interview guides, they were pretested with two participants at a different district hospital. The interviews were conducted face to face in March 2020 and were held in private rooms at both Masaka and Nyamata district hospitals to ensure confidentiality. Interviews were conducted in the language of the participants\u0026rsquo; preference, which was Kinyarwanda. The first author conducted the interviews with assistance of an experienced research assistant who assisted in taking notes and supervising the tape recording. FGDs lasted between 60 and 90 minutes and KII interviews between 30 and 45 minutes. All FGDs and KIIs were digitally recorded and transcribed in Kinyarwanda by two research assistants. Transcriptions were done following the true verbatim method to accurately capture meanings, perceptions, and context [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e]. Then, transcripts were translated from Kinyarwanda to English by a professional translator. In order to ensure that everything was translated as correctly as possible, all transcriptions were compared with the audio files and field notes. The data were analyzed using thematic analysis with a hybrid approach (inductive and deductive) [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e]. Two authors (AN and DNC) first read all transcripts independently, then jointly developed a codebook. Disagreements were resolved by discussion. The final list of codes was applied to all transcripts using Nvivo 11 Plus software. Coded output was then read, merged into major themes, and illustrative quotations were identified. The consolidated criteria for reporting qualitative research [\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e] guided reporting for the qualitative data (Additional file 3).\u003c/p\u003e\n \u003cp\u003eFindings from both quantitative and qualitative data were integrated during the data-interpretation stage. The goal was to obtain different but complementary data that validate the overall results about the SDA acceptability.\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec9\"\u003e\n \u003ch2\u003eEthical considerations\u003c/h2\u003e\n \u003cp\u003eEthical approvals were obtained from the Human Research Ethics Committee of the University of the Witwatersrand (M190258) and the University of Rwanda, College of Medicine and Health Sciences\u0026rsquo; Institutional Review Board (No. 377/CMHS IRB/2018). All participants gave written informed consent and consents for recording the interviews prior to their participation. Participants were assured that their identities would remain anonymous in all reporting of the study and that their personal information would be kept confidential. All methods in this study were carried out in accordance with relevant guidelines and regulations in the Ethical Declarations.\u003c/p\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Results","content":"\u003cdiv class=\"Section2\" id=\"Sec11\"\u003e\n \u003ch2\u003eCharacteristics of quantitative study participants\u003c/h2\u003e\n \u003cp\u003eA total of 54 nurses and midwives completed the acceptability survey. The majority of participants were midwives (n=33). More than half were from Masaka district hospital (n = 30, 56%), the majority were female (n = 33, 61%), their average age was 33 years (SD = 7.1). The level of education was predominantly the advanced diploma (A1) in midwifery with 27 of 54 (50%) participants having advanced diploma (A1 level) in midwifery. The least represented level of education was the secondary school level (A2) in nursing with only one participant with A2 level in nursing. A detailed description of the survey respondents is provided in Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable border=\"1\" id=\"Tab1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eDemographic characteristics of the acceptability survey respondents ( N=54)\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 52.7567%;\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003en (%)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 55.4047%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHospital Affiliation\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eMasaka District Hospital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e30 (55.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eNyamata District Hospital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e24 (44.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 55.4047%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSex\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e21 (38.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e33 (61.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 55.4047%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eEducation Level\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eMidwife A0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e6 (11.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eMidwife A1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e27 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eNurse A0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e5 (9.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eNurse A1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e15 (27.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 2.7499%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 52.7567%;\"\u003e\n \u003cp\u003eNurse A2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e1 (1.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 55.4047%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAge, years, Mean (SD)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 19.249%;\"\u003e\n \u003cp\u003e33 (7.12)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"3\" style=\"width: 77.4036%;\"\u003e\n \u003cp\u003eAbbreviations: %: Weighted percent, A0 = bachelor\u0026rsquo;s degree, A1 = Advanced Diploma, A2 = Secondary school level.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003ch2\u003e\u003cstrong\u003eThe nurses\u0026rsquo; and midwives\u0026rsquo; experiences of the SDA during the study implementation.\u003c/strong\u003e\u003c/h2\u003e\n \u003cdiv class=\"Section3\" id=\"Sec12\"\u003e\n \u003ch2\u003eThe utilization of the SDA by nurses and midwives\u003c/h2\u003e\n \u003cp\u003eThe majority of participants (n=31) reported using the SDA four to six times per week. Features of the SDA that were most consulted include; the action cards (n=15), my learning (n=14) and the drug list (n=10). A half of the participants used the SDA to revise their knowledge. Other participants used the SDA in emergency situations (n=14), in a normal work situation (n=5) and to discuss with co-workers (n=8). Nurses and midwives predominantly accessed the SDA on both their own smartphones and on the smartphones provided by the researcher (n=28). A large number of participants (n=35) often used the SDA with a co-worker. Participants reported having access to most of the equipments and drugs seen in the SDA (n=40). More details on the SDA use and evaluation by end-users are shown in Table \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable border=\"1\" id=\"Tab2\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eSDA access and evaluation by nurses and midwives ( N=54)\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 66.2257%;\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003en (%)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 68.2161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFrequency of SDA use in a week\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eI used it almost every day\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e15 (27.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eI used it 4-6 times\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e31 (57.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eI used it 1-3 times\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e8 (14.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eI didn\u0026rsquo;t use it at all\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e0 (0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 68.2161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFeatures of the SDA most consulted\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eVideos\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e9 (16.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eAction cards\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e15 (27.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eDrug list\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e10 (18.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eProcedures\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e6 (11.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eMy Learning\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e14 (25.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 68.2161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eMost often quoted reasons for using the SDA\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eTo revise my knowledge\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e27 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eDuring an emergency\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e14 (25.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eIn a normal work situation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e5 (9.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eTo discuss with co-workers\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e8 (14.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 68.2161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eThe device often used to access the SDA\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eOwn phone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e7 (13)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eThe phone provided by the researcher\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e19 (35.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eBoth\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e28 (51.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 68.2161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eWith who do you often use the SDA?\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eBy myself\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e19 (35.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eWith a co-worker\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e35 (64.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 68.2161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccess to the equipment and/or drugs seen in the SDA.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eAll\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e14 (25.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eMost\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e40 (74.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 1.9904%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 66.2257%;\"\u003e\n \u003cp\u003eNone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 12.0328%;\"\u003e\n \u003cp\u003e0 (0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"3\" style=\"width: 81.2442%;\"\u003e\n \u003cp\u003eAbbreviations: Wt.% : Weighted percent; SDA: Safe Delivery mHealth Application\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eWhen rating their experiences of using the SDA particularly, half of respondents (n=27) indicated that it was very easy to the SDA, however one participant reported that it was very difficult to use a smartphone. SDA features that were most appreciated included: the action cards (33%), the drug list (22%), and my learning (19%). They also reported feeling more confident (53.7%) and becoming better (40.7%) at doing their jobs at work. Few participants (n=5) reported what they liked least about the SDA, three participants stated that they did not understand instructions, and the other two said that it was difficult to navigate the smartphone. More details on the experiences with the SDA and its added value are displayed in Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable border=\"1\" id=\"Tab3\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eExperiences with the SDA and its added value ( N=54)\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" style=\"width: 79.6946%;\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003en (%)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 81.2691%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eExperience with the smartphone and the SDA\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eVery easy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e27 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eSomewhat easy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e23 (42.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eSomewhat difficult\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e3 (5.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eVery difficult\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e1 (1.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 81.2691%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFeatures of the SDA most appreciated\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eAction cards\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e18 (33.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eVideos\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e10 (18.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eDrug list\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e5 (9.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eProcedures\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e12 (22.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eMy Learning\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e9 (16.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 81.2691%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAdded value from the SDA\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eI have become better at doing my job.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e22 (40.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eI feel more confident at work.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e29 (53.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eFewer mortalities/fatal cases\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e3 (5.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eI did not notice any change.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e0 (0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 81.2691%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eWhat do you like least about the SDA?\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eI do not understand the instructions.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e3 (5.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eI think it is difficult to navigate the smartphone.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e2 (3.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eThe app does not add to my existing knowledge.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e0 (0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 79.6946%;\"\u003e\n \u003cp\u003eNone applicable.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 15.7451%;\"\u003e\n \u003cp\u003e49 (90.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 97.0142%;\"\u003e\n \u003cp\u003eAbbreviations: Wt.% : Weighted percent; SDA: Safe Delivery mHealth Application\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003ch2\u003e\u003cstrong\u003eAcceptability of the SDA as rated by nurses and midwives.\u003c/strong\u003e\u003c/h2\u003e\n \u003cp\u003eNurses and midwives were asked to rate their acceptance of the SDA in a Likert scale. They were asked whether they \u0026quot;strongly disagree\u0026quot;, \u0026quot;disagree\u0026quot;, were \u0026quot;undecided\u0026quot;, \u0026quot;agree\u0026quot;, or \u0026quot;strongly agree\u0026quot; to a series of statements relating to the SDA acceptability. Responses were assigned point values as follows; strongly disagree =1, disagree =2, undecided =3, agree =4, strongly agree =5. Likert scale survey results showed that the SDA was: well understood (Mean=4.00); easy (4.46) and comfortable (4.31) to use; an effective decision support (4.63) and training (4.65) tool; and useful in improving efficiency in BEmONC (4.59). Most participants reported that they intended to continue using the SDA in daily practices (4.68). On the other hand, a lower rating was given to using the SDA when seeing notifications (2.48), implying that most participants did not only use the SDA when they saw notifications. The notifications are push messages that appear occasionally on the phones of the SDA users to remind them to revisit the SDA. The messages mainly guide the users to interesting parts that they may want to explore in the SDA. Overall participants reported good acceptability of the SDA. Likert scale responses are highlighted in Table \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable border=\"1\" id=\"Tab4\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eLikert scale survey responses of nurses and midwives about SDA acceptance\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eStatement\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eN\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMean (SD) [Range]\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI have a good understanding of what the SDA is\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.00 (0.71) [\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe SDA is easy to use\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.46 (0.50) [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI am comfortable using the SDA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.31 (0.57) [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe SDA is an effective decision support tool for BEmONC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.63 (0.69) [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe SDA is an effective training tool for BEmONC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.65 (0.48) [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe SDA is useful in improving efficiency in BEmONC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.59 (0.56) [\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI intend to continue using the SDA in my daily practices\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.68 (0.50) [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI use the SDA when I see notifications\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.48 (0.40) [\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"3\"\u003e\n \u003cp\u003eNote: Likert scale response options were as follows: 1Strongly Disagree, 2Disagree, 3Neutral, 4Agree, and 5Strongly Agree\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec13\"\u003e\n \u003ch2\u003ePerceptions about the usefulness and acceptability of the SDA\u003c/h2\u003e\n \u003cp\u003eThe results of the interviews with end-users of the SDA emerged into four main themes: (1) the perceived usefulness of the SDA; (2) professional growth acquired through the use of the SDA; (3) SDA, an empowering, intuitive and user-friendly technology; and (4) the desired SDA features and functions. The results of the interviews with key stakeholders revealed two main themes (1) benefits of the SDA as perceived by the key informants, and (2) future use of the SDA. These identified themes are shown in Figure \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. More details on the main themes and sub-themes are presented in the discussion of results, and are illustrated by verbatim quotations from the two FGDs and six KIIs.\u003c/p\u003e\n \u003cp\u003e\u003cspan class=\"BoldUnderline\" name=\"Emphasis\" type=\"BoldUnderline\"\u003eInterviews with end-users of the SDA\u003c/span\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec14\"\u003e\n \u003ch2\u003ePerceived usefulness of the SDA\u003c/h2\u003e\n \u003cp\u003eParticipants described the SDA as a useful, intuitive, empowering and practical tool that supported their daily practices. They described how it provided important information and guidelines on management of obstetric emergencies, and thus, increases their knowledge and skills on how to manage normal childbirth as well as birth complications.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;This application came as a golden opportunity for us \u0026hellip;normally, it is very hard to go to a library all the time to read; but the library, I am hereby talking about is that SDA application we have received\u0026hellip;It provides very useful information and guidelines that upgrade our knowledge and skills in delivery care\u0026rdquo;\u003c/em\u003e\u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eIn addition, nurses and midwives reported using the SDA both as a decision support tool while managing birth complications including PPH or NR, and as a debriefing tool for reflections on performed care to identify if essential procedures had been missed.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;When you have an emergency case to manage, you don\u0026rsquo;t have difficulties figuring out how to handle it\u0026hellip;all the steps are clearly outlined in this application and so well summarized that you immediately find what you need whenever you refer to it\u0026hellip;The SDA support our clinical decisions in emergency situations\u0026hellip;Sometimes, when I go home from a night duty, on the bus, I consult the SDA to see if I have performed well during the night\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eParticipants also described the SDA as a practical and easy to use tool that helps strengthen their theoretical knowledge. They articulated how the SDA helped them remember the management of rare clinical cases. Procedures for those rare cases are easy to forget as they rarely had to apply the theory learnt from school into practice.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;Before SDA, you could sometimes start wondering what management was appropriate for a case which is not so frequent after you had forgotten\u0026hellip;Now, we refer to the SDA when there is a doubt about anything\u0026hellip; With this new application, we have become all fresh and we can easily remember the essential procedures step by step as per international standards\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOther participants expressed appreciation of the SDA as a training tool, highlighting the ingenuity of the visuals when dealing with life threatening complications of mothers and their newborns. They considered the SDA to be a kind of simulated clinical training that continuously enhances their knowledge and practical skills, contributing to their professional growth. They also reported improved performance when managing childbirth complications. Such improvement was attributed to continuous learning, facilitated by the SDA and the fact that learning could take place anywhere.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;This application has been an important learning tool\u0026hellip;I consider it as a training manual with all clinical guidelines accompanied by video materials\u0026hellip; we learn a lot from the SDA in addition to what we learned at school\u0026hellip;You can learn from this application from anywhere: at the hospital, on the bus, at home or anywhere. So, having your telephone means being able to learn more anywhere.\u0026rdquo;\u003c/em\u003e\u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eThen again, participants revealed that they used to consult other internet sources for learning like youtube. They found such sources not so easy to use, as it was hard to get concise updated information and guidance in a timely manner, the way the SDA does.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;We used to visit youtube to learn something about the services we deliver, but most materials in there are not so clear as in this app. When you consult the SDA, you find that everything is easy to understand. You quickly get information on which medication is good for which case\u0026hellip;\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;Some of us are mentors in health centers and we have been consulting this application while preparing training materials\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSome found the SDA notifications useful as reminders for them to continue learning. The notifications that appeared intermittently on their phones, reminded them to revisit the SDA that also pointed them to other interesting parts of the SDA that they may want to explore.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;We have posters displayed in maternity telling us what to always bear in mind. Some of the information on them is \u0026ldquo;remember to close the door to keep the newborn warm\u0026rdquo; \u0026hellip; I compare this with the SDA notifications that I frequently get on my smartphone\u0026hellip;these are very nice reminders that I have to keep learning\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I would come back to what a colleague said about SDA notifications. This is a way of alerting you and I like that\u0026hellip; There exists a wrong belief that nurses don\u0026rsquo;t like reading. Maybe, it is true but this application is not too demanding. It doesn\u0026rsquo;t consume your time as if you go to a library\u0026hellip; This app is a good source and we need to regularly use it\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD1.\u003c/strong\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec15\"\u003e\n \u003ch2\u003eProfessional growth acquired through the use of the SDA\u003c/h2\u003e\n \u003cp\u003eThe perceived usefulness of the SDA described by the participants was associated with substantial improvements in their childbirth care practices. Nurses and midwives narrated experiencing an increase in their knowledge, skills, and confidence after using the SDA and how such professional growth has been reflected in the quality service they offer.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;The reason number one I like this app is that it upgrades my knowledge and skills, and makes me confident than before\u0026hellip;it is a kind of continuous professional development\u0026hellip;It is so good in a way that if you fail to understand the text when reading, you have an option to search for more explanation using videos. It is really very useful\u0026hellip;\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;The biggest difference for me when I compare now and the past experience is service delivery which has so positively improved. The special aspect of the application is that the content therein has everything we theoretically studied at school\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eThe study participants indicated that the SDA has helped them to recollect and revise their skills, which have made them more capable at managing childbirth complications in accordance with the recommended essential procedures. One of the significant changes expressed by participants was that they had learned the lifesaving steps of how to stop a bleeding by applying bimanual compression.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;This application has been helping us refresh our memories and equipping us with techniques that we use to handle obstetric emergencies.\u0026hellip; My colleagues have talked about bimanual compression\u0026hellip;You may even not know anything about it \u0026hellip; Then, when you try to familiarize with it on the SDA, you get to know about it and perform it correctly\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOther participants explained how the SDA supported them during practice. They described the before SDA childbirth care experience and compared it with after SDA experience by highlighting the specific instances in which the SDA supported them. Previously, in situations where nurses and midwives were struggling to assist a mother or a newborn, they would immediately call upon a medical doctor to come and help. And, sometimes the doctor assigned to maternity department was busy with caesarean surgery which would delay his assistance to attend to the complicated case. But, now with the support of the SDA, nurses and midwives have been able to provide essential assistance to mothers and newborns while waiting for the doctors.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I agree that BEmONC services were also delivered before we started using SDA but the situation was not the same. For example, when there was a complication with a patient, the only thing we would do was to ask a colleague or to call a medical doctor. This was time consuming and would delay service delivery and could complicate the situation even cause maternal deaths... However, when there is a problem nowadays, you just take your phone, go to the application and get a quick solution. As a result, our patients get quick and more efficient service\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eBoth nurses and midwives revealed that they used to fear occurrence of birth complications, because of the limited skills and experience on how to manage such events. They noted that the SDA had reduced this fear and had led them to have more confidence in their work.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;In using this app, you are always aware of what might happen in a given case. So, when you have a real case, you don\u0026rsquo;t panic much\u0026hellip;For instance, when it comes to PPH management, with the SDA you can easily figure out the steps to follow\u003c/em\u003e\u0026rdquo; \u003cstrong\u003eNurse, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;This application can help you when you are stressed. You may have had a lot of cases to manage in a day and end up in confusion somewhere. When this happens, you take your phone and see what should be done to handle a particular case with confidence.\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSome participants described how solving problems using the SDA motivated and enabled them to be more efficient in providing care at work.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;when you have a problem and you find a way to solve it, that is self-explanatory; you feel motivated automatically. As we get support from the application, our job becomes easier and more efficient\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eFurther, nurses and midwives explained that they used the SDA either at home to update knowledge or while they were at work, either before a delivery when the woman is in labor or when complication arises in the course of a delivery. Some participants described using the SDA to guide co-workers when they work in pairs in order to follow the necessary steps during a delivery. In emergency situations, the SDA was often used as a decision support tool to see how to manage e.g. PPH and neonatal resuscitation or as a reminder of drug dosages and administration. Participants also acknowledged having access to most of the equipments and drugs seen in the SDA.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;We have most of the materials we need to perform well. But, the SDA has been there to support us in upgrading our knowledge wherever we are - at home, before assisting a delivery or during an emergency\u0026hellip;You may forget something like medication dosage, even though you learned it at school\u0026hellip; but since we got this app, it helps us to remember the dosages in emergencies, we use it as a job aid in a simple way\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD2.\u003c/strong\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec16\"\u003e\n \u003ch2\u003eSDA an empowering, intuitive and user friendly technology\u003c/h2\u003e\n \u003cp\u003eThe study participants also voiced that the SDA is easy to use, intuitive, user friendly, and enabled them to provide first aid to patients before calling upon doctors.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003eThis application is special in that it is so focused and shows you clear sequences\u0026hellip;For example, you can easily find what specifically refers to steps for removing a retained placenta when you have such a case. We really like it because it is easy to use\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I would say something additional. With this application, we have learnt how at least one can provide first aid for a mother or do resuscitation for a newborn\u0026hellip; That is so important because you don\u0026rsquo;t need to call upon a doctor for everything\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eLikewise, participants indicated that the SDA has helped them to get a common understanding with medical doctors about managing birth complications. They explained on how consulting the SDA helped them to have informed recommendations and they were happy to contribute with confidence to care plans and sometimes supporting doctors.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I would say that when I have a special case, I first consult the application, see how I can myself manage it before I talk to a medical doctor. Even when I have to talk to the doctor, at least I provide an informed suggestion and contribute to further treatment steps. We are supporting our doctors much more.\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eThen again, nurses and midwives described how the SDA has helped them to get updated information and to have a common understanding among themselves during practice.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I can easily remember what I studied in school and when I tell my colleague X, he sometimes tells me a different story. This is due to the fact that he studied so recently, let\u0026rsquo;s say three years ago, while I studied the similar subject 12 years ago\u0026hellip;There is a need for all of us to have the same updates. This application is a solution because it keeps getting updated\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eWhen asked if they will continue using the SDA in their future practice. Both nurses and midwives stated that due to the SDA benefits, it is their own advantage to continue using it in their routine practice for efficient provision of care.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I would say that we have been lucky to be the champions of this application. I agree that we have a lot of work to do in maternity but we can\u0026rsquo;t forget that we also need updates in our career\u0026hellip;We will always need to refer to the app in our routine practices and find some time to learn new things from the SDA. We shall keep using it and it is for our own interest\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;This application is user friendly, affordable, covers a lot of areas, and does not take your internet megabytes when using\u0026hellip; It is not like a book or a computer that would be heavy for you to carry\u0026hellip;It is in a mobile phone that you always carry in your hands\u0026hellip;You can check up anything, anywhere and anytime.\u0026hellip; Unless something unexpected happens, I don\u0026rsquo;t see how people can stop using the SDA\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eWhen asked how they perceive the SDA sustainability in district hospitals in Rwanda, all nurses and midwives indicated that the SDA will be likely sustained.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;This application will be maintained because it is an important tool for us to deliver good service to the patients. When you use it properly, you deal with various cases with confidence and you perform your duties with success\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I am also confident that SDA will be sustained for two main reasons. The first reason is that it is easy to use and provides text content accompanied by video materials. Secondly, SDA is so relevant in that it is equipped with modules that reflect our day to day clinical practices\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eFurthermore, participants requested the SDA scalability to the whole country of Rwanda in consideration of the present country\u0026rsquo;s agenda to support the adoption of technology in healthcare service delivery.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u0026ldquo;Personally, I would say that this is the most appealing application on my phone\u0026hellip;When we are discussing with colleagues on social media (WhatsApp), there may be some questions raised in relation to our career. When there is a confusion, I immediately take my phone, get reliable information from the SDA and in a fast way I share with them\u0026hellip; Is it possible to implement this app in all hospitals in Rwanda? \u0026hellip;Since most people have smartphones, this should not be a problem, also our country is currently promoting application of technology in healthcare. I think this would make a lot of difference\u0026rdquo;\u003c/em\u003e\u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I have to tell you that this is a very good application, those using the application have already started seeing the benefits and they gain a lot from it. I wish it could have been shared with other hospitals too. It should be scaled up to national level. I wish to see all midwives in Rwanda using this app\u0026hellip; It is easy to use, once you have it installed, you don\u0026rsquo;t need much guidance on how to use it, then we will be able to save more maternal and newborn lives\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I remember I was in Musanze some time back and when I talked about this application to my friends, they were surprised. They started asking me how they can access it too. I just showed them and they became very interested. I am inviting everyone to have this spirit of sharing so that it reaches more people\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOther participants recommended that the SDA should be part of learning materials used at University.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;Ever since I started using the SDA, there has been no single case that I have failed to get information about in this application. I think this application should be part of tools that midwifery students use at University before they graduate. If this is done, using it at workplace would be easier. I am confident that if you once saw something at school and you need to use it in your job, it becomes easily implementable\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec17\"\u003e\n \u003ch2\u003eDesired SDA features and functions\u003c/h2\u003e\n \u003cp\u003eThe study participants were asked if there might be any other information or functionalities they would want to be added to the SDA for improving the application. Most participants stated that the SDA is useful as it is. However, some participants suggested features and functions they wish to be added to the SDA. Firstly, participants requested if the SDA could have a version for desktop and laptop computers so that it could be installed in computers available at their respective hospitals or if they could access the SDA using a desktop/laptop browser.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I wonder if this application can\u0026rsquo;t be installed in our office computers because sometimes one\u0026rsquo;s phone might not be available maybe because it is not charged, it is stolen or broken or the owner simply doesn\u0026rsquo;t have it with him/her. Having the application available in various devices would be much better\u003c/em\u003e and \u003cem\u003eaccessing it might be easier for everyone\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOther participants expressed their need for locally appropriate clinical decision support tools and emphasized their wish to have a version of the SDA available in the local language \u0026lsquo;Kinyarwanda\u0026rsquo;.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I would like to add that it would be great to have this application in Kinyarwanda like I saw it having other language versions including Swahili, Hindi\u0026hellip;. I guess this is a process, but we will be grateful if this is expedited. That will really be helpful. Thanks\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eNurse, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eIn addition, nurses and midwives requested additional animated videos to be added to the SDA. These include; videos on antenatal care in line with the new WHO antenatal care models with a minimum of eight contacts as well as guidance on the identification of risk factors for birth complications before delivery.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026lsquo;\u0026lsquo;I think this application can be very helpful if they add some videos on antenatal care or how one could identify a woman in labor who would probably get a complication\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAlso important was the suggestion that the SDA application could have a discussion forum window or platform for the users across different hospitals and countries to share experiences about managing obstetric emergencies.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I wish that this application can have a kind of worldwide discussion forum where we can put comments and share experiences and views. For example, a midwife from X hospital in Rwanda can discuss and share her experience with midwives in other hospitals across multiple countries\u0026hellip;Sharing like this can upgrade levels of knowledge for everyone. I think this would be very beneficial\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eMidwife, FGD1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cspan class=\"BoldUnderline\" name=\"Emphasis\" type=\"BoldUnderline\"\u003eInterviews with key stakeholders\u003c/span\u003e\u003c/p\u003e\n \u003cp\u003eThe analysis revealed two main themes: (1) benefits of the SDA as perceived by the key informants, and (2) future use of the SDA. These identified themes are shown in Figure \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. The details on the main themes and sub-themes are presented, and are illustrated by quotations from the six KIIs.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec18\"\u003e\n \u003ch2\u003eBenefits of the SDA as perceived by the key informants\u003c/h2\u003e\n \u003cp\u003eThe key informants described the SDA as a useful tool and recommended its implementation in routine practices, specifically for the purpose of continuous learning without interrupting health services. Such benefit was depicted as fitting for the work environment in the context of Rwanda where numbers of healthcare providers were insufficient. Clinical staff require periodic training to stay on track with the knowledge and skills necessary for their practice. For their annual practice licence renewal, they need evidence of continuous professional development in addition to corresponding academic credits. SDA was therefore seen as a valuable solution as it offers on-the-job learning opportunities, assessment strategies, and possibilities of getting certificates.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;This application is useful as it allows on-job continuous learning. We have been facing challenges for getting continuous learning opportunities for all staffs, now the SDA comes with this advantage. I recommend that it should be incorporated in routine practices\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 3.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eWhile the SDA was recommended for staff, because it captures the essential topics for BEmONC, key informants felt that this should in addition to the other sources of information including formal books and Ministry of Health guidelines that gives a broader view of obstetric care.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;The SDA is good particularly for BEmONC but the SDA alone cannot solve all maternity care problems. Our staff need to use this application together with other sources of reliable information like the guidelines from the Ministry of Health\u0026rdquo;\u003c/em\u003e\u003cstrong\u003eRespondent, KII 1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAnother benefit highlighted was that the SDA is applicable in clinical practices by providing guidance on essential procedures that are needed to save maternal and newborn lives.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;\u0026hellip;We need safety in all we do, so this application is welcomed. I was happy to note that it provides concise information that will help nurses and midwives to take good decisions when dealing with childbirth complications\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 6.\u003c/strong\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec19\"\u003e\n \u003ch2\u003eFuture use of the SDA\u003c/h2\u003e\n \u003cp\u003eWhen KII participants were asked their opinions about the SDA cost, scalability, and sustainability, opinions on the future use of the SDA in Rwanda were apparent, along with the aspect of technology application in healthcare and its support in continuous learning. Participants perceived the estimated cost of piloting the SDA as affordable compared to physical training opportunities or online learning resources. They recommended the SDA implementation in consideration of its benefits, particularly the mLearning opportunity.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;The cost is reasonable compared to other online learning tools\u0026hellip;Also physical training has a double cost of time and resources. This application can train many people at the same time depending on individuals\u0026rsquo; needs\u0026hellip;I think this mLearning approach should be promoted in Rwanda as the country is looking to making healthcare services electronic-based\u0026hellip;This application will be one of the solutions\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 5.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOne participant was concerned about the technology cost. But, he indicated that this cost could be alleviated by the fact that many healthcare providers own a smartphone. He suggested that they could use their own smartphone for future use of the SDA.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;The technology cost is a bit high but as many people have smartphones and access to good internet, this might not be a problem. They could download the SDA on their own smartphones\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 6.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eIn addition, participants expressed their wish to see the SDA implemented at a large scale, in all hospitals in Rwanda. However, they recommended that the SDA scalability needs to be preceded by a proper assessment and approvals by all required bodies.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;Technology is in everything we do in healthcare and this application is useful\u0026hellip;For future implementation, you need more sensitization and to involve all key stakeholders like MOH (Ministry of Health), RAM (Rwanda association of midwives), association of gynecologists and pediatricians\u0026hellip;\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I think after in-depth assessment and approval by MOH (Ministry of Health), the whole country (all hospitals) could use this application and it could help us to promote continuous learning while at work\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 3.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAnother participant highlighted the need for possible monitoring and follow-up programs to evaluate if the acquired knowledge and skills are translated into real clinical practices. He also suggested that the monitoring programs could be connected with the existing annual performance contracts and evaluation.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;\u0026hellip;Some mechanisms of monitoring could be put in place to assess if what has been learnt in the SDA is being practiced\u0026hellip;Why not link this monitoring with imihigo (performance contracts)? People should think about it\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 4.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eThen again, participants indicated that the SDA could benefit a large variety of healthcare cadres like medical doctors.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;\u0026hellip;Not only nurses and midwives would benefit from the application, medical doctors can use it \u0026hellip;Everyone needs to know how it works and its importance.\u003c/em\u003e \u003cstrong\u003eRespondent, KII 2.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eMoreover, KII participants perceived the SDA as an easy to use tool with well explained topics. They indicated that the SDA could be sustained because it supports healthcare providers to manage childbirth complications. They also acknowledged the fact that the SDA provides useful information needed in routine clinical practices.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;I think this application will be sustained\u0026hellip;When you see the topics and how they are explained\u0026hellip;This information is what our staff need every day for managing birth complications\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 1.\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eFurthermore, participants suggested additional topics other than topics of BEmONC that could be added to the SDA to make it more useful and sustainable.\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e\u0026ldquo;We have seen some additional topic like COVID 19 and infection prevention in this application. Other topics that are generally relevant for healthcare providers could be added\u0026hellip;I am thinking of topics like: medical bio safety and bio security, patient centered care and quality improvement, ethics in clinical practice\u0026hellip;all these topics could be tied to maternity care\u0026rdquo;\u003c/em\u003e \u003cstrong\u003eRespondent, KII 3.\u003c/strong\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe present study findings indicate that the SDA was acceptable to both the end-users and the key informants. End-users, nurses and midwives, used the SDA frequently, and viewed it as a useful tool that contributes to their professional growth and improve their practice. They frequently used it for education to revise and improve knowledge and skills, and as a decision support tool when immediate guidance was required to manage birth-related complications. Users felt that the SDA has improved their confidence and ability to manage childbirth complications, which they were previously insecure about. These findings are supported by a pre-post intervention study conducted with the same population, which found that the SDA in Rwanda led to a significant increase in nurses and midwives\u0026rsquo; knowledge and skills scores for PPH and NR management six months after SDA introduction [18]. Other researchers similarly indicated that the SDA, when used as a training tool, has shown significant increases in knowledge and skills of different healthcare professionals [25,26]. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe perceived usefulness of the SDA seems to be related to its functionalities when applied to the clinical context of low resource settings [12]. This study emphasized the fact that the SDA offers continuous training opportunity without interrupting health services which is very beneficial due to the shortage of staffs. These findings mirror those of other studies that mHealth applications are increasingly being used in LMICs to improve health professionals\u0026rsquo; access to evidenced and up to date information and training. In maternal and newborn healthcare services, mHealth application are being used as both training tools [27] and decision support tools [28]. The adoption of these applications in resources poor settings depend on their functions that need to be tailored to contexts of interest [29]. Other studies have similarly shown that mLearning are potential solutions to support continuous learning among health professionals in LMICs [30,31]. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAlso, in the current study, the SDA was reported to be practical by helping nurses and midwives translate theoretical knowledge into practice with the support of visual instructions in the animated videos. Then again, with mLearning, the user has the opportunity to study/read multiple times and master the subject. A study in Kenya compared hands-on and video training for PPH management and reported that video training appeared to be as effective as hands-on training [32]. Thus, mHealth applications providing visual instructions via animated videos such as the SDA are potential solutions to improve knowledge and skills of nurses and midwives.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn addition, nurses and midwives had a positive experience with the SDA, they felt confident and better at their job when using the SDA, and reported it as an easy to use, intuitive and empowering tool. Participants were motivated to use the SDA in providing better services and suggested additional features and functions. These findings are in agreement with other studies which reported that the use of mHealth application is perceived as an opportunity for improving the quality of healthcare services with an effect on staff confidence [33], motivation [34], and self-efficacy [35]. Likewise, a realistic review about mHealth and the performance of maternal health care workers in low- and middle-income countries, revealed that health workers can be empowered to adopt and utilize mHealth in contexts where it is aligned to their needs, workload, training, and skills [36]. The review also noted that mHealth can empower health workers with skills and confidence when it is perceived as useful and easy to use [36]. Consequently, the SDA like other mHealth applications is considered an empowering tool that has a positive impact on healthcare delivery processes.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMoreover, in this study, the SDA was highly acceptable by nurses and midwives who admitted to continue using the SDA in their daily clinical practices. This level of user acceptance combined with the documented impact of the SDA is interesting considering the existing gap of ensuring access and adherence to BEmONC evidence-based clinical guidelines in low- and middle-income countries. Several studies have found a low adherence to evidence-based standards for maternal and newborn care particularly in low resource settings [37,38]. Other studies have suggested the use of mHealth to improve the access and adherence to evidence-based guidelines [28,39,40]. Therefore, mHealth tools such as the SDA, which ensures access to the most recent BEmONC evidence-based clinical guidelines, and is demonstrably well accepted by its end-users, may help to improve the adoption of guidelines. The advantages of the SDA are that it is self-explanatory, available in many languages, is open source and free for download, and, once installed on the smartphone, does not require internet to function.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFurthermore, the SDA was appreciated by key informants in the management position at the district hospital level. On the other hand, those key stakeholders acknowledged the benefits of the SDA and were willing to advocate for its scalability for future use in Rwanda. They noted that the cost of piloting the SDA in two district hospitals was reasonable and predicted the possible sustainability of the SDA. However, they recommended a large scale research on the SDA and involvement of the required bodies. Likewise, Eze et al [41] used a stakeholder perspective to analyze existing research on the mHealth process in developing countries, and reported that interactions between system developers and stakeholder groups are important for sustainability of mHealth [41]. Another research documented that \u0026nbsp;in order to realize the potential benefits of mHealth, it is important to identify the relevant stakeholders and how they might be affected [42]. Other studies similarly found that the complexity of introducing mHealth into health care calls for strategies encouraging collaboration between multiple stakeholders to enhance successful implementation [43\u0026ndash;46].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe implications of this study are that the SDA was accepted by both end-users and key stakeholders due to its functions tailored to the Rwandan context. Among the beneficial functionalities of the SDA, there is the capacity of the SDA to function as a training tool and a decision support tool. This increases the confidence and the ability of nurses and midwives in providing quality care during obstetric and neonatal emergencies as well as improved routine obstetric and newborn care. \u0026nbsp;The acceptability and the positive perceptions of the SDA in two district hospitals reflect a possible adoption of the SDA even on a large scale in Rwanda like in other low-resources settings. \u0026nbsp;Evolving efforts for quality maternal and newborn care in Rwanda and similar contexts should consider the integration of SDA as an approach for training and as a clinical decision support tool for BEmONC in order to reduce maternal and newborn mortality. Future work could consider large scale studies of the SDA with maternal and newborn outcomes as primary outcomes.\u003c/p\u003e\n\u003ch2\u003eStrength and limitations of the study\u003c/h2\u003e\n\u003cp\u003eThe study is limited by the small sample size, which reflects the fact that the SDA was implemented only in two district hospitals due to funding limitations. Therefore, the findings of the current study cannot be generalized. Nevertheless, the evidence that the SDA was acceptable and perceived useful by both end-users and key stakeholders at the piloting settings sets the stage for expanding coverage and further testing of the SDA intervention to other hospitals in Rwanda. Additionally, the accuracy of our findings is increased by supporting the acceptability survey with focus group discussions to provide context to and clarify reasons behind some participant responses. \u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe nurses and midwives perceived the SDA as an easy to use, useful, intuitive and empowering tool, which helps them update and revise their knowledge and skills, improves their confidence and makes them feel better to deliver quality obstetric care. Key stakeholders also perceived the SDA as a relevant and useful tool with a reasonable cost and recommended its implementation in routine practices. Overall, both end-users and key informants think that the SDA is beneficial and could be sustained. End-users suggested additional features that would improve the usability of the SDA. These included: a computer/ laptop version of the SDA, a SDA version in Kinyarwanda language, some additional videos, and a SDA discussion forum. Key informants also suggested additional topics that could be added to the SDA such as; medical bio safety and bio security, patient centered care and quality improvement, and ethics in clinical practice. The positive perceptions of the SDA in Rwanda suggest that mLearning and mHealth decision support tools such as the SDA may be a potential way to improve knowledge and skills of nurses and midwives. The mHealth applications like the SDA have the potential to further improve patient care, as they avail evidence-based clinical guidelines and can be easily adapted to contextual needs. Thus, credible, evidence-based, affordable mHealth applications like the SDA are needed to support clinical decision making and provide a platform for updates and continuous learning for nurses and midwives in Rwanda and other limited resource settings. The present study also adds to existing evidence about the benefits of mHealth applications in assuring quality BEmONC.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eBEmONC: Basic Emergency Obstetric and Newborn Care; FGD: Focus Group Discussion; KII: Key informant interview; mHealth: mobile health; mLearning: mobile learning; LMICs: Low- and middle-income countries; NR: Neonatal Resuscitation; PPH: Post-Partum Hemorrhage; RAM: Rwanda Association of midwives; UNICEF: United Nations International Children\u0026apos;s Emergency Fund; UNFPA: United Nations Population Fund, WHO: World Health Organization.\u003c/p\u003e\n"},{"header":"Declarations","content":"\u003ch2\u003eAuthors\u0026rsquo; contributions\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eAN, DNC, MN and, LI were involved in the conception and design of the study. AN drafted the manuscript. DNC and LI assisted with data analysis and interpretation. DNC, MN, and LI reviewed the manuscript. All authors read and approved the final manuscript.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eAcknowledgments\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eWe would like to thank The Ministry of Health, Rwanda, and The Maternity Foundation for supporting this research. We also acknowledge the Consortium for Advanced Research and Training (CARTA) for funding this research.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eFunding\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003e\u003cem\u003eThis research was supported by the Consortium for Advanced Research Training in Africa (CARTA). CARTA is jointly led by the African Population and Health Research Center and the University of the Witwatersrand and funded by the Carnegie Corporation of New York (Grant No--B 8606.R02), Sida (Grant No:54100113), the DELTAS Africa Initiative (Grant No: 107768/Z/15/Z) and Deutscher Akademischer Austauschdienst (DAAD). The DELTAS Africa Initiative is an independent funding scheme of the African Academy of Sciences (AAS)\u0026rsquo;s Alliance for Accelerating Excellence in Science in Africa (AESA) and supported by the New Partnership for Africa\u0026rsquo;s Development Planning and Coordinating Agency (NEPAD Agency) with funding from the Wellcome Trust (UK) and the UK government. The statements made and views expressed are solely the responsibility of the Fellow\u0026quot;. The funding body/bodies played no role in the design of the study and data collection process, analysis, interpretation of data, and in writing the manuscript.\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003ch2\u003eCompeting interests\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eNone declared.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eAvailability of data and materials\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eThe datasets generated for this study will be made available from the corresponding author on a reasonable request.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eConsent for publication\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eNot applicable.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eEthics approval and consent to participate\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eEthical approvals were obtained from the Human Research Ethics Committee of the University of the Witwatersrand (M190258) and the University of Rwanda, College of Medicine and Health Sciences\u0026rsquo; Institutional Review Board (No. 377/CMHS IRB/2018). All participants gave written informed consent and consents for recording the interviews prior to their participation. Participants were assured that their identities would remain anonymous in all reporting of the study and that their personal information would be kept confidential. \u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eOrganization WH. Trends in maternal mortality 2000 to 2017: estimates by WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division. 2019; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://apps.who.int/iris/handle/10665/327595\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHug L, Alexander M, You D, Alkema L, for Child UNIG. National, regional, and global levels and trends in neonatal mortality between 1990 and 2017, with scenario-based projections to 2030: a systematic analysis. Lancet Glob Heal. 2019;7(6):e710\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAmeh C, Msuya S, Hofman J, Raven J, Mathai M, van den Broek N. Status of Emergency Obstetric Care in Six Developing Countries Five Years before the MDG Targets for Maternal and Newborn Health. Frasch MG, editor. PLoS One [Internet]. 2012 Dec 6 [cited 2017 Apr 10];7(12):e49938. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://dx.plos.org/10.1371/journal.pone.0049938\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBhutta ZA, Das JK, Bahl R, Lawn JE, Salam RA, Paul VK, et al. Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost? Lancet. 2014;384(9940).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOtolorin E, Gomez P, Currie S, Thapa K. Essential basic and emergency obstetric and newborn care: From education and training to service delivery and quality of care. Int J Gynecol Obstet [Internet]. 2015 [cited 2017 Apr 5];130:S46\u0026ndash;53. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.sciencedirect.com/science/article/pii/S0020729215001368\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUNFPA. Setting standards for emergency obstetric and newborn care | UNFPA - United Nations Population Fund [Internet]. 2014 [cited 2017 Aug 23]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.unfpa.org/resources/setting-standards-emergency-obstetric-and-newborn-care\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTull K. Inequity and Leave No One Behind: Approaches to Maternal and Newborn Healthcare. 2019 Feb 27 [cited 2019 Apr 22]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://opendocs.ids.ac.uk/opendocs/handle/123456789/14408\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJoannah H RR. What is quality in maternal and neonatal health care? Midwifery [Internet]. 2012 [cited 2017 Oct 24];28(5):e676\u0026ndash;83. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.sciencedirect.com/science/article/pii/S0266613811001367\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSharma G, Mathai M, Dickson KE, Weeks A, Hofmeyr GJ, Lavender T, et al. Quality care during labour and birth: a multi- country analysis of health system bottlenecks and potential solutions. BMC Pregnancy Childbirth [Internet]. 2015 [cited 2017 Apr 7];15(2):S2. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.biomedcentral.com/1471-2393/15\u003c/span\u003e\u003c/span\u003e/S2/S2\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNeeds S, Health C, Philbrick WC. mHealth and MNCH: State of the Evidence. Mhealth Alliance, Un Found [Internet]. 2013;(January):1\u0026ndash;45. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.mhealthknowledge.org/sites/default/files/15_un_007_evidencegapreport_digital_aaa.pdf\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeroz A, Perveen S, Aftab W. Role of mHealth applications for improving antenatal and postnatal care in low and middle income countries: A systematic review. BMC Health Serv Res. 2017;17(1).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMaternity Foundation M for mothers. What is the SDA app - Maternity Foundation [Internet]. 2014 [cited 2018 Jul 26]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.maternity.dk/about-the-app/what-is-the-app/\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWebsite MF. Our Work - Maternity Foundation [Internet]. [cited 2020 Jul 23]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.maternity.dk/our-work/\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSayinzoga F, Bijlmakers L, van Dillen J, Mivumbi V, Ngabo F, van der Velden K. Maternal death audit in Rwanda 2009-2013: a nationwide facility-based retrospective cohort study. BMJ Open [Internet]. 2016 [cited 2016 Jul 24];6(1):e009734. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.ncbi.nlm.nih.gov/pubmed/26801466\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKasine Y, Babenko-Mould Y, Regan S. Translating continuing professional development education to nursing practice in Rwanda: Enhancing maternal and newborn health. Int J Africa Nurs Sci [Internet]. 2018 Jan 1 [cited 2018 Jul 20];8:75\u0026ndash;81. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.sciencedirect.com/science/article/pii/S221413911730063X\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNishimwe A, Nyssen M, Ibisomi L, Nozizwe Conco D. Clinical decision making in basic emergency obstetric and newborn care among nurses and midwives: the role of the safe delivery mhealth application_pre-post-intervention study (research protocol). Informatics Heal Soc Care. 2021;1\u0026ndash;11.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNishimwe A, Conco DN, Nyssen M, Ibisomi L. Context specific realities and experiences of nurses and midwives in basic emergency obstetric and newborn care services in two district hospitals in Rwanda: a qualitative study. BMC Nurs [Internet]. 2022;21(1):9. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s12912-021-00793-y\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNishimwe A, Ibisomi L, Nyssen M, Conco DN. The effect of an mLearning application on nurses\u0026rsquo; and midwives\u0026rsquo; knowledge and skills for the management of postpartum hemorrhage and neonatal resuscitation: pre\u0026ndash;post intervention study. Hum Resour Health. 2021;19(1):1\u0026ndash;10.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMOH RHP. REPUBLIC OF RWANDA. 2015 [cited 2017 Aug 7]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.moh.gov.rw/fileadmin/templates/policies/Health_Sector_Policy___19th_January_2015.pdf\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRwanda NI of S of, Planning/Rwanda M of F and E, Health/Rwanda M of, International I. Rwanda Demographic and Health Survey 2014-15 [Internet]. 2016 [cited 2019 Nov 14]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://dhsprogram.com/publications/publication-fr316-dhs-final-reports.cfm\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePatton MQ. Two decades of developments in qualitative inquiry: A personal, experiential perspective. Qual Soc Work. 2002;1(3):261\u0026ndash;83.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ehttp://www.ncbi.nlm.nih.gov/pubmed/25326092\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePoland BD. Transcription quality as an aspect of rigor in qualitative research. Qual Inq. 1995;1(3):290\u0026ndash;310.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Heal care. 2007;19(6):349\u0026ndash;57.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLund S, Boas IM, Bedesa T, Fekede W, Nielsen HS, S\u0026oslash;rensen BL. Association Between the Safe Delivery App and Quality of Care and Perinatal Survival in Ethiopia: A Randomized Clinical Trial. JAMA Pediatr [Internet]. 2016;170(8):765\u0026ndash;71. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.ncbi.nlm.nih.gov/pubmed/27322089\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBolan NE, Sthreshley L, Ngoy B, Ledy F, Ntayingi M, Makasy D, et al. mLearning in the Democratic Republic of the Congo: A Mixed-Methods Feasibility and Pilot Cluster Randomized Trial Using the Safe Delivery App. Glob Heal Sci Pract [Internet]. 2018 Dec 27 [cited 2019 Apr 22];6(4):693\u0026ndash;710. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.ncbi.nlm.nih.gov/pubmed/30591577\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMichel-Schuldt M, Dayon MB, Klar RT, Subah M, King-Lincoln E, Kpangbala-Flomo C, et al. Continuous professional development of Liberia\u0026rsquo;s midwifery workforce\u0026mdash;A coordinated multi-stakeholder approach. Midwifery. 2018;62:77\u0026ndash;80.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAdepoju I-OO, Albersen BJA, De Brouwere V, van Roosmalen J, Zweekhorst M. mHealth for Clinical Decision-Making in Sub-Saharan Africa: A Scoping Review. JMIR mHealth uHealth [Internet]. 2017 Mar 23 [cited 2019 Sep 15];5(3):e38. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://mhealth.jmir.org/2017/3/e38/\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmed MAA, Gagnon M-P, Hamelin-Brabant L, Mbemba GIC, Alami H. A mixed methods systematic review of success factors of mhealth and telehealth for maternal health in Sub-Saharan Africa. Mhealth. 2017;3.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrusamento S, Kyaw BM, Whiting P, Li L, Car LT. Digital health professions education in the field of pediatrics: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(9):e14231.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDunleavy G, Nikolaou CK, Nifakos S, Atun R, Law GCY, Car LT. Mobile digital education for health professions: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(2):e12937.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNilsson C, S\u0026oslash;rensen BL, S\u0026oslash;rensen JL. Comparing hands-on and video training for postpartum hemorrhage management. Acta Obstet Gynecol Scand. 2014;93(5):517\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTaekman JM, Foureman MF, Bulamba F, Steele M, Comstock E, Kintu A, et al. A novel multiplayer screen-based simulation experience for African learners improved confidence in management of postpartum hemorrhage. Front public Heal. 2017;5:248.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZakane SA, Gustafsson LL, Tomson G, Loukanova S, Si\u0026eacute; A, Nasiell J, et al. Guidelines for maternal and neonatal \u0026ldquo;point of care\u0026rdquo;: needs of and attitudes towards a computerized clinical decision support system in rural Burkina Faso. Int J Med Inform. 2014;83(6):459\u0026ndash;69.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWalker DM, Holme F, Zelek ST, Olvera-Garc\u0026iacute;a M, Montoya-Rodr\u0026iacute;guez A, Fritz J, et al. A process evaluation of PRONTO simulation training for obstetric and neonatal emergency response teams in Guatemala. BMC Med Educ. 2015;15(1):117.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbejirinde IOO, Ilozumba O, Marchal B, Zweekhorst M, Dieleman M. Mobile health and the performance of maternal health care workers in low- and middle-income countries: A realist review. Vol.\u0026nbsp;21, International Journal of Care Coordination. 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMiller S, Abalos E, Chamillard M, Ciapponi A, Colaci D, Comand\u0026eacute; D, et al. Beyond too little, too late and too much, too soon: a pathway towards evidence-based, respectful maternity care worldwide. Lancet. 2016;388(10056):2176\u0026ndash;92.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRitchie LMP, Khan S, Moore JE, Timmings C, van Lettow M, Vogel JP, et al. Low-and middle-income countries face many common barriers to implementation of maternal health evidence products. J Clin Epidemiol. 2016;76:229\u0026ndash;37.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRunnacles J, Rouech\u0026eacute; A, Lachman P. The right care, every time: improving adherence to evidence-based guidelines. Arch Dis Childhood-Education Pract. 2018;103(1):27\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChawla D, Thukral A, Kumar P, Deorari A. Harnessing mobile technology to deliver evidence-based maternal-infant care. In: Seminars in Fetal and Neonatal Medicine. Elsevier; 2021. p.\u0026nbsp;101206.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEze E, Gleasure R, Heavin C. Reviewing mHealth in developing countries: A stakeholder perspective. Procedia Comput Sci. 2016;100:1024\u0026ndash;32.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePetersen C, Adams SA, DeMuro PR. mHealth: don\u0026rsquo;t forget all the stakeholders in the business case. Med 20. 2015;4(2).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMaar MA, Yeates K, Perkins N, Boesch L, Hua-Stewart D, Liu P, et al. A framework for the study of complex mHealth interventions in diverse cultural settings. JMIR mHealth uHealth. 2017;5(4):e47.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBally ELS, Cesuroglu T. Toward integration of mHealth in primary care in the Netherlands: a qualitative analysis of stakeholder perspectives. Front public Heal. 2020;7:407.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFranz-Vasdeki J, Pratt BA, Newsome M, Germann S. Taking mHealth solutions to scale: enabling environments and successful implementation. J Mob Technol Med. 2015;4(1):35\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDharmayat KI. Sustainability of \u0026lsquo;mHealth\u0026rsquo;interventions in sub-Saharan Africa: a stakeholder analysis of an electronic community case management project in Malawi. Malawi Med J. 2019;31(3):177\u0026ndash;83.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-nursing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"nurs","sideBox":"Learn more about [BMC Nursing](http://bmcnurs.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/nurs/default.aspx","title":"BMC Nursing","twitterHandle":"@BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"BEmONC, nurses and midwives, mHealth, Safe Delivery Application, Rwanda","lastPublishedDoi":"10.21203/rs.3.rs-1260261/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1260261/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Innovative use of mobile health (mHealth) technology in timely management of childbirth complications is a promising strategy, but its evidence base is limited. The Safe Delivery mHealth Application (SDA) is one of the recent mhealth applications (loaded in smartphones) which is a clinical decision support and training tool for basic emergency obstetric and newborn care (BEmONC). This paper describes, the health providers’ experiences, perceptions, and acceptability of using the SDA, as well as the perceptions of key stakeholders. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e A mixed-methods approach was utilized. Quantitative methods consisted of a self-reported acceptability survey, administered to 54 nurses and midwives, including questions on their usage and perceptions of the SDA. Descriptive statistics were employed to analyze the survey data. Qualitative methods included two focus group discussions with 24 nurses and midwives, and six key informant interviews with stakeholders (maternity matrons, responsible for maternal and child health, and district hospital managers). Thematic analysis was performed and selected quotations used to illustrate themes. The study took place in two district hospitals in Rwanda.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e Quantitative results found that 31 (57.4%) participants used the SDA four to six times per week. Many participants felt more confident (53.7%) and better at their job (40.7%) since having the SDA. Likert scale survey responses (1-5, 1 =Strongly Disagree, 5 =Strongly Agree) indicated general agreement that SDA is easy to use (Mean =4.46), is an effective decision support tool (4.63), and training tool (4.65). Qualitative results included themes on perceived usefulness; professional growth acquired through the use of the SDA; SDA, an empowering, intuitive, and user-friendly technology; desired SDA features and functions; benefits of SDA as perceived by key informants, and future use of the SDA. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003eConclusion:\u003c/strong\u003e The nurses and midwives perceive the SDA as having improved their ability to manage childbirth complications. Key stakeholders also perceive the SDA as a useful tool with a reasonable cost and recommend its implementation in routine practices. This study deepens the understanding of the potential benefits of mHealth such as the SDA in low-income settings, like Rwanda. It also provides more evidence on the impact of mHealth in assuring quality BEmONC.\u003c/p\u003e","manuscriptTitle":"A mixed-method study exploring experiences, perceptions, and acceptability of using a Safe Delivery mHealth Application in two district hospitals in Rwanda","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-01-27 21:09:53","doi":"10.21203/rs.3.rs-1260261/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2022-06-08T10:48:26+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-06-06T09:46:10+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-04-16T21:02:21+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-04-11T17:15:46+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"3689e055-4323-4000-bb24-9e0628609301","date":"2022-04-11T16:35:06+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"8dffeed1-2cf3-4adf-bf07-aa31d2b6d9a3","date":"2022-04-11T13:03:45+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2022-04-11T09:39:46+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2022-04-06T20:03:42+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2022-02-01T07:27:49+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2022-01-24T17:15:15+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Nursing","date":"2022-01-14T10:05:27+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-nursing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"nurs","sideBox":"Learn more about [BMC Nursing](http://bmcnurs.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/nurs/default.aspx","title":"BMC Nursing","twitterHandle":"@BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"de9e2931-021c-448d-886c-5fac67bdcae1","owner":[],"postedDate":"January 27th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2022-06-21T10:59:22+00:00","versionOfRecord":[],"versionCreatedAt":"2022-01-27 21:09:53","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1260261","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1260261","identity":"rs-1260261","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}