Acceptability and usability of a digital tool to support long-lasting insecticide-treated net distribution in Northern Bahr el Ghazal State, South Sudan

Acceptability and usability of a digital tool to support long-lasting insecticide-treated net distribution in Northern Bahr el Ghazal State, South Sudan | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Acceptability and usability of a digital tool to support long-lasting insecticide-treated net distribution in Northern Bahr el Ghazal State, South Sudan Jamshed Khan, Denis Mubiru, Elisabeth G. Chestnutt, Louise Cook, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4344384/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Oct, 2024 Read the published version in Malaria Journal → Version 1 posted 11 You are reading this latest preprint version Abstract Background : Long-lasting insecticide-treated nets (LLINs) have been the backbone of malaria prevention for decades. In South Sudan, LLINs are typically distributed by volunteers who use paper-based systems to collect distribution data. Paper-based systems are simple to use but have a higher occurrence of data inaccuracies and can hinder the timely use of data for decision making. In 2022, a digital tool was introduced to collect data during the LLIN campaign in Northern Bahr el Ghazal (NBeG). The tool aimed to improve the accuracy of data entry and enable data to be use in real-time for decision making during the campaign. This study assessed the acceptability and usability of the digital tool. Methods : A questionnaire containing open and closed questions was conducted with users of the digital tool, supervisors and other key stakeholders in five counties of NBeG. The questionnaire was administered using Malaria Consortium’s Projects Results System Android mobile application. Usability was determined through a modified and validated System Usability Scale (SUS) approach and acceptability was assessed by responses to open questions. Results: A total of 93 participants responded to the usability and acceptability questionnaire. The mean (±standard deviation) usability score across 10 SUS-scoring items was 60.91 (12.87), indicating a moderate level of usability. The majority of users reported the tool was useful for managing the LLIN distribution workflow, was easy to use, reduced workload, and supported stock management and real-time campaign monitoring . There was no significant difference in the usability scores across genders, roles, and counties. The digital tool was perceived to be acceptable, and the majority of respondents with experience of both paper-based and the digital tool reported a preference for the digital tool over paper-based systems. The majority of respondents also said they would recommend the digital tool to colleagues. Conclusion : Digital tools are perceived to be beneficial for collecting data during LLIN campaigns, even in remote areas where network coverage is challenging. Additional improvements can be implemented to overcome operational challenges and improve usability of the tool. Further study is needed to determine the impact of the digital tool on data quality and real-time data use. Long-lasting insecticide-treated nets Digitalisation South Sudan System Usability Scale Figures Figure 1 Figure 2 Figure 3 Figure 4 Background Malaria is a serious public health problem, predominantly affecting countries in sub-Saharan Africa. In South Sudan, malaria accounts for 66% of outpatient consultations, 50% of inpatient consultations and approximately 30% of all mortality ( 1 ). The Ministry of Health and National Malaria Control Programme in South Sudan have developed a National Malaria Strategy which includes activities to prevent, diagnose and treat malaria, including the distribution of long-lasting insecticide-treated nets (LLINs) ( 2 ). LLINs are a proven method for preventing malaria, and have been the backbone of malaria control for over two decades ( 3 ). LLINs are typically distributed through mass campaigns and continuous distribution channels at antenatal care and EPI clinics. In South Sudan, LLIN distribution campaigns are conducted every three years, as recommended by the World Health Organization ( 4 ). During the campaigns, LLINs are typically distributed by volunteers, and LLIN campaign distribution data are collected using a paper-based system. Although paper-based methods are simple to use, handwritten data entry can result in inaccuracies at collection and collation stages and data reporting is slower, which hinders timely decision making ( 5 ). Findings from the introduction of an electronic data system in seven African counties showed electronic data systems can improve data completeness, facilitate data submission and analysis, and reduce the time between data entry and data reporting, when compared to paper-based systems ( 6 ). Despite the benefits of electronic data systems, these digital tools have not previously been introduced in South Sudan due to a perception that volunteers with low literacy levels and limited experience with technology would be unable to use them. Internet connectivity is also a challenge only 10.9% of the population in South Sudan have access to the internet, which limits the use of digital tools in remote areas ( 7 ). To improve data accuracy and timely decisions, a digital application was developed to collect data during the LLIN campaign. To demonstrate the usability and feasibility in South Sudan, the application was designed with two key requirements: offline functionality, so data can be collected without the internet, and interoperability with the Digital Health Information System 2 (DHIS2), to improve real-time decision making. In 2022, a small-scale pilot was planned, and 106 volunteers were trained to use the application for collecting data during the LLIN campaign in Central Equatoria. After completion of the campaign, interviews were conducted with 13 volunteers to gather feedback. The study showed volunteers were able to use the digital tool to correctly upload data directly to DHIS2. In 2023, a LLIN campaign was carried out in Northern Bahr el Ghazal and 965 volunteers were trained to use the digital tool. Following the campaign, a survey was conducted to assess the feasibility and usability of the tool, and to identify challenges and lessons from the implementation. This paper outlines the findings from this work. Methods Description of the digital tool The LLIN distribution system was developed using the DHIS2 tracker. The tool was created in DHIS2 version 2.38, using capture application version 2.8 to train end-users. Key variables were identified and agreed upon between the Ministry of Health and The Global Fund to fight AIDS, Tuberculosis, and malaria to enable both parties to view the locations of the LLIN distribution using the Global Positioning System (GPS) and household information as an accountability mechanism. By selecting all geographical information from a dropdown list, the digital tool generates unique household numbers and by entering total number of household members, the tool generates the number of LLINs to be distributed to that household. The tool also provides geo-mapping of LLINs that have been distributed in each location and summary reports are automatically generated, collated and uploaded to the DHIS2 dashboard which is visible to decision makers, such as the Ministry of Health. Real-time reporting allows decision-makers to identify issues and mobilise a response during the campaign. The digital tool consists of a database of state-, county-, payam- (the administrative level below county containing a minimum 25,000 people) and boma- (the lowest administrative level consisting of a collection of 4–8 villages) level information for the whole South Sudan that can be easily selected from dropdown list. Study design This study was designed to assess the usability and feasibility of deploying a digital tool in Northern Bahr el Ghazal. A questionnaire was designed to gather responses from users on the usability and acceptability of the tool. The questionnaire consistent of two components; a modified system usability scale (SUS) approach to assess usability, and three open questions, to gather feedback on the acceptability of the tool ( 8 ). Study setting Northern Bahr el Ghazal state is part of the Greater Bahr el Ghazal region. The state has a total area of 30,543 km 2 and is made up of five counties: Aweil South, Aweil East, Aweil West, Aweil North and Aweil Centre (Fig. 1 ). Ethnically, most of the state’s population is composed of Dinka and Jurchol tribe members, with a minority of Luo tribe members. The main means of livelihood are agriculture and livestock farming for the two tribes. Floods occur annually from June to November and hinder routine life, causing internal displacements. The malaria prevalence in Northern Bahr el Ghazal in 53%, the highest of any state in South Sudan, and around 66% of the population have access to LLINs ( 9 ). Northern Bahr el Ghazal also has the lowest literacy rate in the country, with estimates suggesting only 21% of the population aged fifteen years and above are literate ( 10 ). LLIN distribution implementation The digitalised campaign was carried out in all five counties of Northern Bahr el Ghazal state from March to June 2023. A total of 965 volunteers (Table 1 ) including Registrars, Site Managers, Payam Supervisors, and County Health Department and State Ministry of Health staff were trained to use the digital tool. The LLIN campaign was conducted using a house-to-house distribution method and COVID-19 protocols were observed. Table 1 Participants’ distribution across survey in all counties. County Registrars Site Manager Payam Supervisor SMOH CHD Total Sample size Aweil Centre 45 11 6 1 1 64 5 Aweil East 330 66 8 1 1 406 41 Aweil West 178 36 8 1 1 224 19 Aweil South 77 15 8 1 1 102 10 Aweil North 135 27 5 1 1 169 18 Total 765 155 35 5 5 965 93 SMOH: State Ministry of Health staff, CHD: County Health Department staff Sampling and data collection The study questionnaire (Additional file 1) was designed and deployed using Malaria Consortium’s Projects Results System mobile application and administered to participants in the field by Malaria Consortium and County Health Department staff. A minimum sample size of 80 participants was determined to be sufficient for the study, assuming expected mean usability score (standard deviation) of 68 (12.5) based on a previous study ( 11 ). The sample size was calculated to determine the mean SUS score with a 95% level of confidence, precision of ± 3 and non-response rate of 10%. The questionnaire containing the SUS rating scale was administered to a total 93 respondents (Table 1 ). Respondents were randomly selected using random sample selection from a pool of 965 potential participants who were trained to use the digital tool. Adaptations were made to the original 10-item SUS Likert-type rating scale, which combines an equal number of negatively and positively framed Likert items for assessing system usability. Additionally, the modified version of the scale incorporated three open questions to probe and contextualise responses. The adapted SUS rating scale was piloted and validated with 20 respondents who did not participate in the main study. A principal component analysis was used to assess the reliability and interval validity of the adapted SUS tool. The SUS tool was deemed to have good reliability and internal validity based on satisfactory component loadings of each of the 10 items in the usability correlation matrix, Cronbach’s alpha values (ranging from 0.63 to 0.72) and Eigenvalues ( 12 ). The SUS rating scale covered a variety of aspects of system usability, such as ease of use, need for support, complexity and perceived usefulness of the tool. The questionnaire also enabled the collection of data on participants’ characteristics, including location, gender and role. The survey was carried out by seven enumerators from Malaria Consortium’s field team based in Northern Bahr el Ghazal state through visits to data collection sites. Accounts were created for each of the survey participants, and enumerators provided participants with a tablet to complete the survey and then log out of their account. In addition to the SUS rating, open questions were used to collect information on user perceptions and acceptability of the tool. The open questions covered three areas: i) challenges experienced using the digital tool, ii) suggestions for improvement, and iii) overall perception and experience with the digital tool compared with paper-based systems. Data analysis Descriptive statistics were used to summarise participant characteristics, expressed as frequencies and percentages for categorical variables, and means and standard deviations for continuous variables. Individual usability scores were calculated for each item and participant, which were pooled to generate mean usability scores and standard deviations across the entire study sample in accordance with the SUS scoring framework ( 8 ). According to this framework, SUS scores have a range of 0 to 100, computed based on the 10 items in the rating scale covering a variety of aspects of system usability, with scores of > 72.5, 62.7–72.5, 51.7–62.6 and < 51.7 considered as excellent, good, ok and poor, respectively ( 13 ). Differences in SUS scores across participant characteristics were assessed using ANOVA tests of comparison of means. Statistical significance was determined at p value < 0.05. Statistical analyses were conducted using Stata (version 16)( 14 ). Responses from the open questions covering acceptability were grouped by theme. Results During the LLIN campaign a total of 773,387 LLINs were distributed to households across the five counties (Additional file 2). Participants’ characteristics A total of 93 individuals participated in the usability and acceptability assessments. Respondent’s characteristics are summarised in Table 2 and further information provided in Additional file 3. Table 2 SUS score distributions across respondents’ characteristics Variable No. of respondents Mean (± SD) SUS scores # P-value* County Aweil Central 5 73.00 (4.11) 0.542 Aweil East 41 57.32 (14.98) Aweil North 18 64.31 (5.61) Aweil South 10 72.25 (3.99) Aweil West 19 56.32 (11.44) Gender Female 3 50.83 (17.74) 0.204 Male 90 61.25 (12.67) Role CHD 5 73.00 (4.11) 0.062 Manager 21 60.71 (9.91) Payam supervisor 10 68.25 (8.90) Registrar 56 58.66 (14.09) SMOH 1 57.50 Total 93 60.91 (12.87) CHD: County health department, SMOH: State ministry of health, SD: standard deviation #mean SUS scores have a range of 0 to 100 computed based on the 10 items in the rating scale covering a variety of aspects of system usability *ANOVA test of comparison of means Table 2 presents the composite SUS score for the study sample and distribution of scores by participant characteristics. The mean (± SD) usability score across the 10 SUS-scoring items was 60.91 (12.87) for the total study sample. Based on the framework ( 13 ) for interpreting SUS scores, this represents an ‘ok’ level of usability. The majority of users found the digital tool was easy to use, reduced workload, helped in stock management, and facilitated real time campaign monitoring. Usability did not vary significantly by county (p = 0.542), gender (p = 0.204) or role (p = 0.062). Summary of item-level participants’ usability responses Over 85% of participants responded either ‘agree’ or ‘strongly agree’ to the positively framed questions (Fig. 2 ). The highest scoring statement was the tool improved the quality of reporting, followed closely by respondents reporting the tool was easy to use. In addition, respondents disagreed with the majority of the negatively framed questions (Fig. 3 ). The only negatively framed question where the majority of participants answered ‘agree’ or ‘strongly agree’ was regarding improvements to the tool. When asked to compare the digital tool with the former paper-based system the majority of participants (65%) did not respond to the question (Fig. 4 ). However, of the 32 responses, 94% reported the digital tool was an improvement on the paper-based system with 69% reporting it was very good and much improved. Acceptability of the tool Table 3 summarises participants’ responses on the acceptability of the digital tool. All participants responded to at least one of the open questions, however, all participants did not respond to all the questions, and many responses were not detailed. Overall, respondents with experience of both paper-based and the digital tool reported preferring the digital tool for use during LLIN distribution campaigns. In addition, users reported the tool was user-friendly, made their work easier and reduced their workload. The most frequently mentioned challenges were the GPS functionality, which respondents reported as having some issues loading, charging the device and the digital form, which respondents suggested could be shorter. Overall, the majority of respondents considered the tool an improvement and said they would recommend it for future use. Table 3 Thematic summary of participants’ perceptions and acceptability of the digital tool Question Percentage analysis 1. Perceived challenges that can affect functionality of the digital tool Out of 93 participants: • 50% either did not respond or did not report any challenges while using this tool. • 27% mentioned limited access to power source for charging tablet batteries. • 13% mentioned GPS reading caused delays. • 9% responded poor or no network as a challenge. • Only 1% mentioned short training session as challenge. 2. Suggested ways to improve the tool Out of 93 participants: • 41% did not respond or did not mention any suggestion while using this tool. • 17% requested stronger battery charging options. • 12% praised the tool as good to use and suggest for scale up. • 12% suggested designing the tool to entirely work offline. • 9% suggested to reduce or adjust GPS reading time. • 8% suggested to shorten form in the tool to avoid delays. • 2% suggested the need for more internet bundles. 3. Perception on the use of the tool compared with paper-based method Out of 93 participants • 37% did not respond or chose to remain neutral while using this tool. • 57% appreciated the tool as simple, easy to use and reduced workload. • 2% were engaged in paper-based campaign and 100% of them preferred digital tool over paper-based system. • 4% reported other issues with GPS and internet bundles. Discussion The aim of this study was to investigate the feasibility of introducing a digital data collection system for LLIN campaigns in South Sudan. The study demonstrated the tool has a moderate level of usability among respondents. Although the score is below the globally accepted threshold for high usability (68.5), achieving even a moderate usability score contradicts the assumption that volunteers with low literacy levels and limited experience with technology would be unable to use digital tools ( 11 ). This usability score is also supported by responses to the open questions, which reported the majority of participants found the tool simple and easy to use. In addition, the majority of respondents who had experience with both paper-based and digital data collection systems reported a preference for the digital tool. These findings are consistent with similar studies in Benin and the Democratic Republic of the Congo where the introduction of digital data collection tools during LLIN campaigns were also found to be valuable ( 15 , 16 ). The study also identified several challenges and contextual factors influencing the tool’s usability, acceptability and overall user experience. The most commonly reported challenges were internet connectivity and issues charging the devices as the capacity of the power banks supplied was not high enough to charge the devices. Despite the responses regarding poor connectivity, the tool did not require internet connectivity to function, when probing further respondents mentioned they perceived the slow device loading was due to connectivity issues. Volunteers incorrectly assumed this is problem with internet connectivity rather than the device itself. Additional training may be useful to improve volunteer’s understanding of how the tool works and to enable them to troubleshoot any issues. Furthermore, power supply limitations can be remedied by providing higher-capacity power banks and mobile charging systems, which could be solar powered. Another area suggested for improvement was to shorten the digital form, however this form is based on standard reporting templates for LLIN campaigns and therefore is unlikely to change. To enhance the tool’s usability, acceptability and overall benefits, it is imperative to tailor its functionality and future deployment strategies to overcome the challenges and contextual constraints identified. Study strengths, limitations, and implications for future research The SUS scoring method provides a quick and highly efficient way of assessing usability. It is easy to apply and is proven to quickly measure user perception with considerable precision. Notwithstanding the strength of the methods used, the study has some limitations worth acknowledging. While the data collection tool enabled the collection of open and closed responses on usability and user perceptions, the open responses were brief and not detailed enough to provide deeper insights for contextualising the quantitative SUS scores. Secondly, SUS scores rely on respondent's perceptions and judgements which may not always be objective. Consequently, responses are prone to social desirability bias. The study’s lack of a control group poses another limitation. A controlled study design would have enabled the comparison of usability between digital and paper based LLIN campaigns and would have provided more informative insight on the comparative advantage of digitalising public health interventions like LLIN distribution. The study’s ability to determine significant differences in usability was constrained by sample size limitations. Further studies should consider using a larger sample to explore variations in SUS scores across user-level and contextual characteristics. The disparity in the distribution of study participants’ gender, with an underrepresentation of women is perhaps a reflection of the gender imbalance in LLIN digitalisation and distribution campaigns in the study locations. Gender gaps in digital health interventions, if left unaddressed, have the potential to undermine the impact of digital tools and exacerbate existing inequities ( 17 ). The gender imbalance seen in this study therefore warrants efforts to involve female stakeholders across the entire value-chain of public health interventions. In addition, the usability of digital tools is influenced by a complex interplay of cultural, technological, economic, and broader contextual factors which can affect SUS scores. Data on other factors that could potentially influence SUS scores, such as those relating to users’ knowledge and education levels, were not captured and considered as covariates in the analysis. This makes it difficult to compare results with globally defined thresholds and usability scores in other study settings. Furthermore, the findings from this study may not be generalisable to the whole of South Sudan due to the unique context of Northern Bahr el Ghazal state. Finally, this study investigated the usability of the digital tool from users' perspectives and was not designed to assess of the effectiveness of the tool and its ability to improve data quality or optimise decision-making, which requires further research. Conclusion This study demonstrates that digitalising LLIN distribution campaigns across South Sudan is feasible, including in remote rural areas like Northern Bahr el Ghazal. Digitising these campaigns has the potential to improve accurate data entry, timely data reporting and real-time decision making which in turn could help to optimise the delivery of LLINs. Further work is needed to address the challenges identified during this study including changes to the tool that could improve user experience. In addition, further research assessing the degree to which digitisation improves data quality and decision making during campaigns would be valuable. Abbreviations CHD County health department DHIS 2 Digital Health Information System 2 GPS global positioning system LLIN Long-lasting insecticide-treated net NBeG Northern Bahr el Ghazal PReS Project results system SD standard deviation SMOH State ministry of health SUS system usability scale Declarations Availability of data and materials Processed data supporting the findings of this study are included in this published article and its supplementary information files. Original datasets generated and analysed during the current study are available from the corresponding author on reasonable request. Funding This study was funded through Global Fund funding and UNICEF as fund manager, for LLIN received by Malaria Consortium, primarily for the distribution of digitalised LLIN campaign. Acknowledgements We are very grateful to all of the study participants, UNICEF, volunteers, Boma Health Workers, National Malaria Control Programme, National Ministry of Health, State Ministry of Health, county health departments, political and technical leaders of Aweil Centre, Aweil West, Aweil East, Aweil North and Aweil South counties, and to the Boma health teams involved in the distribution of LLIN. Authors’ contributions JK: study conceptualisation, planning, supervision, developed study protocol, data analysis, visualisation, writing – original manuscript and editing, data LC: writing – review, formatting, coordinating writing, JO: supervision of data collection, data analysis, visualisation and interpretation, and editing. CN: study design, data analysis, visualisation, and interpretation, writing – results of manuscript and review of manuscript. DM: study conceptualisation and supervision, writing – review. LLR: training coordination, data collection, fieldwork coordinator and supervisor, writing – review. FO: training coordination, data collection, fieldwork coordinator and supervisor, review. LC: study conceptualisation, writing – review MA: field work coordinator and supervisor, writing – review . KC: principal investigator, study design, writing – review. Ethics approval and consent to participate This study received ethical approval from ministry of health. Data were used in accordance with the MOH’s ethics standards. Informed consent was obtained from all study participants before data collection. Consent for publication All authors read and approved the final manuscript. Competing interests KC is the director general for policy, planning, budget, research, monitoring and evaluation of South Sudan Ministry of Health. All other authors declare no competing interests. References Pasquale HA. Malaria prevention strategies in South Sudan. South Sudan Med J. 2020;13(5):187–90. [South Sudan] National Malaria Control Programme. National Malaria Strategic Plan 2021–2025. [South Sudan] National Malaria Control Programme; 2020. World Health Organization. World Malaria Report 2022. Geneva: World Health Organization; 2022. World Health Organization. Achieving and maintaining universal coverage with long-lasting insecticidal nets for malaria control. Geneva: World Health Organization; 2017. Ley B, Rijal KR, Marfurt J, Adhikari NR, Banjara MR, Shrestha UT, et al. Analysis of erroneous data entries in paper based and electronic data collection. BMC Res Notes. 2019;12(1):537. Burnett SM, Wun J, Evance I, Davis KM, Smith G, Lussiana C, et al. Am J Trop Med Hyg. 2019;100(4):889–98. Introduction and Evaluation of an Electronic Tool for Improved Data Quality and Data Use during Malaria Case Management Supportive Supervision. Kemp SD. 2022: South Sudan: Data Reportal; 2022. https://datareportal.com/reports/digital-2022-south-sudan#:~:text=There%20were%201.25%20million%20internet,percent)%20between%202021%20and%202022.%202022 . [Accessed: 5th February 2024]. Brooke J. SUS: A quick and dirty usability scale. Usability Eval Ind. 1995;189. South Sudan Ministry of Health. Republic of South Sudan Malaria Indicator Survey 2017. Juba: South Sudan Ministry of Health,; 2017. Knoema. Northern Bahr el Ghazal - Literacy, Can read and write (15+): Knoema; 2009. https://knoema.com/atlas/South-Sudan/Northern-Bahr-el-Ghazal/Literacy-Rate . [Accessed: 5th February 2024]. Hyzy M, Bond R, Mulvenna M, Bai L, Dix A, Leigh S, et al. System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis. JMIR mHealth uHealth. 2022;10(8):e37290. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53. DeSanto D. February. System Usability Scale: The Gitlab Handbook. updated 2nd 2024. https://handbook.gitlab.com/handbook/product/ux/performance-indicators/system-usability-scale/ . [Accessed: 5th February 2024]. StataCorp. Stata Statistical Software. 14 ed. College Station, Texas: StataCorp LLC; 2015. Likwela JL, Ngwala PL, Ntumba AK, Ntale DC, Sompwe EM, Mpiana GK, et al. Digitalized long-lasting insecticidal nets mass distribution campaign in the context of Covid-19 pandemic in Kongo Central, Democratic Republic of Congo: challenges and lessons learned. Malar J. 2022;21(1):253. Aguma HB, Rukaari M, Nakamatte R, Achii P, Miti JT, Muhumuza S, et al. Mass distribution campaign of long-lasting insecticidal nets (LLINs) during the COVID-19 pandemic in Uganda: lessons learned. Malar J. 2023;22(1):310. Musizvingoza R, Handforth C. The Digital Gender Gap in HealthCare: Progress, Challenges, and Policy Implications: Gender & Health Hub; 2021. https://www.genderhealthhub.org/articles/the-digital-gender-gap-in-healthcare-progress-challenges-and-policy-implications/ . [Accessed: 5th February 2024]. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 21 Oct, 2024 Read the published version in Malaria Journal → Version 1 posted Editorial decision: Revision requested 24 Jun, 2024 Reviews received at journal 13 Jun, 2024 Reviews received at journal 29 May, 2024 Reviews received at journal 28 May, 2024 Reviewers agreed at journal 28 May, 2024 Reviewers agreed at journal 27 May, 2024 Reviewers agreed at journal 14 May, 2024 Reviewers invited by journal 06 May, 2024 Submission checks completed at journal 01 May, 2024 Editor assigned by journal 01 May, 2024 First submitted to journal 29 Apr, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4344384","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":299400432,"identity":"cc859edc-eeaa-4cf0-97db-4fd391a1526d","order_by":0,"name":"Jamshed Khan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABMUlEQVRIie3RMWuDQBTA8ZPAZXmp6x2m9StECqFDIF/ludihOGXJkAYhcE7t7MewCKFDB4ugi9kTCsEuXUrALsUMLb04pCUm6Vqo/+XE83dPOELq6v5gikPpZgXadAjJ5MN2i/5C2iqEhOAPwg4ReVi51eMelmQbOwQaDrWy9T2BzuLVzHG0bKtuFD8Xw+X4hF0heXu43vNjiXGTSvJkBwzjAbDUujyHdMAos33Fe0mqpClYS5TEZ0gRyBy6miKwJA0I432Ef2zIYhYU+ImgS8LXRwmNtc0U7rWmzBQIHUnk3C0ZVciEWtqpYKCCPb0wbxGM1LI0SJELWPmPXhjuEsMVXb4SvT5tzoJ5/o79sySKeTFEVXftuywPxxUyOXoJckS0+04/8O13lSl1dXV1/64vEX1oUc0IVHoAAAAASUVORK5CYII=","orcid":"","institution":"Malaria Consortium","correspondingAuthor":true,"prefix":"","firstName":"Jamshed","middleName":"","lastName":"Khan","suffix":""},{"id":299400436,"identity":"ec137fb0-f2a8-4e7a-bab7-f71f92e77198","order_by":1,"name":"Denis Mubiru","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Denis","middleName":"","lastName":"Mubiru","suffix":""},{"id":299400440,"identity":"8f8a3a1e-fc69-44e2-8d3a-40a05bdf0c25","order_by":2,"name":"Elisabeth G. Chestnutt","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Elisabeth","middleName":"G.","lastName":"Chestnutt","suffix":""},{"id":299400445,"identity":"8ac0bc8b-fd5b-4f0a-920f-91a189e88481","order_by":3,"name":"Louise Cook","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Louise","middleName":"","lastName":"Cook","suffix":""},{"id":299400448,"identity":"e32952aa-6c4e-45c5-8fd1-75f58f3c3360","order_by":4,"name":"Lual Lual Rinny","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Lual","middleName":"Lual","lastName":"Rinny","suffix":""},{"id":299400451,"identity":"ac5a15fd-bb13-4473-8db1-21e61f685d94","order_by":5,"name":"Francis Okot","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Francis","middleName":"","lastName":"Okot","suffix":""},{"id":299400454,"identity":"da28b604-c2f0-4d56-aa6e-9e9a6c98e267","order_by":6,"name":"Kediende Chong","email":"","orcid":"","institution":"Ministry of Health","correspondingAuthor":false,"prefix":"","firstName":"Kediende","middleName":"","lastName":"Chong","suffix":""},{"id":299400456,"identity":"f4d48c4a-c761-444f-8d3d-7057fce39ad3","order_by":7,"name":"Matur T. Tieng","email":"","orcid":"","institution":"Ministry of Health","correspondingAuthor":false,"prefix":"","firstName":"Matur","middleName":"T.","lastName":"Tieng","suffix":""},{"id":299400458,"identity":"6a7751c5-b1ce-4366-95e1-de78d77573b6","order_by":8,"name":"Tombari Prince Zabbeh","email":"","orcid":"","institution":"United Nation Children's Fund","correspondingAuthor":false,"prefix":"","firstName":"Tombari","middleName":"Prince","lastName":"Zabbeh","suffix":""},{"id":299400460,"identity":"00fc1eb3-468d-4483-977a-aee4207745e4","order_by":9,"name":"Joshua Okafor","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Joshua","middleName":"","lastName":"Okafor","suffix":""},{"id":299400461,"identity":"209f6a48-afa6-4a9d-9cf2-28af655e5787","order_by":10,"name":"Chuks A. Nnaji","email":"","orcid":"","institution":"Malaria Consortium","correspondingAuthor":false,"prefix":"","firstName":"Chuks","middleName":"A.","lastName":"Nnaji","suffix":""}],"badges":[],"createdAt":"2024-04-29 17:10:59","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4344384/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4344384/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12936-024-05092-w","type":"published","date":"2024-10-21T15:57:45+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":56123485,"identity":"6c4d34e6-788b-4d1d-becd-4670b9e945c5","added_by":"auto","created_at":"2024-05-08 20:51:47","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":99578,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eMap showing the Northern Bahr el Ghazal region in South Sudan and the study counties.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4344384/v1/167a728e40ff8423ffc63d75.png"},{"id":56123486,"identity":"bb414225-86a8-4e17-a701-554ff2590141","added_by":"auto","created_at":"2024-05-08 20:51:47","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":42121,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eResponses to the five positively framed questions regarding use of the digital tool.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-4344384/v1/0075fa388a2b5911f28f3aab.png"},{"id":56123262,"identity":"39cc057c-2ea9-4f36-b47e-fdb547896ecc","added_by":"auto","created_at":"2024-05-08 20:43:47","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":46485,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eResponses to the negatively framed questions regarding use of the digital tool\u003c/em\u003e.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-4344384/v1/5b473f78e7603621f80af2f1.png"},{"id":56123266,"identity":"cba1f2e0-f95c-47d1-89d2-3df6adb14d93","added_by":"auto","created_at":"2024-05-08 20:43:47","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":33121,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eVolunteers’ comparison of digital tool versus paper-based system.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-4344384/v1/b6e0432650fb513984211ccb.png"},{"id":67681897,"identity":"2c2a05f7-271b-4a3f-971f-83ffbea18a0d","added_by":"auto","created_at":"2024-10-28 16:11:01","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":867263,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4344384/v1/1f8e4d50-b5e5-4803-bc19-341323e5dbf8.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Acceptability and usability of a digital tool to support long-lasting insecticide-treated net distribution in Northern Bahr el Ghazal State, South Sudan","fulltext":[{"header":"Background","content":"\u003cp\u003eMalaria is a serious public health problem, predominantly affecting countries in sub-Saharan Africa. In South Sudan, malaria accounts for 66% of outpatient consultations, 50% of inpatient consultations and approximately 30% of all mortality (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). The Ministry of Health and National Malaria Control Programme in South Sudan have developed a National Malaria Strategy which includes activities to prevent, diagnose and treat malaria, including the distribution of long-lasting insecticide-treated nets (LLINs) (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eLLINs are a proven method for preventing malaria, and have been the backbone of malaria control for over two decades (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). LLINs are typically distributed through mass campaigns and continuous distribution channels at antenatal care and EPI clinics. In South Sudan, LLIN distribution campaigns are conducted every three years, as recommended by the World Health Organization (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). During the campaigns, LLINs are typically distributed by volunteers, and LLIN campaign distribution data are collected using a paper-based system.\u003c/p\u003e \u003cp\u003eAlthough paper-based methods are simple to use, handwritten data entry can result in inaccuracies at collection and collation stages and data reporting is slower, which hinders timely decision making (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). Findings from the introduction of an electronic data system in seven African counties showed electronic data systems can improve data completeness, facilitate data submission and analysis, and reduce the time between data entry and data reporting, when compared to paper-based systems (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eDespite the benefits of electronic data systems, these digital tools have not previously been introduced in South Sudan due to a perception that volunteers with low literacy levels and limited experience with technology would be unable to use them. Internet connectivity is also a challenge only 10.9% of the population in South Sudan have access to the internet, which limits the use of digital tools in remote areas (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eTo improve data accuracy and timely decisions, a digital application was developed to collect data during the LLIN campaign. To demonstrate the usability and feasibility in South Sudan, the application was designed with two key requirements: offline functionality, so data can be collected without the internet, and interoperability with the Digital Health Information System 2 (DHIS2), to improve real-time decision making.\u003c/p\u003e \u003cp\u003eIn 2022, a small-scale pilot was planned, and 106 volunteers were trained to use the application for collecting data during the LLIN campaign in Central Equatoria. After completion of the campaign, interviews were conducted with 13 volunteers to gather feedback. The study showed volunteers were able to use the digital tool to correctly upload data directly to DHIS2. In 2023, a LLIN campaign was carried out in Northern Bahr el Ghazal and 965 volunteers were trained to use the digital tool. Following the campaign, a survey was conducted to assess the feasibility and usability of the tool, and to identify challenges and lessons from the implementation. This paper outlines the findings from this work.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eDescription of the digital tool\u003c/h2\u003e \u003cp\u003eThe LLIN distribution system was developed using the DHIS2 tracker. The tool was created in DHIS2 version 2.38, using capture application version 2.8 to train end-users. Key variables were identified and agreed upon between the Ministry of Health and The Global Fund to fight AIDS, Tuberculosis, and malaria to enable both parties to view the locations of the LLIN distribution using the Global Positioning System (GPS) and household information as an accountability mechanism. By selecting all geographical information from a dropdown list, the digital tool generates unique household numbers and by entering total number of household members, the tool generates the number of LLINs to be distributed to that household. The tool also provides geo-mapping of LLINs that have been distributed in each location and summary reports are automatically generated, collated and uploaded to the DHIS2 dashboard which is visible to decision makers, such as the Ministry of Health. Real-time reporting allows decision-makers to identify issues and mobilise a response during the campaign. The digital tool consists of a database of state-, county-, payam- (the administrative level below county containing a minimum 25,000 people) and boma- (the lowest administrative level consisting of a collection of 4\u0026ndash;8 villages) level information for the whole South Sudan that can be easily selected from dropdown list.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eStudy design\u003c/h2\u003e \u003cp\u003eThis study was designed to assess the usability and feasibility of deploying a digital tool in Northern Bahr el Ghazal. A questionnaire was designed to gather responses from users on the usability and acceptability of the tool. The questionnaire consistent of two components; a modified system usability scale (SUS) approach to assess usability, and three open questions, to gather feedback on the acceptability of the tool (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eStudy setting\u003c/h2\u003e \u003cp\u003eNorthern Bahr el Ghazal state is part of the Greater Bahr el Ghazal region. The state has a total area of 30,543 km\u003csup\u003e2\u003c/sup\u003e and is made up of five counties: Aweil South, Aweil East, Aweil West, Aweil North and Aweil Centre (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Ethnically, most of the state\u0026rsquo;s population is composed of Dinka and Jurchol tribe members, with a minority of Luo tribe members. The main means of livelihood are agriculture and livestock farming for the two tribes. Floods occur annually from June to November and hinder routine life, causing internal displacements. The malaria prevalence in Northern Bahr el Ghazal in 53%, the highest of any state in South Sudan, and around 66% of the population have access to LLINs (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e). Northern Bahr el Ghazal also has the lowest literacy rate in the country, with estimates suggesting only 21% of the population aged fifteen years and above are literate (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eLLIN distribution implementation\u003c/h2\u003e \u003cp\u003eThe digitalised campaign was carried out in all five counties of Northern Bahr el Ghazal state from March to June 2023. A total of 965 volunteers (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) including Registrars, Site Managers, Payam Supervisors, and County Health Department and State Ministry of Health staff were trained to use the digital tool. The LLIN campaign was conducted using a house-to-house distribution method and COVID-19 protocols were observed.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eParticipants\u0026rsquo; distribution across survey in all counties.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCounty\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRegistrars\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSite Manager\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePayam Supervisor\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSMOH\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eCHD\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eSample size\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAweil Centre\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAweil East\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e330\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e66\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e406\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e41\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAweil West\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e178\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e36\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e224\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e19\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAweil South\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e77\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e102\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAweil North\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e135\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e169\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eTotal\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e765\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e155\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e35\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e5\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e5\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e965\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e93\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003eSMOH: State Ministry of Health staff, CHD: County Health Department staff\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eSampling and data collection\u003c/h2\u003e \u003cp\u003eThe study questionnaire (Additional file 1) was designed and deployed using Malaria Consortium\u0026rsquo;s Projects Results System mobile application and administered to participants in the field by Malaria Consortium and County Health Department staff. A minimum sample size of 80 participants was determined to be sufficient for the study, assuming expected mean usability score (standard deviation) of 68 (12.5) based on a previous study (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). The sample size was calculated to determine the mean SUS score with a 95% level of confidence, precision of \u0026plusmn;\u0026thinsp;3 and non-response rate of 10%. The questionnaire containing the SUS rating scale was administered to a total 93 respondents (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Respondents were randomly selected using random sample selection from a pool of 965 potential participants who were trained to use the digital tool.\u003c/p\u003e \u003cp\u003eAdaptations were made to the original 10-item SUS Likert-type rating scale, which combines an equal number of negatively and positively framed Likert items for assessing system usability. Additionally, the modified version of the scale incorporated three open questions to probe and contextualise responses. The adapted SUS rating scale was piloted and validated with 20 respondents who did not participate in the main study. A principal component analysis was used to assess the reliability and interval validity of the adapted SUS tool. The SUS tool was deemed to have good reliability and internal validity based on satisfactory component loadings of each of the 10 items in the usability correlation matrix, Cronbach\u0026rsquo;s alpha values (ranging from 0.63 to 0.72) and Eigenvalues (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe SUS rating scale covered a variety of aspects of system usability, such as ease of use, need for support, complexity and perceived usefulness of the tool. The questionnaire also enabled the collection of data on participants\u0026rsquo; characteristics, including location, gender and role. The survey was carried out by seven enumerators from Malaria Consortium\u0026rsquo;s field team based in Northern Bahr el Ghazal state through visits to data collection sites. Accounts were created for each of the survey participants, and enumerators provided participants with a tablet to complete the survey and then log out of their account.\u003c/p\u003e \u003cp\u003eIn addition to the SUS rating, open questions were used to collect information on user perceptions and acceptability of the tool. The open questions covered three areas: i) challenges experienced using the digital tool, ii) suggestions for improvement, and iii) overall perception and experience with the digital tool compared with paper-based systems.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eData analysis\u003c/h2\u003e \u003cp\u003eDescriptive statistics were used to summarise participant characteristics, expressed as frequencies and percentages for categorical variables, and means and standard deviations for continuous variables. Individual usability scores were calculated for each item and participant, which were pooled to generate mean usability scores and standard deviations across the entire study sample in accordance with the SUS scoring framework (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). According to this framework, SUS scores have a range of 0 to 100, computed based on the 10 items in the rating scale covering a variety of aspects of system usability, with scores of \u0026gt;\u0026thinsp;72.5, 62.7\u0026ndash;72.5, 51.7\u0026ndash;62.6 and \u0026lt;\u0026thinsp;51.7 considered as excellent, good, ok and poor, respectively (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). Differences in SUS scores across participant characteristics were assessed using ANOVA tests of comparison of means. Statistical significance was determined at p value\u0026thinsp;\u0026lt;\u0026thinsp;0.05. Statistical analyses were conducted using Stata (version 16)(\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e). Responses from the open questions covering acceptability were grouped by theme.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eDuring the LLIN campaign a total of 773,387 LLINs were distributed to households across the five counties (Additional file 2).\u003c/p\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eParticipants\u0026rsquo; characteristics\u003c/h2\u003e \u003cp\u003eA total of 93 individuals participated in the usability and acceptability assessments. Respondent\u0026rsquo;s characteristics are summarised in Table \u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e and further information provided in Additional file 3.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eSUS score distributions across respondents\u0026rsquo; characteristics\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003eVariable\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo. of respondents\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eMean (\u0026plusmn;\u0026thinsp;SD) SUS scores\u003csup\u003e#\u003c/sup\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eP-value*\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003e\u003cb\u003eCounty\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAweil Central\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e73.00 (4.11)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003e0.542\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAweil East\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e57.32 (14.98)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAweil North\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e64.31 (5.61)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAweil South\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e72.25 (3.99)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAweil West\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e56.32 (11.44)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eGender\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFemale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e50.83 (17.74)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e0.204\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e90\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e61.25 (12.67)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003e\u003cb\u003eRole\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCHD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e73.00 (4.11)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003e0.062\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eManager\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e60.71 (9.91)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePayam supervisor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e68.25 (8.90)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRegistrar\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e58.66 (14.09)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSMOH\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e57.50\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eTotal\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e93\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e60.91 (12.87)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"5\"\u003eCHD: County health department, SMOH: State ministry of health, SD: standard deviation\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e#mean SUS scores have a range of 0 to 100 computed based on the 10 items in the rating scale covering a variety of aspects of system usability\u003c/p\u003e \u003cp\u003e*ANOVA test of comparison of means\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e presents the composite SUS score for the study sample and distribution of scores by participant characteristics. The mean (\u0026plusmn;\u0026thinsp;SD) usability score across the 10 SUS-scoring items was 60.91 (12.87) for the total study sample. Based on the framework (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) for interpreting SUS scores, this represents an \u0026lsquo;ok\u0026rsquo; level of usability. The majority of users found the digital tool was easy to use, reduced workload, helped in stock management, and facilitated real time campaign monitoring. Usability did not vary significantly by county (p\u0026thinsp;=\u0026thinsp;0.542), gender (p\u0026thinsp;=\u0026thinsp;0.204) \u003cb\u003eor\u003c/b\u003e role (p\u0026thinsp;=\u0026thinsp;0.062).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eSummary of item-level participants\u0026rsquo; usability responses\u003c/h2\u003e \u003cp\u003eOver 85% of participants responded either \u0026lsquo;agree\u0026rsquo; or \u0026lsquo;strongly agree\u0026rsquo; to the positively framed questions (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). The highest scoring statement was the tool improved the quality of reporting, followed closely by respondents reporting the tool was easy to use. In addition, respondents disagreed with the majority of the negatively framed questions (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). The only negatively framed question where the majority of participants answered \u0026lsquo;agree\u0026rsquo; or \u0026lsquo;strongly agree\u0026rsquo; was regarding improvements to the tool. When asked to compare the digital tool with the former paper-based system the majority of participants (65%) did not respond to the question (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). However, of the 32 responses, 94% reported the digital tool was an improvement on the paper-based system with 69% reporting it was very good and much improved.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eAcceptability of the tool\u003c/h2\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e summarises participants\u0026rsquo; responses on the acceptability of the digital tool. All participants responded to at least one of the open questions, however, all participants did not respond to all the questions, and many responses were not detailed. Overall, respondents with experience of both paper-based and the digital tool reported preferring the digital tool for use during LLIN distribution campaigns. In addition, users reported the tool was user-friendly, made their work easier and reduced their workload. The most frequently mentioned challenges were the GPS functionality, which respondents reported as having some issues loading, charging the device and the digital form, which respondents suggested could be shorter. Overall, the majority of respondents considered the tool an improvement and said they would recommend it for future use.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eThematic summary of participants\u0026rsquo; perceptions and acceptability of the digital tool\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eQuestion\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePercentage analysis\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1. \u003cb\u003ePerceived challenges that can affect functionality of the digital tool\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOut of 93 participants:\u003c/p\u003e \u003cp\u003e\u0026bull; 50% either did not respond or did not report any challenges while using this tool.\u003c/p\u003e \u003cp\u003e\u0026bull; 27% mentioned limited access to power source for charging tablet batteries.\u003c/p\u003e \u003cp\u003e\u0026bull; 13% mentioned GPS reading caused delays.\u003c/p\u003e \u003cp\u003e\u0026bull; 9% responded poor or no network as a challenge.\u003c/p\u003e \u003cp\u003e\u0026bull; Only 1% mentioned short training session as challenge.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2. \u003cb\u003eSuggested ways to improve the tool\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOut of 93 participants:\u003c/p\u003e \u003cp\u003e\u0026bull; 41% did not respond or did not mention any suggestion while using this tool.\u003c/p\u003e \u003cp\u003e\u0026bull; 17% requested stronger battery charging options.\u003c/p\u003e \u003cp\u003e\u0026bull; 12% praised the tool as good to use and suggest for scale up.\u003c/p\u003e \u003cp\u003e\u0026bull; 12% suggested designing the tool to entirely work offline.\u003c/p\u003e \u003cp\u003e\u0026bull; 9% suggested to reduce or adjust GPS reading time.\u003c/p\u003e \u003cp\u003e\u0026bull; 8% suggested to shorten form in the tool to avoid delays.\u003c/p\u003e \u003cp\u003e\u0026bull; 2% suggested the need for more internet bundles.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3. \u003cb\u003ePerception on the use of the tool compared with paper-based method\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOut of 93 participants\u003c/p\u003e \u003cp\u003e\u0026bull; 37% did not respond or chose to remain neutral while using this tool.\u003c/p\u003e \u003cp\u003e\u0026bull; 57% appreciated the tool as simple, easy to use and reduced workload.\u003c/p\u003e \u003cp\u003e\u0026bull; 2% were engaged in paper-based campaign and 100% of them preferred digital tool over paper-based system.\u003c/p\u003e \u003cp\u003e\u0026bull; 4% reported other issues with GPS and internet bundles.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe aim of this study was to investigate the feasibility of introducing a digital data collection system for LLIN campaigns in South Sudan. The study demonstrated the tool has a moderate level of usability among respondents. Although the score is below the globally accepted threshold for high usability (68.5), achieving even a moderate usability score contradicts the assumption that volunteers with low literacy levels and limited experience with technology would be unable to use digital tools (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). This usability score is also supported by responses to the open questions, which reported the majority of participants found the tool simple and easy to use. In addition, the majority of respondents who had experience with both paper-based and digital data collection systems reported a preference for the digital tool. These findings are consistent with similar studies in Benin and the Democratic Republic of the Congo where the introduction of digital data collection tools during LLIN campaigns were also found to be valuable (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe study also identified several challenges and contextual factors influencing the tool\u0026rsquo;s usability, acceptability and overall user experience. The most commonly reported challenges were internet connectivity and issues charging the devices as the capacity of the power banks supplied was not high enough to charge the devices. Despite the responses regarding poor connectivity, the tool did not require internet connectivity to function, when probing further respondents mentioned they perceived the slow device loading was due to connectivity issues. Volunteers incorrectly assumed this is problem with internet connectivity rather than the device itself. Additional training may be useful to improve volunteer\u0026rsquo;s understanding of how the tool works and to enable them to troubleshoot any issues. Furthermore, power supply limitations can be remedied by providing higher-capacity power banks and mobile charging systems, which could be solar powered. Another area suggested for improvement was to shorten the digital form, however this form is based on standard reporting templates for LLIN campaigns and therefore is unlikely to change. To enhance the tool\u0026rsquo;s usability, acceptability and overall benefits, it is imperative to tailor its functionality and future deployment strategies to overcome the challenges and contextual constraints identified.\u003c/p\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003eStudy strengths, limitations, and implications for future research\u003c/h2\u003e \u003cp\u003eThe SUS scoring method provides a quick and highly efficient way of assessing usability. It is easy to apply and is proven to quickly measure user perception with considerable precision. Notwithstanding the strength of the methods used, the study has some limitations worth acknowledging. While the data collection tool enabled the collection of open and closed responses on usability and user perceptions, the open responses were brief and not detailed enough to provide deeper insights for contextualising the quantitative SUS scores. Secondly, SUS scores rely on respondent's perceptions and judgements which may not always be objective. Consequently, responses are prone to social desirability bias. The study\u0026rsquo;s lack of a control group poses another limitation. A controlled study design would have enabled the comparison of usability between digital and paper based LLIN campaigns and would have provided more informative insight on the comparative advantage of digitalising public health interventions like LLIN distribution. The study\u0026rsquo;s ability to determine significant differences in usability was constrained by sample size limitations. Further studies should consider using a larger sample to explore variations in SUS scores across user-level and contextual characteristics.\u003c/p\u003e \u003cp\u003eThe disparity in the distribution of study participants\u0026rsquo; gender, with an underrepresentation of women is perhaps a reflection of the gender imbalance in LLIN digitalisation and distribution campaigns in the study locations. Gender gaps in digital health interventions, if left unaddressed, have the potential to undermine the impact of digital tools and exacerbate existing inequities (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). The gender imbalance seen in this study therefore warrants efforts to involve female stakeholders across the entire value-chain of public health interventions.\u003c/p\u003e \u003cp\u003eIn addition, the usability of digital tools is influenced by a complex interplay of cultural, technological, economic, and broader contextual factors which can affect SUS scores. Data on other factors that could potentially influence SUS scores, such as those relating to users\u0026rsquo; knowledge and education levels, were not captured and considered as covariates in the analysis. This makes it difficult to compare results with globally defined thresholds and usability scores in other study settings. Furthermore, the findings from this study may not be generalisable to the whole of South Sudan due to the unique context of Northern Bahr el Ghazal state.\u003c/p\u003e \u003cp\u003eFinally, this study investigated the usability of the digital tool from users' perspectives and was not designed to assess of the effectiveness of the tool and its ability to improve data quality or optimise decision-making, which requires further research.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study demonstrates that digitalising LLIN distribution campaigns across South Sudan is feasible, including in remote rural areas like Northern Bahr el Ghazal. Digitising these campaigns has the potential to improve accurate data entry, timely data reporting and real-time decision making which in turn could help to optimise the delivery of LLINs. Further work is needed to address the challenges identified during this study including changes to the tool that could improve user experience. In addition, further research assessing the degree to which digitisation improves data quality and decision making during campaigns would be valuable.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eCHD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eCounty health department\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eDHIS 2\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eDigital Health Information System 2\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eGPS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eglobal positioning system\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eLLIN\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eLong-lasting insecticide-treated net\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eNBeG\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eNorthern Bahr el Ghazal\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePReS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eProject results system\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003estandard deviation\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSMOH\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eState ministry of health\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSUS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003esystem usability scale\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAvailability of data and materials\u003c/h2\u003e\n\u003cp\u003eProcessed data supporting the findings of this study are included in this published article and its supplementary information files. Original datasets generated and analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eThis study was funded through Global Fund funding and UNICEF as fund manager, for LLIN received by Malaria Consortium, primarily for the distribution of digitalised LLIN campaign.\u003c/p\u003e\n\u003ch2\u003eAcknowledgements\u003c/h2\u003e\n\u003cp\u003eWe are very grateful to all of the study participants, UNICEF, volunteers, Boma Health Workers, National Malaria Control Programme, National Ministry of Health, State Ministry of Health, county health departments, \u0026nbsp;political and technical leaders of Aweil Centre, Aweil West, Aweil East, Aweil North and Aweil South counties, and to the Boma health teams involved in the distribution of LLIN.\u003c/p\u003e\n\u003ch2\u003eAuthors\u0026rsquo; contributions\u003c/h2\u003e\n\u003cp\u003eJK: study conceptualisation, planning, supervision, developed study protocol, data analysis, visualisation, writing \u0026ndash; original manuscript and editing, data LC: writing \u0026ndash; review, formatting, coordinating writing, JO: supervision of data collection, data analysis, visualisation and interpretation, and editing. CN: study design, data analysis, visualisation, and interpretation, writing \u0026ndash; results of manuscript and review of manuscript. DM: study conceptualisation and supervision, writing \u0026ndash; review. LLR: training coordination, data collection, fieldwork coordinator and supervisor, writing \u0026ndash; review. FO: training coordination, data collection, fieldwork coordinator and supervisor, review. LC: study conceptualisation, writing \u0026ndash; review MA: field work coordinator and supervisor, writing \u0026ndash; review\u003cstrong\u003e.\u003c/strong\u003e KC: principal investigator, study design, writing \u0026ndash; review.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eEthics approval and consent to participate\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study received ethical approval from ministry of health. Data were used in accordance with the MOH\u0026rsquo;s ethics standards. Informed consent was obtained from all study participants before data collection.\u003c/p\u003e\n\u003ch2\u003eConsent for publication\u003c/h2\u003e\n\u003cp\u003eAll authors read and approved the final manuscript.\u003c/p\u003e\n\u003ch2\u003eCompeting interests\u003c/h2\u003e\n\u003cp\u003eKC is the director general for policy, planning, budget, research, monitoring and evaluation of South Sudan Ministry of Health. All other authors declare no competing interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003ePasquale HA. Malaria prevention strategies in South Sudan. South Sudan Med J. 2020;13(5):187\u0026ndash;90.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003e[South Sudan] National Malaria Control Programme. National Malaria Strategic Plan 2021\u0026ndash;2025. [South Sudan] National Malaria Control Programme; 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWorld Health Organization. World Malaria Report 2022. Geneva: World Health Organization; 2022.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWorld Health Organization. Achieving and maintaining universal coverage with long-lasting insecticidal nets for malaria control. Geneva: World Health Organization; 2017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLey B, Rijal KR, Marfurt J, Adhikari NR, Banjara MR, Shrestha UT, et al. Analysis of erroneous data entries in paper based and electronic data collection. BMC Res Notes. 2019;12(1):537.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBurnett SM, Wun J, Evance I, Davis KM, Smith G, Lussiana C, et al. Am J Trop Med Hyg. 2019;100(4):889\u0026ndash;98. Introduction and Evaluation of an Electronic Tool for Improved Data Quality and Data Use during Malaria Case Management Supportive Supervision.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKemp SD. 2022: South Sudan: Data Reportal; 2022. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://datareportal.com/reports/digital-2022-south-sudan#:~:text=There%20were%201.25%20million%20internet,percent)%20between%202021%20and%202022.%202022\u003c/span\u003e\u003cspan address=\"https://datareportal.com/reports/digital-2022-south-sudan#:~:text=There%20were%201.25%20million%20internet,percent)%20between%202021%20and%202022.%202022\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. [Accessed: 5th February 2024].\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrooke J. SUS: A quick and dirty usability scale. Usability Eval Ind. 1995;189.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSouth Sudan Ministry of Health. Republic of South Sudan Malaria Indicator Survey 2017. Juba: South Sudan Ministry of Health,; 2017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKnoema. Northern Bahr el Ghazal - Literacy, Can read and write (15+): Knoema; 2009. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://knoema.com/atlas/South-Sudan/Northern-Bahr-el-Ghazal/Literacy-Rate\u003c/span\u003e\u003cspan address=\"https://knoema.com/atlas/South-Sudan/Northern-Bahr-el-Ghazal/Literacy-Rate\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. [Accessed: 5th February 2024].\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHyzy M, Bond R, Mulvenna M, Bai L, Dix A, Leigh S, et al. System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis. JMIR mHealth uHealth. 2022;10(8):e37290.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeSanto D. February. System Usability Scale: The Gitlab Handbook. updated 2nd 2024. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://handbook.gitlab.com/handbook/product/ux/performance-indicators/system-usability-scale/\u003c/span\u003e\u003cspan address=\"https://handbook.gitlab.com/handbook/product/ux/performance-indicators/system-usability-scale/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. [Accessed: 5th February 2024].\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStataCorp. Stata Statistical Software. 14 ed. College Station, Texas: StataCorp LLC; 2015.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLikwela JL, Ngwala PL, Ntumba AK, Ntale DC, Sompwe EM, Mpiana GK, et al. Digitalized long-lasting insecticidal nets mass distribution campaign in the context of Covid-19 pandemic in Kongo Central, Democratic Republic of Congo: challenges and lessons learned. Malar J. 2022;21(1):253.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAguma HB, Rukaari M, Nakamatte R, Achii P, Miti JT, Muhumuza S, et al. Mass distribution campaign of long-lasting insecticidal nets (LLINs) during the COVID-19 pandemic in Uganda: lessons learned. Malar J. 2023;22(1):310.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMusizvingoza R, Handforth C. The Digital Gender Gap in HealthCare: Progress, Challenges, and Policy Implications: Gender \u0026amp; Health Hub; 2021. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.genderhealthhub.org/articles/the-digital-gender-gap-in-healthcare-progress-challenges-and-policy-implications/\u003c/span\u003e\u003cspan address=\"https://www.genderhealthhub.org/articles/the-digital-gender-gap-in-healthcare-progress-challenges-and-policy-implications/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. [Accessed: 5th February 2024].\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"malaria-journal","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"malj","sideBox":"Learn more about [Malaria Journal](http://malariajournal.biomedcentral.com/)","snPcode":"12936","submissionUrl":"https://submission.nature.com/new-submission/12936/3","title":"Malaria Journal","twitterHandle":"@malariajournal","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Long-lasting insecticide-treated nets, Digitalisation, South Sudan, System Usability Scale","lastPublishedDoi":"10.21203/rs.3.rs-4344384/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4344384/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e: Long-lasting insecticide-treated nets (LLINs) have been the backbone of malaria prevention for decades. In South Sudan, LLINs are typically distributed by volunteers who use paper-based systems to collect distribution data. Paper-based systems are simple to use but have a higher occurrence of \u0026nbsp;data inaccuracies and can hinder the timely use of data for decision making. In 2022, a digital tool was introduced to collect data during the LLIN campaign in Northern Bahr el Ghazal (NBeG). The tool aimed to improve the accuracy of data entry and enable data to be use in real-time for decision making during the campaign. This study assessed the acceptability and usability of the digital tool.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e: A questionnaire containing open and closed questions was conducted with users of the digital tool, supervisors and other key stakeholders in five counties of NBeG. The questionnaire was administered using Malaria Consortium’s Projects Results System Android mobile application. Usability was determined through a modified and validated System Usability Scale (SUS) approach and acceptability was assessed by responses to open questions.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e A total of 93 participants responded to the usability and acceptability questionnaire. The mean (±standard deviation) usability score across 10 SUS-scoring items was 60.91 (12.87), indicating a moderate level of usability. The majority of users reported the tool was useful for managing the LLIN distribution workflow, was easy to use, reduced workload, and supported stock management and real-time campaign monitoring\u003cstrong\u003e. \u003c/strong\u003eThere was no significant difference in the usability scores across genders, roles, and counties.\u003cstrong\u003e \u003c/strong\u003eThe digital tool was perceived to be acceptable, and the majority of respondents with experience of both paper-based and the digital tool reported a preference for the digital tool over paper-based systems. The majority of respondents also said they would recommend the digital tool to colleagues.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e: Digital tools are perceived to be beneficial for collecting data during LLIN campaigns, even in remote areas where network coverage is challenging. Additional improvements can be implemented to overcome operational challenges and improve usability of the tool. Further study is needed to determine the impact of the digital tool on data quality and real-time data use.\u003c/p\u003e","manuscriptTitle":"Acceptability and usability of a digital tool to support long-lasting insecticide-treated net distribution in Northern Bahr el Ghazal State, South Sudan","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-08 20:43:42","doi":"10.21203/rs.3.rs-4344384/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-06-24T22:00:35+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-13T13:56:14+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-29T17:32:13+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-28T20:02:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"91378181582241379311157025489154061059","date":"2024-05-28T05:24:42+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"283148836221220561653999596414262295632","date":"2024-05-28T00:37:29+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"286482370065636237192520650624668688379","date":"2024-05-14T23:26:14+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-05-06T18:45:52+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-01T05:28:28+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-01T05:28:28+00:00","index":"","fulltext":""},{"type":"submitted","content":"Malaria Journal","date":"2024-04-29T16:47:08+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"malaria-journal","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"malj","sideBox":"Learn more about [Malaria Journal](http://malariajournal.biomedcentral.com/)","snPcode":"12936","submissionUrl":"https://submission.nature.com/new-submission/12936/3","title":"Malaria Journal","twitterHandle":"@malariajournal","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"23d29095-2ecb-4916-b683-fbda7c21b58e","owner":[],"postedDate":"May 8th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-10-28T16:02:50+00:00","versionOfRecord":{"articleIdentity":"rs-4344384","link":"https://doi.org/10.1186/s12936-024-05092-w","journal":{"identity":"malaria-journal","isVorOnly":false,"title":"Malaria Journal"},"publishedOn":"2024-10-21 15:57:45","publishedOnDateReadable":"October 21st, 2024"},"versionCreatedAt":"2024-05-08 20:43:42","video":"","vorDoi":"10.1186/s12936-024-05092-w","vorDoiUrl":"https://doi.org/10.1186/s12936-024-05092-w","workflowStages":[]},"version":"v1","identity":"rs-4344384","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4344384","identity":"rs-4344384","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}