Strengthening Expanded Program for Immunization Service Delivery through the Hayat mHealth Application: A Cross-Sectional Study in Upper and Lower Chitral, Pakistan

Strengthening Expanded Program for Immunization Service Delivery through the Hayat mHealth Application: A Cross-Sectional Study in Upper and Lower Chitral, Pakistan | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Strengthening Expanded Program for Immunization Service Delivery through the Hayat mHealth Application: A Cross-Sectional Study in Upper and Lower Chitral, Pakistan Saira Samnani, Nisha Asif, Abdul Muqeet, Ahsan Nawaz, Saleem Sayani This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6828546/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 22 You are reading this latest preprint version Abstract Background Mobile health (mhealth) technologies are revolutionizing and transforming healthcare delivery, particularly in low resource settings, by improving data accuracy, accessibility and decision making. This study aimed to evaluate the effectiveness and usability of the Hayat m-health application in enhancing immunization service delivery ae well as data accuracy, compared to traditional manual EPI registers in Upper and Lower Chitral, Khyber Pakhtunkhwa KPK, Pakistan. Methodology A cross-sectional study conducted across 63 healthcare facilities using structured tools and data comparisons from the month of January to March 2024. Results The results showed that cumulative percentage difference (delta) between Hayat and manual records remained within acceptable 10% threshold for most vaccine antigens, indicating strong data accuracy. Whereas 100% of the 19 participating vaccinators demonstrated proficiency in features including registration and child search, although gaps were identified in advanced functions, like data integration and offline sharing. Vaccinator satisfaction with the application was undisputed. The matching of record between Hayat Application and vaccination cards exceeded 95% for all the essential antigens (BCG, Penta 1, Penta 2, Penta 3, MR 1, and MR 2). Therefore, these results support Hayat Application potential for improving immunization data accuracy and service delivery in the remote areas. Conclusion The Hayat mhealth application demonstrated strong potential for improving immunization of data accuracy and service delivery in remote areas and resource limited settings. mHealth application BCG Penta MR LMIC (low-middle income countries) Figures Figure 1 Introduction In recent years, mobile health (mHealth) technologies have emerged as transformative tools for enhancing healthcare delivery, particularly in low-resource settings ( 1 ), where access to quality care is often constrained by various barriers, including infrastructure limitations and workforce shortages. One of the most significant advancements facilitated by mHealth is the efficiency and reliability of data collection using mobile devices. Unlike traditional pen-and-paper methods, mobile data collection offers faster processing and reduces the risk of errors, ensuring higher accuracy in capturing and managing health information ( 1 ). By leveraging the accessibility and convenience of mobile platforms, mHealth applications empower healthcare workers to extend critical services to remote and underserved communities. These applications address logistical challenges, such as difficult terrain and inadequate transportation networks, while also mitigating the impact of limited healthcare infrastructure ( 2 ). For example, immunization tracking, remote patient monitoring, and disease surveillance are increasingly being facilitated through mobile solutions, ensuring that healthcare reaches even the most isolated populations ( 1 ). The significance of mHealth is particularly pronounced in low-and middle-income countries (LMICs), where it serves as an important bridge to overcome systemic healthcare challenges. By leveraging mHealth, LMICs addressed critical gaps in healthcare accessibility, affordability and quality. mHealth provides scalable solutions for the management of disease, emergency response and patient education which are hindered often by financial and logistical barriers( 3 ). mHealth platforms significantly enable real-time data sharing among professionals and communication accessibility between healthcare workers and central facilities that fostered more coordinated and responsive quality care delivery to patients, however significantly in resource constrained environments where timely interventions have lifesaving implications. For maternal and child health, mHealth has proven to be instrumental in improving postnatal, antenatal care through education, communication and promoting care-seeking behaviors. Through mHealth technologies in LMICs, governments and organizations to effectively reach vulnerable populations vulnerable populations effectively, ensuring better maternal health outcomes ( 4 ). Further studies illustrated potential of mhealth, example research conducted in Georgia, highlighted potential of mHealth in addressing maternal health challenges among mothers in rural areas. Through Interviews with 14 participants the study identified barriers such as limited healthcare facilities, racial bias in providers interactions, the results showed that participants emphasized the potential of mhealth applications in offering virtual support and resources. However, this study highlighted the need for patient-centered practices through mhealth applications( 5 ). Another large-scale study was conducted in LMIC over 100 days on Stepthlon mHealth programs that included 69,000 participants, 92% were from developing countries. This program focused on an accessible platform for tracking activity and encouraging healthy behaviors in resource limited setting. These findings highlighted the potential of mHealth, that overcome barriers to reach underserved communities and support sustainable interventions and bridge gaps in healthcare accessibility( 6 ). One more study conducted that review and categories mHealth initiatives in Kenya, identifying different active projects focused on improving healthcare access and outcomes in underserved regions. The study highlighted the importance of health in addressing healthcare disparities, particularly in rural and marginalized areas. They targeted vulnerable populations, including those also living in remote areas, with a particular focus on maternal and child health, disease prevention and any emergency responses. These findings explored the potential of mhealth to increased healthcare delivery service where traditional services are limited ( 7 ). Globally, countries are prioritizing investments in mobile networks over traditional infrastructure such as road transport and power generation, reflecting the critical role that mobile connectivity plays in driving innovation and expanding access to essential services ( 7 ). One such innovative mHealth application is Hayat, designed to empower health workers by enhancing the accessibility and effectiveness of Reproductive, Maternal, Newborn, and Child Health (RMNCH), immunizations, and child growth services, particularly in remote or underserved areas of Pakistan. It streamlines the operations of various health worker roles, including Lady Health Workers (LHWs), vaccinators, and their supervisors, by digitizing paper-based forms. It enables health workers to efficiently record service details, track patients, and raise awareness within communities through educational videos. Till present date, the Hayat application has been able to register 482,835 beneficiaries with over 192,000 educational awareness sessions conducted, with an average number of 10 participants per session. Additionally, it features a supervisory application for remote monitoring, conducting audits, and verifying monthly reports. Hayat was recognized internationally for its novelty and innovativeness at the International Digital Health Awards 2023. Since its launch, the Hayat has demonstrated significant potential in enhancing service utilization among the population. It currently operates in two districts of Khyber Pakhtunkhwa (KPK) that is Upper and Lower Chitral and five districts of Gilgit Baltistan (GB) including Hunza, Nagar, Astore, Ghizer, and Skardu. However, governments in LMIC often face a range of challenges and competing priorities, which limit their capacity to adopt new innovations. As a result, they require strong, credible evidence on mHealth projects to evaluate their potential alongside other essential health interventions, as well as guidance on which mHealth solutions are most likely to contribute to achieving broader health system goals( 2 ). Therefore, KPK provincial EPI team with public private partnership aims to evaluate the usability and effectiveness of the Hayat mHealth application in complementing traditional manual daily registers for accurate data recording by health workers. The objective of an evaluation was to determine the accuracy of data, ensuring that the data from the manual daily register(s) is accurately reflected in the Hayat application. Methodology Study design and study sites A cross-sectional study was conducted to assess the usability of the Hayat mHealth application among vaccinators in upper and lower Chitral districts of KPK. The study sites included all five union councils (UCs) across upper and lower Chitral, encompassing 63 healthcare facilities where the Hayat mHealth application was being used alongside manual EPI registers. These registers include the daily register(s), maintained separately by each vaccinator, which record all vaccinations administered by that vaccinator, regardless of whether the children reside inside or outside the respective UCs. Additionally, the permanent register exclusively documents vaccinations of all children residing within the respective UCs. Study population The study population includes all the healthcare facilities in upper and lower Chitral using the Hayat mHealth application alongside manual EPI registers for recording children's vaccination data. Eligibility criteria Inclusion All the healthcare facilities in upper and lower Chitral using the Hayat mHealth application alongside manual EPI registers for recording children's vaccination data were eligible for inclusion in the study. Exclusion All the healthcare facilities that had not yet implemented or adopted the Hayat m Health application during the study period. Facilities with missing, incomplete, or inconsistent data in the manual EPI registers. Sampling technique Simple random probability sampling was employed to ensure the unbiased selection of healthcare facilities, wherein healthcare facilities from all five UCs in both upper and lower Chitral districts were randomly selected using computer-generated random numbers. Subsequently, 63 healthcare facilities within these UCs were visited by the KPK provincial EPI team for evaluation. Data collection tools and procedures Three evaluation teams were established, each supported by a member of the Aga Khan University (AKU) team to document the evaluation process. A structured questionnaire was developed by a team of experts to evaluate the efficiency of vaccinators in using the Hayat mHealth application (refer to the supplementary file). Given the purpose of the evaluation, it was essential to assess the accuracy of the data recorded by vaccinators in both the Hayat application and their daily registers to identify any discrepancies. To achieve this, the vaccinations recorded in the vaccinators' daily register(s) were compared with those in their Hayat mHealth application from January to March 2024, following EPI guidelines. Additionally, to verify that a child had received the vaccination, field visits were conducted using a convenience sampling method, and parents were asked to provide vaccination cards for comparison with the entries in the Hayat application. Ethical considerations Ethical approval was obtained from the Ethical Review Committee of Aga Khan University, the country’s National Bioethics Committee, the Ministry of Health, or the relevant authority before the project was initiated. Training for the evaluators was conducted to ensure uniformity throughout the evaluation process. The rights and dignity of all participants were respected, and privacy and confidentiality of the data were maintained throughout the study and thereafter. All personally identifiable information collected was securely stored and protected with password encryption. Access to data was restricted to the research team. The data will be retained for up to 7 years in accordance with Aga Khan University policies. All data were anonymized to prevent any link to individual participants and were used solely for research purposes. Statistical Analysis Frequencies and percentages were reported for vaccinators in healthcare facilities who were familiar with all the essential features of the Hayat mHealth application. The difference percentage between the daily register(s) and the Hayat mHealth application for each antigen was assessed with a cutoff limit of 10%, indicating that the acceptable difference percentage should not exceed this threshold. The percentage of children's records matching the vaccine card and the Hayat mHealth application was calculated for all the essential vaccines (BCG, Penta 1, Penta 2, Penta 3, MR 1, and MR 2). These vaccines represent six scheduled interactions between the child and healthcare providers for routine immunization against vaccine-preventable diseases (VPDs) by the age of 15 months. All the analysis was conducted using STATA version 17.0. Results Table 1 a presents the percentage difference (delta) between daily register(s) and the Hayat mHealth application for each vaccine antigen in the upper and lower Chitral. The analysis was conducted using data from eight sites (four from each district) that routinely utilized the “vaccinate now” feature in the Hayat mHealth application. The “vaccinate now” feature enables vaccinators to record information on children’s vaccination while administering them, thus reflecting the accuracy of the data concerning its comparison from the daily register(s). The cumulative delta for each vaccine antigen was less than 10% except for Penta 1. Table 1 a: Percentage difference between the daily register and the Hayat mHealth application for each antigen in Upper and Lower Districts of KPK Vaccine Antigen Delta - Upper Chitral Delta - Lower Chitral Delta – Cumulative BCG 14.28 2.64 6.57 Penta 1 8.42 2.96 11.76 Penta 2 14.54 11.85 0 Penta 3 5.15 19.17 9.66 MR 1 5.10 11.71 9.19 *The analysis was conducted using data taken from eight sites (four from each district) religiously utilizing the “vaccinate now” feature in the Hayat application. *Cumulative reporting was taken from Jan-Mar 2024. *Values within 10% are denoted in green. Table 2 a shows the percentage of vaccinators in healthcare facilities who were familiar with all essential features of the Hayat mHealth application in upper Chitral, lower Chitral and cumulatively for both districts of KPK. In upper Chitral eight vaccinators (42.10%), while in lower Chitral 11 vaccinators (57.89%) were evaluated. Using Hayat, all 19 vaccinators (100%) could perform the new registration of a child inside UC, recording previous vaccinations for a child residing inside UC, and searching for a child using a QR code, name, computerized national identification card (CNIC), mobile phone, and other identifiers (see Appendix 2: Vaccinator Evaluation). Findings suggest that some vaccinators lack knowledge about data sharing (14, 73.6%), data merging (11, 57.8%), and the ‘not vaccinated’ feature (13, 68.4%) (refer to Table 2 a). According to some vaccinators, the current version of the Hayat mHealth application is a significant improvement as compared to the previous version. Furthermore, some vaccinators suggest including data editing capabilities, particularly an option to change the vaccination date for a child in case it was entered incorrectly. Many of them expressed satisfaction with the digital transition. Cumulatively, the user satisfaction reported by vaccinators in using Hayat mHealth application was 100% in across both districts of KPK. Table 2 a: Percentage of vaccinators (n = 19) in healthcare facilities who were familiar with all the essential features of the Hayat application district-wise Upper Chitral Lower Chitral Cumulative Participating Vaccinators 8 (42.10%) 11 (57.89%) 19 (100%) Registration New registration of a child outside the UC 8 (100%) 10 (90.90%) 18 (94.7%) Record previous vaccinations for a child residing outside the UC 8 (100%) 10 (90.90%) 18 (94.7%) Defaulters Identify of the children who are the defaulters of vaccination(s) 8 (100%) 9 (81.81%) 17 (89.4%) Vaccination coverage for children who have missed their appointments 8 (100%) 10 (90.90%) 18 (94.7%) Stock Vaccinators, who received training in stock management within the application 6 (85.71%) 10 (90.9%) 16 (84.2%) Stock returns or wastage after opening a vial of BCG or Typhoid Conjugate Vaccine (TCV), etc. 7 (87.5%) 10 (90.9%) 17 (89.4%) Add stock in the application with batch number and expiry 7 (87.5%) 10 (90.9%) 17 (89.4%) Technical Sync data from the application to the server 8 (100%) 10 (90.9%) 18 (94.7%) Assign or reassign the QR code of the registered child 7 (87.5%) 10 (90.9%) 17 (89.4%) Share data with/without an internet connection 6 (85.7%) 8 (72.7%) 14 (73.6%) Record reason for a child not getting vaccinated 5 (62.5%) 8 (72.7%) 13 (68.4%) Merge records of children/women 4 (50%) 7 (63.6%) 11 (57.8%) *Vaccinators were purposively selected from healthcare facilities for this evaluation. Figure 1 below illustrates the percentage of children's records matching between the vaccination card and the Hayat mHealth application in the upper Chitral, lower Chitral, and cumulatively for both districts in KPK. A total of 91 children were visited, including 46 from upper Chitral and 45 from lower Chitral. The matching of the records exceeded 95% for all the essential vaccines (BCG, Penta 1, Penta 2, Penta 3, MR 1, and MR 2) in the upper Chitral, lower Chitral, and both districts of KPK. Discussion During site visits to healthcare facilities, evaluators made several observations related to vaccine management, which were subjective in nature and not systematically assessed against specific, predefined parameters. A key issue identified was that vaccinators consistently recorded stock-in data in the Hayat mHealth application but neglected to enter corresponding stock-out information. This oversight led to incomplete and inaccurate stock management within the system. The failure to document stock-out data disrupted the accurate tracking of vaccine inventory levels, as the application lacked a comprehensive record of vaccines dispensed. As a result, the evaluators encountered significant challenges in comparing manual stock records with the data in the Hayat mHealth application. This discrepancy was particularly evident when attempting to assess vaccine returns and wastage. Without proper documentation of stock-out events, it became difficult to distinguish between vaccines that had been returned or wasted and those that had been administered. This lack of synchronization between manual and digital records highlights the critical need for more comprehensive data entry practices in the Hayat mHealth application to ensure proper stock control and reduce the risk of vaccine wastage or expiration. The findings align with the study that highlighted the potential of mobile devices to address the stock-out challenges in healthcare settings, as it emphasizes that their effectiveness depends on accurate and consistent data entry by healthcare workers. This supports the observation that incomplete documentation of stock-out events disrupts the inventory tracking and compromises the systems’ reliability ( 8 ). A recurring issue identified during the evaluation was that many vaccinators relied on the "already vaccinated" feature in the Hayat mHealth application, rather than entering data in real time using the "vaccinate now" feature. Nearly half of the healthcare facilities visited showed limited use of the "vaccinate now" feature, contributing to data entry gaps and discrepancies between manual records and the Hayat mHealth application. This delay in data entry hindered the accurate recording of vaccine administration and led to neglect of stock-related information, particularly stock-out data, which is essential for effective inventory management. Consequently, the system lacked real-time tracking, complicating vaccine coverage monitoring and wastage assessment. Additionally, some vaccinators face challenges in entering data simultaneously due to being overburdened by their workload, prioritizing clinical tasks over data entry. These issues highlight the need for improved training, better workload management, and adherence to standard operating procedures. The SAGE working group findings align with Hayat mHealth evaluation as it emphasizes that data quality depends on its accurate, timely and complete data entry at the facility level and similar to the issue identified with already vaccinated feature. The SAGE findings highlighted need to improve workforce capacity, reduce workload burdens and ensue adherence to standardized processes to enhance real time data tracking also underline importance of targeted training in order to optimize immunization program management( 9 ). Moreover, there were some technical challenges identified during the evaluation that further impacted the accuracy and effectiveness of the vaccination tracking system. One major issue was the enablement of the option to select a date before the due date for the "already vaccinated" feature. This functionality led to inaccurate data entry, as vaccinators occasionally selected incorrect vaccination dates, causing discrepancies between the actual vaccination date and the recorded data in the system. This error in date selection compromised the integrity of the vaccination records, which could have serious implications for monitoring vaccine coverage and adherence to immunization schedules. Another technical issue related to data sharing was the identification of defaulters at the UC level rather than at the individual user level. While data sharing can be an effective tool for tracking missed vaccinations, this broad-level identification made it difficult to pinpoint specific defaulters or target interventions by individual vaccinators for particular catchment areas of healthcare facilities. This lack of granularity in data sharing limited the ability to take timely and targeted action to address vaccination gaps at the individual level. To address these complications, the improvements in systems data entry validation and also refinement of data sharing mechanisms are essential for accurate tracking and focused interventions at individual level. Additionally, at several non-governmental healthcare facilities, the documentation of vaccinations according to the EPI protocol was inaccurate. Furthermore, it was observed that training on key features of the mHealth application, such as data sharing, data merging, and the "not vaccinated" feature, was not consistently provided to vaccinators in some healthcare facilities. Without adequate training on these features, vaccinators were unable to fully utilize the system’s capabilities, leading to underreporting or mismanagement of vaccination data. This gap in training further exacerbated the challenges in ensuring accurate and real-time tracking of vaccination coverage and addressing defaulters. These technical issues, coupled with gaps in training, highlight the need for system adjustments and including improved data entry validation mechanism and also enhanced training protocols to ensure that vaccinators can effectively use the Hayat mHealth application to manage vaccine data and improve immunization outcomes. By focusing all these gaps, the system functionality can be optimized to vaccination tracking, leading to better targeted interventions and improving immunization coverage. Way forward The acceptability of digital data systems among vaccinators in Upper and Lower Chitral highlights the readiness of healthcare providers to embrace modern tools for data management. Feedback from the field indicates that vaccinators appreciated the ease of use and functionality of mHealth applications like Hayat, which provided real-time data entry and tracking capabilities. alignment with the strategic direction, the new and modified version of the Hayat application that is KPEIR is planned for rollout across 34 districts of Kyber Pakhtunkhwa KPK. The Hayat EPI vaccinator and facility stock management application aims to enhance vaccine administration by implementing the detailed comprehensive tracking measures. This includes vaccine administration to children and mothers, while also ensuring detailed monitoring of the vaccinator responsible for retrieving vaccines from the supply. Therefore, by maintain the thorough vaccination history for each patient this project would document when and where the vaccines are administrated. This integrated approach will be going to strength the vaccine delivery and accountability within the program and ultimately improving overall immunization efforts. To ensure the successful adoption of KPEIR system, frequent reinforcement training for vaccinators would be essential. These training courses should focus on equipping vaccinators with the skills required to fully utilize the application's features, including real-time data entry, stock management, and defaulter tracking. Reinforcement training sessions can address initial challenges, clarify operational procedures, and provide vaccinators with a platform to share their experiences and solutions. The inclusion of targeted support mechanisms, such as regular supervisory visits and technical assistance, will further strengthen the vaccinators' capacity to manage digital immunization records effectively. A phased rollout of the modified Hayat KPEIR across all 34 districts of KPK is recommended to ensure smooth transition and successful implementation. The phased approach will allow for the identification and resolution of potential challenges in smaller pilot areas before scaling to larger regions. Early adopters, such as Upper and Lower Chitral, can serve as model districts, providing valuable insights and lessons to guide the broader implementation. This gradual approach will also enable the systematic allocation of resources, including infrastructure, training, and technical support, to optimize the adoption process. Conclusion The evaluation of the Hayat application in the Upper Chitral and Lower Chitral districts of KPK provided invaluable insights that are instrumental in shaping the broader implementation strategy of the application across the region. A comparative analysis of data taken from manual daily registers assessed Hayat’s efficacy and accuracy of data management practices. Furthermore, the evaluation assessed vaccinator competency in using the Hayat application identifying areas for potential training and support. Community visits were integral to the evaluation. They helped validate the consistency and reliability of the information on vaccination cards compared to the data recorded within the Hayat application. The verification process was essential to ensure data integrity and trustworthiness. The evaluation not only laid the groundwork for broader deployment of the Hayat application but also underscored the importance for a strategic rollout across all districts in KPK. Taking feedback from frontline healthcare workers and community stakeholders will further improve the system’s feasibility and usability. The insights gained from this evaluation are invaluable for future planning and decision-making, ensuring Hayat’s deployment is carried out with efficacy and accuracy. This systematic approach aims to achieve sustainable improvements within the provincial and national health care infrastructure that would promote long term health outcomes and ensure equitable access to improved immunization services. Declarations Ethics approval and consent to participate Ethical approval for the study was granted by the Aga Khan University Ethical Review Committee (ERC) under application number 2018-0375-951. All procedures were carried out following the Declaration of Helsinki. Approval for the visit was obtained from representatives of the healthcare facilities for the purpose of the evaluation. Voluntary informed consent was obtained from all participants prior to initiating the evaluation/data collection. Consent for publication Not applicable Availability of data and materials The data analyzed in this study contain private healthcare facility information and is not publicly accessible. However, they can be obtained from the corresponding author upon reasonable request. Competing interest The authors declare that they have no competing interests. Funding The study was funded by Grand Challenges Canada (GCC) and Aga Khan Foundation Canada (AKFC) Clinical Trial Number Not applicable Author information Authors and Affiliations Research Instructor, Department of Radiology, Aga Khan University, Karachi, Pakistan Saira Samnani (SS) Research Associate, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan Dr. Nisha Asif (NA) Senior Manager, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan Abdul Muqeet (AM) Manager, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan Ahsan Nawaz (AN) Director, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan Saleem Sayani (SSI) Author’s contribution SS, SSI, AM, and AN contributed to the conceptualization of the study. SS, SSI, AM, and AN contributed to the methodology formulation. SS, AM, and AN developed the study proforma, which was reviewed by SSI. SS conducted the statistical analysis. SS, AM and AN were responsible for project management. NA initiated manuscript writing, which was further expanded by SS and reviewed by SS, SSI, AM, and AN. SSI oversaw the entire research project, providing advice and feedback throughout the study. All authors read and approved of the final manuscript. References Sharma S, Kumari B, Ali A, Yadav RK, Sharma AK, Sharma KK et al (2022) Mobile technology: A tool for healthcare and a boon in pandemic. J Family Med Prim Care [Internet]. ;11(1):37–43. Available from: http://www.ncbi.nlm.nih.gov/pubmed/35309626 Addotey-Delove M, Scott RE, Mars M (2023) Healthcare Workers’ Perspectives of mHealth Adoption Factors in the Developing World: Scoping Review, vol 20. International Journal of Environmental Research and Public Health. MDPI Wallis L, Blessing P, Dalwai M, Shin S, Do (2017) Integrating mHealth at point of care in low- and middle-income settings: The system perspective. Glob Health Action ;10 Mccool J, Dobson R, Whittaker R, Paton C (2024) Mobile Health (mHealth) in Low-and Middle-Income Countries. Annual Review of Public Health Annu Rev Public Health [Internet]. ;50:17. Available from: https://doi.org/10.1146/annurev-publhealth- Hernandez-Green N, Davis MV, Farinu O, Hernandez-Spalding K, Lewis K, Beshara MS et al (2024) Using mHealth to reduce disparities in Black maternal health: Perspectives from Black rural postpartum mothers. Women’s Health. ;20 Ganesan AN, Louise J, Horsfall M, Bilsborough SA, Hendriks J, Mcgavigan AD et al (2016) International Mobile-Health Intervention on Physical Activity, Sitting, and Weight The Stepathlon Cardiovascular Health Study Njoroge M, Zurovac D, Ogara EAA, Chuma J, Kirigia D (2017) Assessing the feasibility of eHealth and mHealth: A systematic review and analysis of initiatives implemented in Kenya. BMC Res Notes. ;10(1) Iwu CJ, Ngcobo N, Cooper S, Mathebula L, Mangqalaza H, Magwaca A et al (2020) Mobile reporting of vaccine stock-levels in primary health care facilities in the Eastern Cape Province of South Africa: perceptions and experiences of health care workers. Hum Vaccin Immunother 16(8):1911–1917 Scobie HM, Edelstein M, Nicol E, Morice A, Rahimi N, MacDonald NE et al (2020) Improving the quality and use of immunization and surveillance data: Summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization. Vaccine. Elsevier Ltd, pp 7183–7197 Appendix Appendix not available with this version. Additional Declarations No competing interests reported. Supplementary Files Questionnaireforevaluation.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 27 Aug, 2025 Reviews received at journal 26 Aug, 2025 Reviews received at journal 17 Aug, 2025 Reviews received at journal 13 Aug, 2025 Reviewers agreed at journal 09 Aug, 2025 Reviewers agreed at journal 08 Aug, 2025 Reviewers agreed at journal 07 Aug, 2025 Reviewers agreed at journal 03 Aug, 2025 Reviews received at journal 09 Jul, 2025 Reviews received at journal 03 Jul, 2025 Reviews received at journal 28 Jun, 2025 Reviewers agreed at journal 27 Jun, 2025 Reviewers agreed at journal 27 Jun, 2025 Reviewers agreed at journal 26 Jun, 2025 Reviewers agreed at journal 25 Jun, 2025 Reviewers agreed at journal 25 Jun, 2025 Reviewers agreed at journal 25 Jun, 2025 Reviewers invited by journal 25 Jun, 2025 Editor assigned by journal 25 Jun, 2025 Editor invited by journal 13 Jun, 2025 Submission checks completed at journal 12 Jun, 2025 First submitted to journal 12 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6828546","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":470165845,"identity":"91eee309-230c-470b-8b05-54b61e00909e","order_by":0,"name":"Saira Samnani","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/ElEQVRIiWNgGAWjYFACxgY480BChQ1IpPEA8VoenEkDixDQgqz7YdthiF58quTbD7d9+PHHLtqc/ezBA4lt5+3Wth8G2lJjE41Li8GZxOaZvW3JuTt78hIOJJy7nbztTCJQy7G03AZcWhgSmxl4G5hzNxzIMTiQUHY72ewAUAtjw2GcWuT7HzYz/vlTn7vh/BugFrZzyWbnH+LXwnAjsZmZh+1w7oYbIFvaDtiZ3SBgi8GNh83Msm3HgVpAtpxJTjC7AbQlAY9f5PvTHzO++VMNdFiO8ccfFXb2ZufTHz74UGOD22HoIBGsMoFY5SBgT4riUTAKRsEoGBkAAAMVcA4IYAs0AAAAAElFTkSuQmCC","orcid":"","institution":"Aga Khan University","correspondingAuthor":true,"prefix":"","firstName":"Saira","middleName":"","lastName":"Samnani","suffix":""},{"id":470165846,"identity":"b71b5fed-12fd-4119-b80e-6f52ccdd645a","order_by":1,"name":"Nisha Asif","email":"","orcid":"","institution":"Aga Khan University","correspondingAuthor":false,"prefix":"","firstName":"Nisha","middleName":"","lastName":"Asif","suffix":""},{"id":470165847,"identity":"1f022772-4260-4329-a345-eb3cf6e70a79","order_by":2,"name":"Abdul Muqeet","email":"","orcid":"","institution":"Aga Khan University","correspondingAuthor":false,"prefix":"","firstName":"Abdul","middleName":"","lastName":"Muqeet","suffix":""},{"id":470165848,"identity":"5e325a6f-ffe6-4e14-a1ef-bd34b65913e4","order_by":3,"name":"Ahsan Nawaz","email":"","orcid":"","institution":"Aga Khan University","correspondingAuthor":false,"prefix":"","firstName":"Ahsan","middleName":"","lastName":"Nawaz","suffix":""},{"id":470165849,"identity":"0b505fa8-2351-4376-9144-e9490b48ae46","order_by":4,"name":"Saleem Sayani","email":"","orcid":"","institution":"Aga Khan University","correspondingAuthor":false,"prefix":"","firstName":"Saleem","middleName":"","lastName":"Sayani","suffix":""}],"badges":[],"createdAt":"2025-06-05 11:08:27","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6828546/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6828546/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":84678084,"identity":"0beb80bb-ab4e-4d03-9a51-24a0c76b8e5d","added_by":"auto","created_at":"2025-06-16 07:54:27","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":15056,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePercentage of children's records matching between vaccination card and Hayat\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e91 children (46 from Upper Chitral, 45 from Lower Chitral) from the selected UCs were visited using a convenient sampling method, and their parents were requested to provide vaccination cards for comparison with the entries recorded in the Hayat app.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-6828546/v1/46225d2b717a091ef26b2c50.png"},{"id":84678800,"identity":"8bc6fd43-e124-4a7d-a021-30831f99c364","added_by":"auto","created_at":"2025-06-16 08:01:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":680316,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6828546/v1/47baef45-ca22-4f92-b551-aff29b86c0ba.pdf"},{"id":84678083,"identity":"2435087f-82ee-426e-ad3c-b1f062d94355","added_by":"auto","created_at":"2025-06-16 07:54:26","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":25014,"visible":true,"origin":"","legend":"","description":"","filename":"Questionnaireforevaluation.docx","url":"https://assets-eu.researchsquare.com/files/rs-6828546/v1/85266e53e22b2009b31a63e2.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Strengthening Expanded Program for Immunization Service Delivery through the Hayat mHealth Application: A Cross-Sectional Study in Upper and Lower Chitral, Pakistan","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn recent years, mobile health (mHealth) technologies have emerged as transformative tools for enhancing healthcare delivery, particularly in low-resource settings (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e), where access to quality care is often constrained by various barriers, including infrastructure limitations and workforce shortages. One of the most significant advancements facilitated by mHealth is the efficiency and reliability of data collection using mobile devices. Unlike traditional pen-and-paper methods, mobile data collection offers faster processing and reduces the risk of errors, ensuring higher accuracy in capturing and managing health information (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). By leveraging the accessibility and convenience of mobile platforms, mHealth applications empower healthcare workers to extend critical services to remote and underserved communities. These applications address logistical challenges, such as difficult terrain and inadequate transportation networks, while also mitigating the impact of limited healthcare infrastructure (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). For example, immunization tracking, remote patient monitoring, and disease surveillance are increasingly being facilitated through mobile solutions, ensuring that healthcare reaches even the most isolated populations (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe significance of mHealth is particularly pronounced in low-and middle-income countries (LMICs), where it serves as an important bridge to overcome systemic healthcare challenges. By leveraging mHealth, LMICs addressed critical gaps in healthcare accessibility, affordability and quality. mHealth provides scalable solutions for the management of disease, emergency response and patient education which are hindered often by financial and logistical barriers(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). mHealth platforms significantly enable real-time data sharing among professionals and communication accessibility between healthcare workers and central facilities that fostered more coordinated and responsive quality care delivery to patients, however significantly in resource constrained environments where timely interventions have lifesaving implications. For maternal and child health, mHealth has proven to be instrumental in improving postnatal, antenatal care through education, communication and promoting care-seeking behaviors. Through mHealth technologies in LMICs, governments and organizations to effectively reach vulnerable populations vulnerable populations effectively, ensuring better maternal health outcomes (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eFurther studies illustrated potential of mhealth, example research conducted in Georgia, highlighted potential of mHealth in addressing maternal health challenges among mothers in rural areas. Through Interviews with 14 participants the study identified barriers such as limited healthcare facilities, racial bias in providers interactions, the results showed that participants emphasized the potential of mhealth applications in offering virtual support and resources. However, this study highlighted the need for patient-centered practices through mhealth applications(\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). Another large-scale study was conducted in LMIC over 100 days on Stepthlon mHealth programs that included 69,000 participants, 92% were from developing countries. This program focused on an accessible platform for tracking activity and encouraging healthy behaviors in resource limited setting. These findings highlighted the potential of mHealth, that overcome barriers to reach underserved communities and support sustainable interventions and bridge gaps in healthcare accessibility(\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). One more study conducted that review and categories mHealth initiatives in Kenya, identifying different active projects focused on improving healthcare access and outcomes in underserved regions. The study highlighted the importance of health in addressing healthcare disparities, particularly in rural and marginalized areas. They targeted vulnerable populations, including those also living in remote areas, with a particular focus on maternal and child health, disease prevention and any emergency responses. These findings explored the potential of mhealth to increased healthcare delivery service where traditional services are limited (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eGlobally, countries are prioritizing investments in mobile networks over traditional infrastructure such as road transport and power generation, reflecting the critical role that mobile connectivity plays in driving innovation and expanding access to essential services (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). One such innovative mHealth application is Hayat, designed to empower health workers by enhancing the accessibility and effectiveness of Reproductive, Maternal, Newborn, and Child Health (RMNCH), immunizations, and child growth services, particularly in remote or underserved areas of Pakistan. It streamlines the operations of various health worker roles, including Lady Health Workers (LHWs), vaccinators, and their supervisors, by digitizing paper-based forms. It enables health workers to efficiently record service details, track patients, and raise awareness within communities through educational videos. Till present date, the Hayat application has been able to register 482,835 beneficiaries with over 192,000 educational awareness sessions conducted, with an average number of 10 participants per session. Additionally, it features a supervisory application for remote monitoring, conducting audits, and verifying monthly reports. Hayat was recognized internationally for its novelty and innovativeness at the International Digital Health Awards 2023. Since its launch, the Hayat has demonstrated significant potential in enhancing service utilization among the population. It currently operates in two districts of Khyber Pakhtunkhwa (KPK) that is Upper and Lower Chitral and five districts of Gilgit Baltistan (GB) including Hunza, Nagar, Astore, Ghizer, and Skardu. However, governments in LMIC often face a range of challenges and competing priorities, which limit their capacity to adopt new innovations. As a result, they require strong, credible evidence on mHealth projects to evaluate their potential alongside other essential health interventions, as well as guidance on which mHealth solutions are most likely to contribute to achieving broader health system goals(\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Therefore, KPK provincial EPI team with public private partnership aims to evaluate the usability and effectiveness of the Hayat mHealth application in complementing traditional manual daily registers for accurate data recording by health workers. The objective of an evaluation was to determine the accuracy of data, ensuring that the data from the manual daily register(s) is accurately reflected in the Hayat application.\u003c/p\u003e"},{"header":"Methodology","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStudy design and study sites\u003c/h2\u003e \u003cp\u003eA cross-sectional study was conducted to assess the usability of the Hayat mHealth application among vaccinators in upper and lower Chitral districts of KPK. The study sites included all five union councils (UCs) across upper and lower Chitral, encompassing 63 healthcare facilities where the Hayat mHealth application was being used alongside manual EPI registers. These registers include the daily register(s), maintained separately by each vaccinator, which record all vaccinations administered by that vaccinator, regardless of whether the children reside inside or outside the respective UCs. Additionally, the permanent register exclusively documents vaccinations of all children residing within the respective UCs.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eStudy population\u003c/h3\u003e\n\u003cp\u003eThe study population includes all the healthcare facilities in upper and lower Chitral using the Hayat mHealth application alongside manual EPI registers for recording children's vaccination data.\u003c/p\u003e\n\u003ch3\u003eEligibility criteria\u003c/h3\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eInclusion\u003c/h2\u003e \u003cp\u003eAll the healthcare facilities in upper and lower Chitral using the Hayat mHealth application alongside manual EPI registers for recording children's vaccination data were eligible for inclusion in the study.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eExclusion\u003c/h3\u003e\n\u003cp\u003eAll the healthcare facilities that had not yet implemented or adopted the Hayat m Health application during the study period.\u003c/p\u003e \u003cp\u003eFacilities with missing, incomplete, or inconsistent data in the manual EPI registers.\u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eSampling technique\u003c/h2\u003e \u003cp\u003eSimple random probability sampling was employed to ensure the unbiased selection of healthcare facilities, wherein healthcare facilities from all five UCs in both upper and lower Chitral districts were randomly selected using computer-generated random numbers. Subsequently, 63 healthcare facilities within these UCs were visited by the KPK provincial EPI team for evaluation.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eData collection tools and procedures\u003c/h3\u003e\n\u003cp\u003eThree evaluation teams were established, each supported by a member of the Aga Khan University (AKU) team to document the evaluation process. A structured questionnaire was developed by a team of experts to evaluate the efficiency of vaccinators in using the Hayat mHealth application (refer to the supplementary file). Given the purpose of the evaluation, it was essential to assess the accuracy of the data recorded by vaccinators in both the Hayat application and their daily registers to identify any discrepancies. To achieve this, the vaccinations recorded in the vaccinators' daily register(s) were compared with those in their Hayat mHealth application from January to March 2024, following EPI guidelines. Additionally, to verify that a child had received the vaccination, field visits were conducted using a convenience sampling method, and parents were asked to provide vaccination cards for comparison with the entries in the Hayat application.\u003c/p\u003e\n\u003ch3\u003eEthical considerations\u003c/h3\u003e\n\u003cp\u003eEthical approval was obtained from the Ethical Review Committee of Aga Khan University, the country\u0026rsquo;s National Bioethics Committee, the Ministry of Health, or the relevant authority before the project was initiated. Training for the evaluators was conducted to ensure uniformity throughout the evaluation process. The rights and dignity of all participants were respected, and privacy and confidentiality of the data were maintained throughout the study and thereafter. All personally identifiable information collected was securely stored and protected with password encryption. Access to data was restricted to the research team. The data will be retained for up to 7 years in accordance with Aga Khan University policies. All data were anonymized to prevent any link to individual participants and were used solely for research purposes.\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eStatistical Analysis\u003c/h2\u003e \u003cp\u003eFrequencies and percentages were reported for vaccinators in healthcare facilities who were familiar with all the essential features of the Hayat mHealth application. The difference percentage between the daily register(s) and the Hayat mHealth application for each antigen was assessed with a cutoff limit of 10%, indicating that the acceptable difference percentage should not exceed this threshold. The percentage of children's records matching the vaccine card and the Hayat mHealth application was calculated for all the essential vaccines (BCG, Penta 1, Penta 2, Penta 3, MR 1, and MR 2). These vaccines represent six scheduled interactions between the child and healthcare providers for routine immunization against vaccine-preventable diseases (VPDs) by the age of 15 months. All the analysis was conducted using STATA version 17.0.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea presents the percentage difference (delta) between daily register(s) and the Hayat mHealth application for each vaccine antigen in the upper and lower Chitral. The analysis was conducted using data from eight sites (four from each district) that routinely utilized the \u0026ldquo;vaccinate now\u0026rdquo; feature in the Hayat mHealth application. The \u0026ldquo;vaccinate now\u0026rdquo; feature enables vaccinators to record information on children\u0026rsquo;s vaccination while administering them, thus reflecting the accuracy of the data concerning its comparison from the daily register(s). The cumulative delta for each vaccine antigen was less than 10% except for Penta 1.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ea: Percentage difference between the daily register and the Hayat mHealth application for each antigen in Upper and Lower Districts of KPK\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVaccine Antigen\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDelta -\u003c/p\u003e \u003cp\u003eUpper Chitral\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDelta -\u003c/p\u003e \u003cp\u003eLower Chitral\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDelta \u0026ndash; Cumulative\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBCG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e14.28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e6.57\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePenta 1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e11.76\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePenta 2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e14.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e11.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePenta 3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5.15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e19.17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e9.66\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMR 1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5.10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e11.71\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e9.19\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cem\u003e*The analysis was conducted using data taken from eight sites (four from each district) religiously utilizing the \u0026ldquo;vaccinate now\u0026rdquo; feature in the Hayat application.\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003e*Cumulative reporting was taken from Jan-Mar 2024.\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003e*Values within 10% are denoted in green.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003ea shows the percentage of vaccinators in healthcare facilities who were familiar with all essential features of the Hayat mHealth application in upper Chitral, lower Chitral and cumulatively for both districts of KPK. In upper Chitral eight vaccinators (42.10%), while in lower Chitral 11 vaccinators (57.89%) were evaluated. Using Hayat, all 19 vaccinators (100%) could perform the new registration of a child inside UC, recording previous vaccinations for a child residing inside UC, and searching for a child using a QR code, name, computerized national identification card (CNIC), mobile phone, and other identifiers (see Appendix 2: Vaccinator Evaluation). Findings suggest that some vaccinators lack knowledge about data sharing (14, 73.6%), data merging (11, 57.8%), and the \u0026lsquo;not vaccinated\u0026rsquo; feature (13, 68.4%) (refer to Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003ea). According to some vaccinators, the current version of the Hayat mHealth application is a significant improvement as compared to the previous version. Furthermore, some vaccinators suggest including data editing capabilities, particularly an option to change the vaccination date for a child in case it was entered incorrectly. Many of them expressed satisfaction with the digital transition. Cumulatively, the user satisfaction reported by vaccinators in using Hayat mHealth application was 100% in across both districts of KPK.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ea: Percentage of vaccinators (n\u0026thinsp;=\u0026thinsp;19) in healthcare facilities who were familiar with all the essential features of the Hayat application district-wise\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUpper Chitral\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLower Chitral\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eCumulative\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eParticipating Vaccinators\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8 (42.10%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e11 (57.89%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e19 (100%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eRegistration\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNew registration of a child outside the UC\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8 (100%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.90%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e18 (94.7%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRecord previous vaccinations for a child residing outside the UC\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8 (100%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.90%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e18 (94.7%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eDefaulters\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eIdentify of the children who are the defaulters of vaccination(s)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8 (100%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e9 (81.81%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e17 (89.4%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVaccination coverage for children who have missed their appointments\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8 (100%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.90%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e18 (94.7%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eStock\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVaccinators, who received training in stock management within the application\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6 (85.71%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.9%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e16 (84.2%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStock returns or wastage after opening a vial of BCG or Typhoid Conjugate Vaccine (TCV), etc.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e7 (87.5%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.9%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e17 (89.4%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdd stock in the application with batch number and expiry\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e7 (87.5%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.9%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e17 (89.4%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eTechnical\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSync data from the application to the server\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8 (100%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.9%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e18 (94.7%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAssign or reassign the QR code of the registered child\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e7 (87.5%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10 (90.9%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e17 (89.4%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eShare data with/without an internet connection\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6 (85.7%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8 (72.7%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e14 (73.6%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRecord reason for a child not getting vaccinated\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5 (62.5%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8 (72.7%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13 (68.4%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMerge records of children/women\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4 (50%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7 (63.6%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e11 (57.8%)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cem\u003e*Vaccinators were purposively selected from healthcare facilities for this evaluation.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e below illustrates the percentage of children's records matching between the vaccination card and the Hayat mHealth application in the upper Chitral, lower Chitral, and cumulatively for both districts in KPK. A total of 91 children were visited, including 46 from upper Chitral and 45 from lower Chitral. The matching of the records exceeded 95% for all the essential vaccines (BCG, Penta 1, Penta 2, Penta 3, MR 1, and MR 2) in the upper Chitral, lower Chitral, and both districts of KPK.\u003c/p\u003e "},{"header":"Discussion","content":"\u003cp\u003eDuring site visits to healthcare facilities, evaluators made several observations related to vaccine management, which were subjective in nature and not systematically assessed against specific, predefined parameters. A key issue identified was that vaccinators consistently recorded stock-in data in the Hayat mHealth application but neglected to enter corresponding stock-out information. This oversight led to incomplete and inaccurate stock management within the system. The failure to document stock-out data disrupted the accurate tracking of vaccine inventory levels, as the application lacked a comprehensive record of vaccines dispensed. As a result, the evaluators encountered significant challenges in comparing manual stock records with the data in the Hayat mHealth application. This discrepancy was particularly evident when attempting to assess vaccine returns and wastage. Without proper documentation of stock-out events, it became difficult to distinguish between vaccines that had been returned or wasted and those that had been administered. This lack of synchronization between manual and digital records highlights the critical need for more comprehensive data entry practices in the Hayat mHealth application to ensure proper stock control and reduce the risk of vaccine wastage or expiration. The findings align with the study that highlighted the potential of mobile devices to address the stock-out challenges in healthcare settings, as it emphasizes that their effectiveness depends on accurate and consistent data entry by healthcare workers. This supports the observation that incomplete documentation of stock-out events disrupts the inventory tracking and compromises the systems\u0026rsquo; reliability (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eA recurring issue identified during the evaluation was that many vaccinators relied on the \"already vaccinated\" feature in the Hayat mHealth application, rather than entering data in real time using the \"vaccinate now\" feature. Nearly half of the healthcare facilities visited showed limited use of the \"vaccinate now\" feature, contributing to data entry gaps and discrepancies between manual records and the Hayat mHealth application. This delay in data entry hindered the accurate recording of vaccine administration and led to neglect of stock-related information, particularly stock-out data, which is essential for effective inventory management. Consequently, the system lacked real-time tracking, complicating vaccine coverage monitoring and wastage assessment. Additionally, some vaccinators face challenges in entering data simultaneously due to being overburdened by their workload, prioritizing clinical tasks over data entry. These issues highlight the need for improved training, better workload management, and adherence to standard operating procedures. The SAGE working group findings align with Hayat mHealth evaluation as it emphasizes that data quality depends on its accurate, timely and complete data entry at the facility level and similar to the issue identified with already vaccinated feature. The SAGE findings highlighted need to improve workforce capacity, reduce workload burdens and ensue adherence to standardized processes to enhance real time data tracking also underline importance of targeted training in order to optimize immunization program management(\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eMoreover, there were some technical challenges identified during the evaluation that further impacted the accuracy and effectiveness of the vaccination tracking system. One major issue was the enablement of the option to select a date before the due date for the \"already vaccinated\" feature. This functionality led to inaccurate data entry, as vaccinators occasionally selected incorrect vaccination dates, causing discrepancies between the actual vaccination date and the recorded data in the system. This error in date selection compromised the integrity of the vaccination records, which could have serious implications for monitoring vaccine coverage and adherence to immunization schedules. Another technical issue related to data sharing was the identification of defaulters at the UC level rather than at the individual user level. While data sharing can be an effective tool for tracking missed vaccinations, this broad-level identification made it difficult to pinpoint specific defaulters or target interventions by individual vaccinators for particular catchment areas of healthcare facilities. This lack of granularity in data sharing limited the ability to take timely and targeted action to address vaccination gaps at the individual level. To address these complications, the improvements in systems data entry validation and also refinement of data sharing mechanisms are essential for accurate tracking and focused interventions at individual level.\u003c/p\u003e \u003cp\u003eAdditionally, at several non-governmental healthcare facilities, the documentation of vaccinations according to the EPI protocol was inaccurate. Furthermore, it was observed that training on key features of the mHealth application, such as data sharing, data merging, and the \"not vaccinated\" feature, was not consistently provided to vaccinators in some healthcare facilities. Without adequate training on these features, vaccinators were unable to fully utilize the system\u0026rsquo;s capabilities, leading to underreporting or mismanagement of vaccination data. This gap in training further exacerbated the challenges in ensuring accurate and real-time tracking of vaccination coverage and addressing defaulters. These technical issues, coupled with gaps in training, highlight the need for system adjustments and including improved data entry validation mechanism and also enhanced training protocols to ensure that vaccinators can effectively use the Hayat mHealth application to manage vaccine data and improve immunization outcomes. By focusing all these gaps, the system functionality can be optimized to vaccination tracking, leading to better targeted interventions and improving immunization coverage.\u003c/p\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003eWay forward\u003c/h2\u003e \u003cp\u003eThe acceptability of digital data systems among vaccinators in Upper and Lower Chitral highlights the readiness of healthcare providers to embrace modern tools for data management. Feedback from the field indicates that vaccinators appreciated the ease of use and functionality of mHealth applications like Hayat, which provided real-time data entry and tracking capabilities. alignment with the strategic direction, the new and modified version of the Hayat application that is KPEIR is planned for rollout across 34 districts of Kyber Pakhtunkhwa KPK. The Hayat EPI vaccinator and facility stock management application aims to enhance vaccine administration by implementing the detailed comprehensive tracking measures. This includes vaccine administration to children and mothers, while also ensuring detailed monitoring of the vaccinator responsible for retrieving vaccines from the supply. Therefore, by maintain the thorough vaccination history for each patient this project would document when and where the vaccines are administrated. This integrated approach will be going to strength the vaccine delivery and accountability within the program and ultimately improving overall immunization efforts.\u003c/p\u003e \u003cp\u003eTo ensure the successful adoption of KPEIR system, frequent reinforcement training for vaccinators would be essential. These training courses should focus on equipping vaccinators with the skills required to fully utilize the application's features, including real-time data entry, stock management, and defaulter tracking. Reinforcement training sessions can address initial challenges, clarify operational procedures, and provide vaccinators with a platform to share their experiences and solutions. The inclusion of targeted support mechanisms, such as regular supervisory visits and technical assistance, will further strengthen the vaccinators' capacity to manage digital immunization records effectively.\u003c/p\u003e \u003cp\u003eA phased rollout of the modified Hayat KPEIR across all 34 districts of KPK is recommended to ensure smooth transition and successful implementation. The phased approach will allow for the identification and resolution of potential challenges in smaller pilot areas before scaling to larger regions. Early adopters, such as Upper and Lower Chitral, can serve as model districts, providing valuable insights and lessons to guide the broader implementation. This gradual approach will also enable the systematic allocation of resources, including infrastructure, training, and technical support, to optimize the adoption process.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe evaluation of the Hayat application in the Upper Chitral and Lower Chitral districts of KPK provided invaluable insights that are instrumental in shaping the broader implementation strategy of the application across the region. A comparative analysis of data taken from manual daily registers assessed Hayat\u0026rsquo;s efficacy and accuracy of data management practices. Furthermore, the evaluation assessed vaccinator competency in using the Hayat application identifying areas for potential training and support. Community visits were integral to the evaluation. They helped validate the consistency and reliability of the information on vaccination cards compared to the data recorded within the Hayat application. The verification process was essential to ensure data integrity and trustworthiness. The evaluation not only laid the groundwork for broader deployment of the Hayat application but also underscored the importance for a strategic rollout across all districts in KPK. Taking feedback from frontline healthcare workers and community stakeholders will further improve the system\u0026rsquo;s feasibility and usability. The insights gained from this evaluation are invaluable for future planning and decision-making, ensuring Hayat\u0026rsquo;s deployment is carried out with efficacy and accuracy. This systematic approach aims to achieve sustainable improvements within the provincial and national health care infrastructure that would promote long term health outcomes and ensure equitable access to improved immunization services.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEthical approval for the study was granted by the Aga Khan University Ethical Review Committee (ERC) under application number 2018-0375-951. All procedures were carried out following the Declaration of Helsinki. Approval for the visit was obtained from representatives of the healthcare facilities for the purpose of the evaluation. Voluntary informed consent was obtained from all participants prior to initiating the evaluation/data collection.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe data analyzed in this study contain private healthcare facility information and is not publicly accessible. However, they can be obtained from the corresponding author upon reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study was funded by Grand Challenges Canada (GCC) and Aga Khan Foundation Canada (AKFC)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical Trial Number\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor information\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAuthors and Affiliations\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eResearch Instructor, Department of Radiology, Aga Khan University, Karachi, Pakistan\u003c/p\u003e\n\u003cp\u003eSaira Samnani (SS)\u003c/p\u003e\n\u003cp\u003eResearch Associate, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan\u003c/p\u003e\n\u003cp\u003eDr. Nisha Asif (NA)\u003c/p\u003e\n\u003cp\u003eSenior Manager, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan\u003c/p\u003e\n\u003cp\u003eAbdul Muqeet (AM)\u003c/p\u003e\n\u003cp\u003eManager, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan\u003c/p\u003e\n\u003cp\u003eAhsan Nawaz (AN)\u003c/p\u003e\n\u003cp\u003eDirector, Aga Khan Development Network Digital Health Resource Centre, Aga Khan University, Karachi, Pakistan\u003c/p\u003e\n\u003cp\u003eSaleem Sayani (SSI)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor\u0026rsquo;s contribution\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSS, SSI, AM, and AN contributed to the conceptualization of the study. SS, SSI, AM, and AN contributed to the methodology formulation. SS, AM, and AN developed the study proforma, which was reviewed by SSI. SS conducted the statistical analysis. \u0026nbsp;SS, AM and AN were responsible for project management. NA initiated manuscript writing, which was further expanded by SS and reviewed by SS, SSI, AM, and AN. SSI oversaw the entire research project, providing advice and feedback throughout the study. All authors read and approved of the final manuscript.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eSharma S, Kumari B, Ali A, Yadav RK, Sharma AK, Sharma KK et al (2022) Mobile technology: A tool for healthcare and a boon in pandemic. J Family Med Prim Care [Internet]. ;11(1):37\u0026ndash;43. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.ncbi.nlm.nih.gov/pubmed/35309626\u003c/span\u003e\u003cspan address=\"http://www.ncbi.nlm.nih.gov/pubmed/35309626\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAddotey-Delove M, Scott RE, Mars M (2023) Healthcare Workers\u0026rsquo; Perspectives of mHealth Adoption Factors in the Developing World: Scoping Review, vol 20. International Journal of Environmental Research and Public Health. MDPI\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWallis L, Blessing P, Dalwai M, Shin S, Do (2017) Integrating mHealth at point of care in low- and middle-income settings: The system perspective. Glob Health Action ;10\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMccool J, Dobson R, Whittaker R, Paton C (2024) Mobile Health (mHealth) in Low-and Middle-Income Countries. Annual Review of Public Health Annu Rev Public Health [Internet]. ;50:17. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1146/annurev-publhealth-\u003c/span\u003e\u003cspan address=\"10.1146/annurev-publhealth-\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHernandez-Green N, Davis MV, Farinu O, Hernandez-Spalding K, Lewis K, Beshara MS et al (2024) Using mHealth to reduce disparities in Black maternal health: Perspectives from Black rural postpartum mothers. Women\u0026rsquo;s Health. ;20\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGanesan AN, Louise J, Horsfall M, Bilsborough SA, Hendriks J, Mcgavigan AD et al (2016) International Mobile-Health Intervention on Physical Activity, Sitting, and Weight The Stepathlon Cardiovascular Health Study\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNjoroge M, Zurovac D, Ogara EAA, Chuma J, Kirigia D (2017) Assessing the feasibility of eHealth and mHealth: A systematic review and analysis of initiatives implemented in Kenya. BMC Res Notes. ;10(1)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIwu CJ, Ngcobo N, Cooper S, Mathebula L, Mangqalaza H, Magwaca A et al (2020) Mobile reporting of vaccine stock-levels in primary health care facilities in the Eastern Cape Province of South Africa: perceptions and experiences of health care workers. Hum Vaccin Immunother 16(8):1911\u0026ndash;1917\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eScobie HM, Edelstein M, Nicol E, Morice A, Rahimi N, MacDonald NE et al (2020) Improving the quality and use of immunization and surveillance data: Summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization. Vaccine. Elsevier Ltd, pp 7183\u0026ndash;7197\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Appendix","content":"\u003cp\u003eAppendix not available with this version.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-digital-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Digital Health](https://bmcdigitalhealth.biomedcentral.com/)","snPcode":"44247","submissionUrl":"https://submission.nature.com/new-submission/44247/3","title":"BMC Digital Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"mHealth application, BCG, Penta, MR, LMIC (low-middle income countries)","lastPublishedDoi":"10.21203/rs.3.rs-6828546/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6828546/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eBackground\u003c/b\u003e\u003c/p\u003e \u003cp\u003eMobile health (mhealth) technologies are revolutionizing and transforming healthcare delivery, particularly in low resource settings, by improving data accuracy, accessibility and decision making. This study aimed to evaluate the effectiveness and usability of the Hayat m-health application in enhancing immunization service delivery ae well as data accuracy, compared to traditional manual EPI registers in Upper and Lower Chitral, Khyber Pakhtunkhwa KPK, Pakistan.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethodology\u003c/b\u003e\u003c/p\u003e \u003cp\u003eA cross-sectional study conducted across 63 healthcare facilities using structured tools and data comparisons from the month of January to March 2024.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe results showed that cumulative percentage difference (delta) between Hayat and manual records remained within acceptable 10% threshold for most vaccine antigens, indicating strong data accuracy. Whereas 100% of the 19 participating vaccinators demonstrated proficiency in features including registration and child search, although gaps were identified in advanced functions, like data integration and offline sharing. Vaccinator satisfaction with the application was undisputed. The matching of record between Hayat Application and vaccination cards exceeded 95% for all the essential antigens (BCG, Penta 1, Penta 2, Penta 3, MR 1, and MR 2). Therefore, these results support Hayat Application potential for improving immunization data accuracy and service delivery in the remote areas.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusion\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe Hayat mhealth application demonstrated strong potential for improving immunization of data accuracy and service delivery in remote areas and resource limited settings.\u003c/p\u003e","manuscriptTitle":"Strengthening Expanded Program for Immunization Service Delivery through the Hayat mHealth Application: A Cross-Sectional Study in Upper and Lower Chitral, Pakistan","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-16 07:53:16","doi":"10.21203/rs.3.rs-6828546/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-08-27T09:36:27+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-26T04:43:58+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-17T12:19:46+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-13T04:43:55+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"246418648321759932118392668594690214574","date":"2025-08-09T04:54:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"80823496577309761234294911357172399766","date":"2025-08-08T17:55:54+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"39872913885565827502158665726251639147","date":"2025-08-07T08:44:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"29683907746357924668251621264021437232","date":"2025-08-04T01:22:31+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-09T19:03:35+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-03T16:00:05+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-06-29T01:32:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"158613177077602646269020654407035558765","date":"2025-06-27T15:31:18+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"118566764891936524634997780829667944694","date":"2025-06-27T15:28:38+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"5026579340720135824996417398043950619","date":"2025-06-26T08:50:15+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"199056368772052278680130205673845164117","date":"2025-06-25T15:21:27+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"121917429738850622475406253217341431731","date":"2025-06-25T15:04:52+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"238901498190659976906891401294672976316","date":"2025-06-25T14:59:13+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-06-25T14:51:40+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-06-25T13:33:53+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-06-13T10:05:35+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-12T06:12:24+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Digital Health","date":"2025-06-12T06:09:17+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-digital-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Digital Health](https://bmcdigitalhealth.biomedcentral.com/)","snPcode":"44247","submissionUrl":"https://submission.nature.com/new-submission/44247/3","title":"BMC Digital Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"9a24ca8f-a972-43ec-8415-8f8c31cc2c38","owner":[],"postedDate":"June 16th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-01-05T09:53:31+00:00","versionOfRecord":[],"versionCreatedAt":"2025-06-16 07:53:16","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6828546","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6828546","identity":"rs-6828546","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}