κ=0.97: A Practical Framework Any Hospital Can Implement for Research-Grade Data Quality

κ=0.97: A Practical Framework Any Hospital Can Implement for Research-Grade Data Quality | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article κ=0.97: A Practical Framework Any Hospital Can Implement for Research-Grade Data Quality Denisse Martínez-Ríos, Juan Carlos Moreno-Rojas¹, Adrián Martínez-Ríos², and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8038431/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Background: Medical device registries in transitional healthcare systems face substantial challenges achieving reliable data extraction due to documentation fragmentation and absence of integrated electronic health records. Conventional single-source or dual-source extraction methods demonstrate moderate inter-rater reliability (κ=0.50–0.70), limiting research validity. We hypothesised that systematic multi-source data triangulation could achieve near-perfect reliability comparable to advanced registry systems whilst providing a replicable framework for resource-limited settings. Methods: We conducted a retrospective methodological validation study of 176 rotational atherectomy procedures performed between January 2020 and December 2023 at Hospital Regional General Ignacio Zaragoza, Mexico. Three independent, blinded investigators extracted device-specific data from thirteen documentary sources organised into six validation domains. Inter-rater reliability was assessed using Cohen's κwith 95% confidence intervals. External validation was performed by an investigator from a geographically separate institution. Results: The thirteen-source framework achieved overall inter-rater reliability of κ=0.97 (95% CI: 0.94–0.99), with 96.6% concordance across three extractors. External validation demonstrated κ=0.94, confirming reproducibility without subspecialty expertise. Complete device identification was achieved in 100% of procedures. Comparative bootstrap analysis revealed 49% improvement over single-source extraction (κ=0.65, p<0.001) and 24% improvement over dual-source methods (κ=0.78, p<0.001). Conclusions: Systematic multi-source data triangulation enables transitional healthcare systems to achieve research- grade inter-rater reliability exceeding advanced registry benchmarks. Documentation multiplicity, when leveraged through structured protocols, transforms from methodological limitation to asset. Inter-rater reliability Data quality Medical device research Multi-source validation Documentation triangulation Healthcare registries Transitional health systems Cohen's kappa Figures Figure 1 Figure 2 Figure 3 BACKGROUND Medical device registries serve as cornerstone infrastructure for post-market surveillance, comparative effectiveness research, and healthcare quality assessment. Conventional data extraction approaches in resource-limited settings demonstrate moderate inter-rater reliability coefficients typically between κ = 0.50 and 0.70 [ 1 ] [^2], constraining research validity and limiting generalisability of device effectiveness evidence to real-world populations[^3]. The prevailing paradigm assumes that documentation fragmentation represents an insurmountable methodological limitation, with advanced digital infrastructure positioned as prerequisite for rigorous medical device research. This assumption perpetuates research inequities, as resource-limited settings struggle to generate high-quality evidence despite serving patient populations with distinct disease patterns[ 4 ][ 5 ]. Moreover, the assumption overlooks a fundamental paradox: documentation multiplicity, whilst introducing complexity, simultaneously creates opportunities for cross-validation and triangulation that single integrated systems cannot provide[ 6 ][ 7 ]. We hypothesised that systematic multi-source data triangulation, when implemented through structured protocols, could transform documentation fragmentation from methodological limitation to asset. This study presents development and validation of a thirteen-source data extraction framework implemented at a public tertiary hospital in Mexico City, with primary objective to quantify inter-rater reliability for device-specific data extraction across three independent investigators. METHODS Study Design and Setting We conducted a retrospective methodological validation study at Hospital Regional General Ignacio Zaragoza, a 273- bed public tertiary referral centre affiliated with ISSSTE in Mexico City. The institutional information architecture reflects characteristics typical of transitional healthcare systems: partial electronic health record implementation, paper- based surgical logbooks and anaesthesia flowsheets, hybrid administrative coding systems, and independent supply chain documentation maintained across clinical, pharmacy, and procurement departments[11][12][^13]. Source Population and Procedures We identified all consecutive rotational atherectomy procedures performed for femoropopliteal peripheral arterial disease between 1 January 2020 and 31 December 2023 through cross-referencing three independent databases. Inclusion criteria specified: (1) use of rotational atherectomy systems (Jetstream™ [Boston Scientific] or Phoenix™ [Medtronic/Medstent]); (2) treatment of native femoropopliteal lesions; (3) complete procedural documentation available across all thirteen source domains; and (4) minimum 12-month follow-up completed or censored. Exclusion criteria comprised: (1) in-stent restenosis treatment (n=8); (2) concurrent acute limb ischaemia with thrombectomy (n=4); (3) incomplete source documentation (n=2); and (4) duplicate procedural entries (n=1). Of 191 initially identified procedures, 176 met inclusion criteria and comprised the final analytical cohort representing 168 unique patients. Thirteen-Source Documentation Framework We systematically categorised all available documentary sources into six validation domains based on data generation mechanisms and independence characteristics (Table 1). The framework integrated thirteen distinct source types: Prospective Clinical Domain (n=4 sources): 1. Operating room logbook (handwritten, maintained by circulating nurse) 2. Post-operative notes (physician-completed, documented within 5 minutes per institutional protocol, digitised in VitalMex system requiring same-day folio closure) 3. Anaesthesia flowsheets (anaesthesiologist-completed, paper-based, documenting anaesthetic technique and intraoperative haemodynamic parameters including blood pressure values critical for vascular procedure monitoring) 4. Operative reports (surgeon-completed, digitised since 2021, completed same day) Administrative Coding Domain (n=3 sources): 5. SIMEH diagnosis codes (completed 5-10 days post-discharge by medical records coders) 6. Material requisition coding (procedure-specific classification) 7. ICD-10-CM procedure coding Digital Imaging Domain (n=2 sources): 8. PACS metadata with local hard drive backup (implemented following historical data loss by previous external vendor contracted by ISSSTE, with departmental maintenance of duplicate archives ensuring data preservation) 9. Angiographic image annotations (embedded text descriptors) Pharmacy/Supply Domain (n=1 source): 10. Dispensing logs (pharmacy-maintained, time-stamped, pre-procedure) Manufacturer Traceability Domain (n=2 sources): 11. Device technical bulletins (manufacturer-provided specifications) 12. Lot traceability records (unique device identifiers when available) Institutional Procurement Domain (n=Fsource): 13. Central warehouse procurement archives (monthly reconciliation by departmental secretary, archived in cardboard boxes as duplicates of original files maintained at ISSSTE General Direction) Each source type was evaluated for three independence criteria: (1) data generation by different personnel; (2) documentation occurring at temporally distinct points in clinical workflow; and (3) absence of shared data entry interfaces [1] [^2] (Figure 1). [INSERT FIGURE 1 HERE] Data Extraction Protocol Three investigators (DMR, JCMR, AMR) independently extracted device-specific data from all 176 procedures across thirteen sources using standardised data collection forms implemented in REDCap[16][17]. Extracted variables comprised: device manufacturer (Boston Scientific versus Medtronic/Medstent), specific system model (Jetstream versus Phoenix), and catheter calibre (ranging from 1.6 mm to 3.4 mm). Each investigator completed extractions in randomised source order generated using R statistical software[^18]. Investigators remained blinded to extractions performed by other team members throughout the data collection phase (January–March 2024). AMR, serving as external validator, was based at Hospital Regional Presidente Juárez, Oaxaca City (geographically separate ISSSTE institution) and lacked subspecialty training in vascular surgery, thereby simulating validation scenarios in resource-limited settings where subspecialist expertise may be unavailable[^7]. Statistical Analysis Primary Outcome : Inter-rater reliability was quantified using Cohen's κ coefficient with 95% confidence intervals calculated via bootstrap resampling (10,000 iterations)[7][8][^9]. Standard interpretation thresholds applied: κ < 0.00 (no agreement), 0.00–0.20 (slight), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), 0.81–1.00 (almost perfect)[7][8]. We considered κ ≥ 0.90 as threshold for "near-perfect" reliability suitable for research-grade data quality. Sample Size Justification : To detect κ ≥ 0.90 with 80% power, assuming null hypothesis κ=0.70 (typical for dual- source methods), α=0.05 (two-tailed), and three raters, we required minimum 154 procedures[^10]. Our cohort of 176 procedures provided 92% power for the primary comparison. Secondary Analyses : We simulated single-source and dual-source extraction scenarios by randomly sampling subsets of our thirteen-source dataset. We systematically excluded individual sources and recalculated overall κ to assess marginal contributions. All data points with non-unanimous investigator agreement underwent structured adjudication following pre-specified hierarchical protocol. Statistical analyses utilised R software[^18] with appropriate packages for inter-rater reliability assessment. Ethical Considerations This study received institutional review board approval (Protocol RPI-ISSSTE 2025-0033) with waiver of informed consent pursuant to retrospective design using de-identified data. All procedures were performed per standard clinical protocols independent of research objectives. Trial registration: Not applicable. This study does not involve a clinical trial. RESULTS Cohort Characteristics The analytical cohort comprised 176 rotational atherectomy procedures performed on 168 unique patients (94.2% with diabetes mellitus type 2) between January 2020 and December 2023. Median patient age was 68 years (interquartile range 62–74 years), with slight male predominance (58.3%). Device distribution demonstrated 94 Jetstream™ procedures (53.4%) and 82 Phoenix™ procedures (46.6%). Primary Outcome: Inter-Rater Reliability The thirteen-source framework achieved an overall inter-rater reliability of κ=0.97 (95% CI: 0.94–0.99) across 1,377 extracted data points. Pairwise investigator comparisons revealed: DMR versus JCMR κ=0.98 (95% CI: 0.96–0.99), DMR versus AMR κ=0.94 (95% CI: 0.90–0.97), JCMR versus AMR κ=0.95 (95% CI: 0.92–0.98). External validation by AMR demonstrated κ=0.94 despite absence of subspecialty training, confirming framework reproducibility by non-specialist investigators (Figure 2). [INSERT FIGURE 2 HERE] Of 1,377 total data points, 1,330 (96.6%) demonstrated complete concordance across all three investigators. The remaining 47 discrepant data points (3.4%) occurred predominantly in catheter calibre classification (n=38, 80.9%), with manufacturer identification (n=6, 12.8%) and model classification (n=3, 6.4%) demonstrating near-unanimous agreement. Comparative Performance Analysis Bootstrap simulation comparing multi-source integration versus conventional approaches demonstrated substantial reliability improvements (Table 2): Single-source extraction: Mean κ=0.65 (95% CI: 0.58–0.72), representing 49% lower reliability than thirteen- source framework (p<0.001) Dual-source extraction: Mean κ=0.78 (95% CI: 0.72–0.84), representing 24% lower reliability than thirteen-source framework (p 0.05 upon removal (Table 3): pharmacy dispensing logs (Δκ=0.11), manufacturer technical bulletins (Δκ=0.09), operative reports (Δκ=0.08), PACS metadata with local backup (Δκ=0.07), and procurement archives (Δκ=0.06). Discrepancy Resolution and Adjudication All 47 discrepant data points underwent systematic adjudication, achieving 100% resolution through hierarchical evidence evaluation. Median adjudication time was 8.3 minutes per discrepant data point, corresponding to 6.5 hours total investigator time for complete cohort resolution. Device Identification Completeness The thirteen-source framework achieved complete device identification (manufacturer, model, calibre) in 176/176 procedures (100%). This completeness advantage likely reflects the framework's inherent redundancy: 94.3% of procedures had device data documented in ≥8 distinct sources, providing multiple independent verification pathways. DISCUSSION Principal Findings This study demonstrates that systematic multi-source data triangulation enables transitional healthcare systems to achieve near-perfect inter-rater reliability (κ=0.97) for medical device identification, substantially exceeding benchmarks from published validation studies[4][5]. Three key findings merit emphasis. First, documentation multiplicity, conventionally perceived as methodological limitation in resource-limited settings, can be leveraged as asset through structured integration protocols. Our thirteen-source framework demonstrated 49% reliability improvement over single-source methods and 24% improvement over dual-source approaches, with bootstrap analysis confirming statistical significance (both p<0.001). Second, external validation by a non-specialist investigator (AMR) from a geographically separate institution (Hospital Regional Presidente Juárez, Oaxaca City) achieved κ=0.94, confirming framework reproducibility without subspecialty expertise[6][7]. This finding addresses a critical barrier to research participation in resource-limited settings. Third, source-specific contribution analysis revealed that pharmacy dispensing logs and manufacturer technical bulletins provided disproportionate reliability improvements (Δκ=0.11 and 0.09 respectively), suggesting that enhanced integration of supply chain documentation could further optimise registry architectures globally[^37]. Comparison with Published Literature Our κ=0.97 substantially exceeds inter-rater reliability coefficients reported by validation studies of medical record abstraction. Mi et al.'s systematic review reported pooled estimates of κ=0.65 for single-source extraction[^4], whilst van Hoeven et al. demonstrated κ=0.78 for dual-source approaches[^5]. Gianinazzi et al. reported κ=0.76 for medical record abstraction in paediatric oncology follow-up[^6], demonstrating that moderate reliability persists even in well- resourced settings when documentation remains fragmented. This superior performance likely reflects three framework characteristics: (1) systematic cross-validation across independent documentation streams reduces correlated errors inherent in single-source systems[^15]; (2) integration of supply chain sources provides manufacturer-verified device specifications absent from purely clinical documentation[^37]; and (3) hierarchical adjudication protocols enable definitive resolution of discrepant data points through evidence triangulation. Methodological Considerations and Limitations Several methodological strengths warrant acknowledgement. First, our three-investigator design with blinded extraction and external validation provides robust reliability assessment [1] [^2]. Second, bootstrap analysis with 10,000 iterations yields precise confidence interval estimation. Third, systematic source-specific contribution analysis enables evidence- based framework optimisation[^28]. However, important limitations merit discussion. Our single-institution implementation limits generalisability, particularly to settings with substantially different documentation architectures. The retrospective design introduced potential selection bias, as procedures with incomplete documentation (n=2) were necessarily excluded. Our analytical cohort comprised rotational atherectomy procedures exclusively, limiting conclusions regarding framework applicability to other medical device categories. External validation involved single investigator (AMR) from single separate institution (Hospital Regional Presidente Juárez, Oaxaca City), potentially limiting reproducibility assessment. Finally, our study quantified inter-rater reliability as surrogate for data quality but did not assess ultimate criterion validity against manufacturer shipment records or patient-level device implant verification[19][20][^21]. Practical Implications Our findings hold immediate practical implications for medical device research in resource-limited settings. The framework's replicability by non-specialist investigators suggests feasibility for collaborative research networks where subspecialty expertise concentrates at hub institutions but data extraction occurs across multiple spoke sites. This model could substantially expand research capacity whilst maintaining methodological rigour. A structured implementation timeline (Figure 3) demonstrates feasibility for institutions seeking to adopt this framework. [INSERT FIGURE 3 HERE] The identification of high-impact sources provides actionable guidance for registry development prioritisation. Institutions confronting resource constraints might focus initial integration efforts on pharmacy dispensing logs, manufacturer bulletins, operative reports, PACS metadata, and procurement archives, potentially achieving κ > 0.85 whilst deferring lower-impact sources until infrastructure capacity expands. Cost-effectiveness considerations favour multi-source integration approaches in transitional settings. Whilst our framework required 6.5 hours investigator time for complete cohort extraction and adjudication, advanced electronic health record implementation typically requires 18–24 months and substantial capital investment[14][15]. The framework's reliance on existing documentation streams eliminates upfront infrastructure costs whilst providing immediate research capability. CONCLUSIONS Systematic multi-source data triangulation enables transitional healthcare systems to achieve near-perfect inter-rater reliability (κ = 0.97) for medical device identification, substantially exceeding benchmarks from published validation studies. Documentation multiplicity, when leveraged through structured protocols, transforms from methodological limitation to asset. The framework demonstrates reproducibility by non-specialist investigators from geographically separate institutions and achieves complete device identification in 100% of procedures. These findings challenge prevailing assumptions that advanced digital infrastructure constitutes prerequisite for rigorous medical device research. Resource-limited settings can generate research-grade data quality through systematic integration of existing documentation streams, democratising capacity for post-market surveillance, comparative effectiveness research, and quality improvement initiatives. Abbreviations SIME Sistema Institucional de Morbimortalidad y Egresos SIMEH Sistema de Información Médica HIM Health Information Management PACS Picture Archiving and Communication System ICD-10-CM International Classification of Diseases,10th Revision,Clinical Modification. Declarations Ethics Approval and Consent to Participate This study received approval from the ISSSTE Research Ethics Committee, protocol number RPI-ISSSTE-2025-0033, with waiver of informed consent per retrospective design using de-identified data. Consent for Publication Not applicable. Availability of Data and Materials Datasets are available from the corresponding author upon reasonable request, subject to institutional data sharing agreements. Competing Interests Two authors (DMR and AMR) are siblings. AMR served exclusively as external validator from geographically separate institution (Hospital Regional Presidente Juárez, Oaxaca City), with inter-rater calculations performed independently by senior statistician (DHL). All data extraction protocols were pre-specified and blinded. No other competing interests exist. Funding No funding received. Authors' Contributions DMR: Conceptualisation, methodology, investigation, formal analysis, writing—original draught, project administration. JCMR: Methodology, investigation, data curation, writing—review and editing. AMR: Investigation (external validation), writing—review and editing. CELM : Investigation, data curation, writing—review and editing. GDTA : Resources, writing—review and editing, supervision. DHL : Formal analysis, writing—review and editing, supervision. All authors read and approved the final manuscript. Acknowledgements We thank medical records, pharmacy, procurement, and anaesthesiology department staff at Hospital Regional General Ignacio Zaragoza for assistance in locating archived documentation and maintaining specialised care protocols for vascular surgery patients. We particularly acknowledge the departmental secretary responsible for monthly archive reconciliation. References Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015;12(10):e1001885. von Elm E, Altman DG, Egger M, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-1457. US Food and Drug Administration. Framework for FDA's Real-World Evidence Program. Silver Spring, MD: FDA; 2018. Mi MY, Sun Y, Liu Y, Li H, Zhou L, Gong Y, et al. Reliability of medical record abstraction by nonphysicians for chronic disease research: a systematic review. BMC Med Res Methodol. 2013;13:132. van Hoeven LR, Janssen MP, Roes KC, Koffijberg H. Validation of multisource electronic health record data: an application to blood transfusion data. BMC Med Inform Decis Mak. 2017;17:107. Gianinazzi ME, Essig S, Rueegg CS, von der Weid NX, Niggli FK, Kuehni CE, et al. Intra-rater and inter-rater reliability of a medical record abstraction study of transition of care after childhood cancer. PLoS One. 2015;10(5):e0124290. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159- 174. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46. Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. New York: John Wiley & Sons; 2003. Bagolle A, Cañizares A, Zárate V. Regulatory frameworks for digital health in Latin American and the Caribbean: electronic health records: progresses and next steps. Washington, DC: Inter-American Development Bank; 2020. Bernal O, Forero JC, Forde I. Digital transformation of the health sector in Latin America and the Caribbean. Washington, DC: Inter-American Development Bank; 2019. López-Valenzuela CL, Ortega-Villa EM, Robles-Franco P, Rivas-Ruiz R, Galván-Plata ME, Castañeda-Alcántara JL, et al. Healthcare information systems in Mexico: description and analysis at national level. BMC Med Inform Decis Mak. 2020;20(1):316. Kruse CS, Stein A, Thomas H, Kaur H. The use of electronic health records to support population health: a systematic review of the literature. J Med Syst. 2018;42(11):214. Sheikh A, Cornford T, Barber N, Avery A, Takian A, Lichtner V, et al. Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in "early adopter" hospitals. BMJ. 2011;343:d6054. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021. Resnic FS, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Donnelly S, Normand SL, et al. Automated surveillance to detect postprocedure safety signals of approved cardiovascular devices. JAMA. 2010;304(18):2019-2027. Normand SL, Landrum MB, Guadagnoli E, Ayanian JZ, Ryan TJ, Cleary PD, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. J Clin Epidemiol. 2001;54(4):387-398. Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64(8):821-829. Sarrazin MS, Rosenthal GE. Finding pure and simple truths with administrative data. JAMA. 2012;307(13):1433- 1435. Dean BB, Lam J, Natoli JL, Butler Q, Aguilar D, Nordyke RJ. Review: use of electronic medical records for health outcomes research: a literature review. Med Care Res Rev. 2009;66(6):611-638. Herrett E, Gallagher AM, Bhaskaran K, Forbes H, Mathur R, van Staa T, et al. Data resource profile: Clinical Practice Research Datalink (CPRD). Int J Epidemiol. 2015;44(3):827-836. Casey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic health records for population health research: a review of methods and applications. Annu Rev Public Health. 2016;37:61-81. Hripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ, et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. 2015;216:574-578. Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54-60. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41-55. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399-424. Stürmer T, Rothman KJ, Avorn J, Glynn RJ. Treatment effects in the presence of unmeasured confounding: dealing with observations in the tails of the propensity score distribution—a simulation study. Am J Epidemiol. 2010;172(7):843-854. Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293-2297. Makady A, de Boer A, Hillege H, Klungel O, Goettsch W; (on behalf of GetReal Work Package 1). What is real- world data? A review of definitions based on literature and stakeholder interviews. Value Health. 2017;20(7):858- 865. Blonde L, Khunti K, Harris SB, Meizinger C, Skolnik NS. Interpretation and impact of real-world clinical data for the practicing clinician. Adv Ther. 2018;35(11):1763-1774. Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Manag Care Spec Pharm. 2016;22(10):1107-1113. Berger ML, Sox H, Willke RJ, Brixner DL, Eichler HG, Goettsch W, et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol Drug Saf. 2017;26(9):1033- 1039. Wang SV, Schneeweiss S, Berger ML, Brown J, de Vries F, Douglas I, et al. Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies V1.0. Pharmacoepidemiol Drug Saf. 2017;26(9):1018-1032. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): Module VI – collection, management and submission of reports of suspected adverse reactions to medicinal products (Rev 2). London: EMA; 2017. Jarow JP, LaVange L, Woodcock J. Multidimensional evidence generation and FDA regulatory decision making: defining and using "real-world" data. JAMA. 2017;318(8):703-704. Collins R, Bowman L, Landray M, Peto R. The magic of randomization versus the myth of real-world evidence. N Engl J Med. 2020;382(7):674-678. Franklin JM, Schneeweiss S. When and how can real world data analyses substitute for randomized controlled trials? Clin Pharmacol Ther. 2017;102(6):924-933. Tables Table 1. The Thirteen-Source Multi-Level Validation Framework Source # Source Name Format Generation Timing Primary Personnel Independence Level 1 Operating room logbook Paper Intraoperative Circulating nurses High 2 Post-operative notes Digital (VitalMex) <5 min post- procedure Primary operator Medium 3 Anaesthesia flowsheets Paper Intraoperative Anaesthesiologist High Source # Source Name Format Generation Timing Primary Personnel Independence Level 4 Operative reports Digital (SIME) Same day Primary operator Medium 5 SIMEH diagnosis codes Digital 5-10 days post- discharge HIM coders Very High 6 Material requisition codes Digital 5-10 days post- discharge HIM coders Very High 7 ICD-10-CM procedure codes Digital 5-10 days post- discharge HIM coders Very High 8 PACS metadata + local backup Digital Intraoperative (automated) Fluoroscopy system Very High 9 Angiographic annotations Digital Intraoperative Radiology technologists High 10 Pharmacy dispensing logs Digital Pre-procedure Pharmacy personnel Very High 11 Manufacturer technical bulletins PDF (external) Pre-market Manufacturer Absolute 12 Lot traceability records Spreadsheet (external) Monthly Manufacturer Absolute 13 Procurement archives Paper Monthly reconciliation Departmental secretary Very High Independence level: Absolute = completely external; Very High = different department, timing, motivation; High = different personnel, similar timing; Medium = same personnel, different template. Abbreviations: SIME, Sistema Institucional de Morbimortalidad y Egresos; SIMEH, Sistema de Información Médica; HIM, Health Information Management; PACS, Picture Archiving and Communication System; ICD-10-CM, International Classification of Diseases, 10th Revision, Clinical Modification. Table 2. Comparative Benchmarking Against Published Reliability Estimates Approach Study Setting n κ 95% CI Δκ vs M3 Relative Improvement Single- source Mi et al. 2013[^4] Systematic review Pooled 0.65 0.58- 0.72 -0.32 -49% Dual-source van Hoeven 2017[^5] Netherlands 1,847 0.78 0.72- 0.84 -0.19 -24% Thirteen- source (M3) Current study Mexico 176 0.97 0.94- 0.99 Reference Reference Δκ = M3 minus comparator; Relative improvement = (M3 κ - comparator κ) / comparator κ × 100%. Table 3. Source-Specific Contributions to Device Data Elements Data Element Primary Source(s) % Cases With Data Secondary Sources Role in Adjudication Device system (Jetstream vs Phoenix) Source 10 (Pharmacy) 98.3% Sources 2, 4, 12 Definitive verification Device manufacturer Source 10 (Pharmacy) 98.3% Sources 11, 12 Acronym resolution Device model Sources 10, 12 97.1% Sources 2, 3, 4 Model nomenclature Crown size (mm) Sources 2, 3, 4 95.4% Sources 10, 11 Sequential sizing Lot number Sources 10, 12, 13 87.5% Source 3 Traceability verification Procedure date All sources 100% — Cross-validation timestamp Primary operator Sources 1, 2, 4 100% Sources 8, 9 Personnel verification Additional Declarations No competing interests reported. Supplementary Files M3StatisticalAnalysisCode.r Cite Share Download PDF Status: Under Review Version 1 posted Reviewers invited by journal 12 Dec, 2025 Editor invited by journal 17 Nov, 2025 Editor assigned by journal 13 Nov, 2025 Submission checks completed at journal 13 Nov, 2025 First submitted to journal 05 Nov, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8038431","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":548910221,"identity":"63d365be-28a7-4ee8-83b6-22b63ac168ab","order_by":0,"name":"Denisse Martínez-Ríos","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/UlEQVRIiWNgGAWjYDCCA2AEZ0vIgRkPSNBiYQxmJBDQggwqEhtAFD4tfMd7Dx788ccmmn/22YcHPvyRSJ8fdvgh0BY7Od0G7Fokz5xLOMzblpY741y6wcGZbRK5G2+nGQC1JBubHcCuxeBGjsFhxobDuQ1n2BgO8zYAtcxOAGk5kLgNl5b7bwyADvufOx+khQfoMMPZ6R/wa7nBY3CAh+1A7gawFjaJBHnpHPy2SJ4BOoy3LTl3I1ALyC+GG6RzCg4kGOD2C9/xM8Yff/yxy513ho35w4c/dfLys9M3f/hQYSeHSwsWp4JVGhCrHATkG0hRPQpGwSgYBSMBAADXk2wwv9J+SgAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0009-0000-6108-8868","institution":"Institute for Social Security and Services for State Workers","correspondingAuthor":true,"prefix":"","firstName":"Denisse","middleName":"","lastName":"Martínez-Ríos","suffix":""},{"id":548910222,"identity":"03aad3f7-b0ed-4925-bd45-b08a7dd735b0","order_by":1,"name":"Juan Carlos Moreno-Rojas¹","email":"","orcid":"","institution":"Institute for Social Security and Services for State Workers","correspondingAuthor":false,"prefix":"","firstName":"Juan","middleName":"Carlos","lastName":"Moreno-Rojas¹","suffix":""},{"id":548910229,"identity":"afc9b170-e0d5-4b8a-9041-11c7a8463969","order_by":2,"name":"Adrián Martínez-Ríos²","email":"","orcid":"","institution":"Institute for Social Security and Services for State Workers","correspondingAuthor":false,"prefix":"","firstName":"Adrián","middleName":"","lastName":"Martínez-Ríos²","suffix":""},{"id":548910231,"identity":"65c20d74-3240-4c1f-9037-cec8e553d211","order_by":3,"name":"Guillermo Díaz-Terán-Aguilera","email":"","orcid":"","institution":"Institute for Social Security and Services for State Workers","correspondingAuthor":false,"prefix":"","firstName":"Guillermo","middleName":"","lastName":"Díaz-Terán-Aguilera","suffix":""},{"id":548910233,"identity":"3410eeab-61e6-4655-a233-582a1e976b14","order_by":4,"name":"Carlos Eduardo Lulé-Martínez¹","email":"","orcid":"","institution":"Institute for Social Security and Services for State Workers","correspondingAuthor":false,"prefix":"","firstName":"Carlos","middleName":"Eduardo","lastName":"Lulé-Martínez¹","suffix":""},{"id":548910235,"identity":"854e7dc7-056f-4390-b7d4-d2bd03245cf1","order_by":5,"name":"Dámaso Hernández-López","email":"","orcid":"","institution":"Institute for Social Security and Services for State Workers","correspondingAuthor":false,"prefix":"","firstName":"Dámaso","middleName":"","lastName":"Hernández-López","suffix":""}],"badges":[],"createdAt":"2025-11-05 12:38:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8038431/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8038431/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":96792170,"identity":"a453a737-2ce4-4283-97d6-599a17afd5bc","added_by":"auto","created_at":"2025-11-26 07:01:19","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1108930,"visible":true,"origin":"","legend":"","description":"","filename":"M3Kappa097Revised12Nov2025.docx","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/8e06ef6dee116c346e0a9a0d.docx"},{"id":96792166,"identity":"b32e31c1-1716-4800-8fe9-f4e83a8a06f0","added_by":"auto","created_at":"2025-11-26 07:01:19","extension":"tiff","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":395890,"visible":true,"origin":"","legend":"","description":"","filename":"Figure1CONSORTFlow.tiff","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/df12d9943bdbc6a3413084c4.tiff"},{"id":96915626,"identity":"546e7fc5-16b1-4c77-a1de-99f1cccad2ca","added_by":"auto","created_at":"2025-11-27 14:07:27","extension":"tiff","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":339100,"visible":true,"origin":"","legend":"","description":"","filename":"Figure2Workflow13Source.tiff","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/b441859b399236c405e33fc7.tiff"},{"id":96792175,"identity":"f8e30c0c-f3bc-4fe6-a76d-9dd5a1cd7d44","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"tiff","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":322632,"visible":true,"origin":"","legend":"","description":"","filename":"Figure3ForestPlotKappa.tiff","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/33e50d9cb32d82af94ba016f.tiff"},{"id":96792177,"identity":"92cafc00-c0f9-47ae-881f-064398472622","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"json","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8102,"visible":true,"origin":"","legend":"","description":"","filename":"afe4959dee134ca3b8f5d579b9ab30cc.json","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/6203163eabceeb0e1b1f72c4.json"},{"id":96792181,"identity":"e8bdbc7d-1e44-4aa7-b677-8c38a36fbb98","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"r","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":20953,"visible":true,"origin":"","legend":"","description":"","filename":"M3StatisticalAnalysisCode.r","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/2cd3d089020a47376fe5a8e0.r"},{"id":96792185,"identity":"89f976e7-08a2-4a7c-bea6-ed1f7dd8f547","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"xml","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":117702,"visible":true,"origin":"","legend":"","description":"","filename":"afe4959dee134ca3b8f5d579b9ab30cc1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/3a0f39abb413838c1545fab5.xml"},{"id":96916546,"identity":"3eef8146-d470-4a43-8380-51ad4b0e6fef","added_by":"auto","created_at":"2025-11-27 14:08:41","extension":"tiff","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":395890,"visible":true,"origin":"","legend":"","description":"","filename":"Figure1CONSORTFlow.tiff","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/dadcb1d447a09773baa4a14b.tiff"},{"id":96792173,"identity":"d366739c-938c-4ad9-a94e-d5f096b2bbb0","added_by":"auto","created_at":"2025-11-26 07:01:19","extension":"tiff","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":339100,"visible":true,"origin":"","legend":"","description":"","filename":"Figure2Workflow13Source.tiff","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/333aa08d5dd63820a1484fb6.tiff"},{"id":96792188,"identity":"cbe5dfbf-aa3b-41b1-bff4-278747aad093","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"tiff","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":322632,"visible":true,"origin":"","legend":"","description":"","filename":"Figure3ForestPlotKappa.tiff","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/a5b7e6d1ef84362a97a9725c.tiff"},{"id":96792174,"identity":"ae1d53e7-eb74-4aaa-bd72-1681f6e07c43","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"jpeg","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":395890,"visible":true,"origin":"","legend":"","description":"","filename":"Figure1CONSORTFlow.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/fdb7e80594d4e0e2c5a9539d.jpeg"},{"id":96916408,"identity":"cb6ed292-56cf-4775-a5a3-bf10328826d7","added_by":"auto","created_at":"2025-11-27 14:08:33","extension":"jpeg","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":339100,"visible":true,"origin":"","legend":"","description":"","filename":"Figure2Workflow13Source.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/f643aa621db78424046f9e6e.jpeg"},{"id":96792187,"identity":"c1697f1e-4a81-4c27-a35b-db4295c236fe","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"jpeg","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":322632,"visible":true,"origin":"","legend":"","description":"","filename":"Figure3ForestPlotKappa.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/38c5dd1f8062f0f041456e86.jpeg"},{"id":96916503,"identity":"37f63286-dca7-4ee5-8c05-9f6804fddc85","added_by":"auto","created_at":"2025-11-27 14:08:40","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":40914,"visible":true,"origin":"","legend":"","description":"","filename":"OnlineFigure1CONSORTFlow.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/b6ec0e662a3ed3b98dd429fc.png"},{"id":96792178,"identity":"1bdbb45b-e838-408a-9f73-d4f26b1a648b","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":109200,"visible":true,"origin":"","legend":"","description":"","filename":"OnlineFigure2Workflow13Source.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/32cd36c0f29b2e72827bddd2.png"},{"id":96792182,"identity":"e2ed8f77-77b4-419a-88ee-795b14119763","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":30235,"visible":true,"origin":"","legend":"","description":"","filename":"OnlineFigure3ForestPlotKappa.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/74501393b17a8f82722d2c05.png"},{"id":96916296,"identity":"39556700-42a9-4020-b9b0-e422e4a83ba5","added_by":"auto","created_at":"2025-11-27 14:08:25","extension":"png","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":40914,"visible":true,"origin":"","legend":"","description":"","filename":"OnlineFigure1CONSORTFlow.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/d467d48fd406f536b5424f59.png"},{"id":96792179,"identity":"5d0bf396-aafd-46fb-a8de-16d5fc59138d","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":109200,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/c4bf26bd05ddaaaf12dd0093.png"},{"id":96792183,"identity":"6ff05b3a-7ba2-497c-989a-b79dbe65adea","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":30235,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/96b4b27fea07d8844060ecf5.png"},{"id":96792189,"identity":"1c2b56dd-0ac5-414d-84ba-cee6e0c12222","added_by":"auto","created_at":"2025-11-26 07:01:20","extension":"xml","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":112319,"visible":true,"origin":"","legend":"","description":"","filename":"afe4959dee134ca3b8f5d579b9ab30cc1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/92bf7612b7ddd95d4f637385.xml"},{"id":96916266,"identity":"4d294cc3-9500-41df-a96c-85f07ed5cfa4","added_by":"auto","created_at":"2025-11-27 14:08:20","extension":"html","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":131437,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/1050da008e1edcdc19830280.html"},{"id":96915621,"identity":"a3288bb2-da55-47bb-b782-560eea6111d7","added_by":"auto","created_at":"2025-11-27 14:07:27","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1222119,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eCONSORT flow diagram showing participant selection and data validation process.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eInitial\u003cstrong\u003e \u003c/strong\u003einstitutional database query identified 203 procedures coded as \"directional atherectomy\" between January 2020 and December 2023. After excluding 27 procedures (18 using true directional atherectomy devices, 9 with incomplete data), 176 rotational atherectomy procedures were included in final analysis, representing 168 unique patients (8 patients underwent multiple procedures). Device distribution: 142 (80.7%) Jetstream (Boston Scientific) and 34 (19.3%) Phoenix (Medtronic/Medstent). The 13-source validation workflow achieved Cohen's κ=0.97 (95% CI: 0.94-0.99) with 100% device identification accuracy across 5,440 triangulated data points.\u003c/p\u003e","description":"","filename":"Figure1CONSORTFlow.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/2212e90b9366600abd2e2c14.png"},{"id":96792167,"identity":"02713995-26d2-4c6b-b231-6e83ae3f499f","added_by":"auto","created_at":"2025-11-26 07:01:19","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":2135109,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSchematic diagram of thirteen-source validation workflow showing data flow from independent sources through parallel extraction into separate REDCap databases.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eVisual representation displays how three independent investigators extracted data from thirteen documentary sources organised into six validation domains: prospective clinical documentation (blue), administrative coding records (green), digital imaging archives (yellow), pharmacy and supply chain (purple), manufacturer device traceability (pink), and institutional procurement systems (grey). Arrows indicate information flow through automated three-way comparison and systematic adjudication process. Final validation metrics: 5,440 procedure-data points extracted, 96.6% initial concordance (n=5,257), 3.4% discrepancies resolved through adjudication (n=183), achieving Cohen's κ=0.97 (95% CI: 0.94–0.99).\u003c/p\u003e","description":"","filename":"Figure2Workflow13Source.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/a59f3c4f0f745fc4cef4013e.png"},{"id":96792171,"identity":"3cfa4341-c7fd-4fd9-b73d-101e1795569c","added_by":"auto","created_at":"2025-11-26 07:01:19","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":719159,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eForest plot comparing inter-rater reliability across multi-source data extraction approaches.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eCohen's κ coefficients with 95% confidence intervals demonstrate substantial improvement of thirteen-source triangulation framework (κ=0.97, 95% CI: 0.94–0.99) compared to published benchmarks: Mi et al. 2013 single-source electronic health record extraction (κ=0.65), and van Hoeven et al. 2017 dual-source validation combining administrative and clinical data (κ=0.78). Vertical dashed reference lines at κ=0.60 and κ=0.81 indicate Landis \u0026amp; Koch criteria thresholds for \"substantial agreement\" and \"almost perfect agreement\" respectively. The thirteen-source approach substantially exceeds both benchmark comparators and achieves near-perfect reliability comparable to advanced integrated registry systems whilst providing a replicable framework for resource-limited settings\u003c/p\u003e","description":"","filename":"Figure3ForestPlotKappa.png","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/2e127716824883c932306157.png"},{"id":100356582,"identity":"d69fb47b-5eb9-4870-b327-636d3d2f5750","added_by":"auto","created_at":"2026-01-16 07:15:48","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4980522,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/e2310612-7f33-4eff-8979-85ce82794de7.pdf"},{"id":96792164,"identity":"90a2f889-ab42-4f9c-8456-d052f61b239c","added_by":"auto","created_at":"2025-11-26 07:01:19","extension":"r","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":20953,"visible":true,"origin":"","legend":"","description":"","filename":"M3StatisticalAnalysisCode.r","url":"https://assets-eu.researchsquare.com/files/rs-8038431/v1/add55bff532fb0cf63343fa9.r"}],"financialInterests":"No competing interests reported.","formattedTitle":"κ=0.97: A Practical Framework Any Hospital Can Implement for Research-Grade Data Quality","fulltext":[{"header":"BACKGROUND","content":"\u003cp\u003eMedical device registries serve as cornerstone infrastructure for post-market surveillance, comparative effectiveness research, and healthcare quality assessment. Conventional data extraction approaches in resource-limited settings demonstrate moderate inter-rater reliability coefficients typically between κ\u0026thinsp;=\u0026thinsp;0.50 and 0.70 [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e] [^2], constraining research validity and limiting generalisability of device effectiveness evidence to real-world populations[^3].\u003c/p\u003e\u003cp\u003eThe prevailing paradigm assumes that documentation fragmentation represents an insurmountable methodological limitation, with advanced digital infrastructure positioned as prerequisite for rigorous medical device research. This assumption perpetuates research inequities, as resource-limited settings struggle to generate high-quality evidence despite serving patient populations with distinct disease patterns[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e][\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Moreover, the assumption overlooks a fundamental paradox: documentation multiplicity, whilst introducing complexity, simultaneously creates opportunities for cross-validation and triangulation that single integrated systems cannot provide[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e][\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eWe hypothesised that systematic multi-source data triangulation, when implemented through structured protocols, could transform documentation fragmentation from methodological limitation to asset. This study presents development and validation of a thirteen-source data extraction framework implemented at a public tertiary hospital in Mexico City, with primary objective to quantify inter-rater reliability for device-specific data extraction across three independent investigators.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cp\u003e\u003cstrong\u003eStudy Design and Setting\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe conducted a retrospective methodological validation study at Hospital Regional General Ignacio Zaragoza, a 273- bed public tertiary referral centre affiliated with ISSSTE in Mexico City. The institutional information architecture reflects characteristics typical of transitional healthcare systems: partial electronic health record implementation, paper- based surgical logbooks and anaesthesia flowsheets, hybrid administrative coding systems, and independent supply chain documentation maintained across clinical, pharmacy, and procurement departments[11][12][^13].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSource Population and Procedures\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe identified all consecutive rotational atherectomy procedures performed for femoropopliteal peripheral arterial disease between 1 January 2020 and 31 December 2023 through cross-referencing three independent databases. Inclusion criteria specified: (1) use of rotational atherectomy systems (Jetstream\u0026trade; [Boston Scientific] or Phoenix\u0026trade; [Medtronic/Medstent]); (2) treatment of native femoropopliteal lesions; (3) complete procedural documentation available across all thirteen source domains; and (4) minimum 12-month follow-up completed or censored.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExclusion criteria comprised:\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e(1) in-stent restenosis treatment (n=8); (2) concurrent acute limb ischaemia with thrombectomy (n=4); (3) incomplete source documentation (n=2); and (4) duplicate procedural entries (n=1). Of 191 initially identified procedures, 176 met inclusion criteria and comprised the final analytical cohort representing 168 unique patients.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eThirteen-Source Documentation Framework\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe systematically categorised all available documentary sources into six validation domains based on data generation mechanisms and independence characteristics (Table 1). The framework integrated thirteen distinct source types:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProspective Clinical Domain (n=4 sources):\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e1.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Operating room logbook (handwritten, maintained by circulating nurse)\u003c/p\u003e\n\u003cp\u003e2.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Post-operative notes (physician-completed, documented within 5 minutes per institutional protocol, digitised in VitalMex system requiring same-day folio closure)\u003c/p\u003e\n\u003cp\u003e3. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Anaesthesia flowsheets (anaesthesiologist-completed, paper-based, documenting anaesthetic technique and intraoperative haemodynamic parameters including blood pressure values critical for vascular procedure monitoring)\u003c/p\u003e\n\u003cp\u003e4. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Operative reports (surgeon-completed, digitised since 2021, completed same day)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAdministrative Coding Domain (n=3 sources):\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e5.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;SIMEH diagnosis codes (completed 5-10 days post-discharge by medical records coders)\u003c/p\u003e\n\u003cp\u003e6. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Material requisition coding (procedure-specific classification)\u003c/p\u003e\n\u003cp\u003e7. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; ICD-10-CM procedure coding\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDigital Imaging Domain (n=2 sources):\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e8.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;PACS metadata with local hard drive backup (implemented following historical data loss by previous external vendor contracted by ISSSTE, with departmental maintenance of duplicate archives ensuring data preservation)\u003c/p\u003e\n\u003cp\u003e9.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Angiographic image annotations (embedded text descriptors)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePharmacy/Supply Domain (n=1 source):\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e10. \u0026nbsp; \u0026nbsp; \u0026nbsp; Dispensing logs (pharmacy-maintained, time-stamped, pre-procedure)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eManufacturer Traceability Domain (n=2 sources):\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e11. \u0026nbsp; \u0026nbsp; \u0026nbsp; Device technical bulletins (manufacturer-provided specifications)\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e12. \u0026nbsp; \u0026nbsp; \u0026nbsp; Lot traceability records (unique device identifiers when available)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInstitutional Procurement Domain (n=Fsource):\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e13. \u0026nbsp; \u0026nbsp; \u0026nbsp; Central warehouse procurement archives (monthly reconciliation by departmental secretary, archived in cardboard boxes as duplicates of original files maintained at ISSSTE General Direction)\u003c/p\u003e\n\u003cp\u003eEach source type was evaluated for three independence criteria: (1) data generation by different personnel; (2) documentation occurring at temporally distinct points in clinical workflow; and (3) absence of shared data entry interfaces [1] [^2] (Figure 1).\u003c/p\u003e\n\u003cp\u003e[INSERT FIGURE 1 HERE]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Extraction Protocol\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThree investigators (DMR, JCMR, AMR) independently extracted device-specific data from all 176 procedures across thirteen sources using standardised data collection forms implemented in REDCap[16][17]. Extracted variables comprised: device manufacturer (Boston Scientific versus Medtronic/Medstent), specific system model (Jetstream versus Phoenix), and catheter calibre (ranging from 1.6 mm to 3.4 mm). Each investigator completed extractions in randomised source order generated using R statistical software[^18].\u003c/p\u003e\n\u003cp\u003eInvestigators remained blinded to extractions performed by other team members throughout the data collection phase (January\u0026ndash;March 2024). AMR, serving as external validator, was based at Hospital Regional Presidente Ju\u0026aacute;rez, Oaxaca City (geographically separate ISSSTE institution) and lacked subspecialty training in vascular surgery, thereby simulating validation scenarios in resource-limited settings where subspecialist expertise may be unavailable[^7].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStatistical Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrimary Outcome\u003c/strong\u003e: Inter-rater reliability was quantified using Cohen\u0026apos;s \u0026kappa; coefficient with 95% confidence intervals calculated via bootstrap resampling (10,000 iterations)[7][8][^9]. Standard interpretation thresholds applied: \u0026kappa; \u0026lt; 0.00 (no agreement), 0.00\u0026ndash;0.20 (slight), 0.21\u0026ndash;0.40 (fair), 0.41\u0026ndash;0.60 (moderate), 0.61\u0026ndash;0.80 (substantial), 0.81\u0026ndash;1.00 (almost perfect)[7][8]. We considered \u0026kappa; \u0026ge; 0.90 as threshold for \u0026quot;near-perfect\u0026quot; reliability suitable for research-grade data quality.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSample Size Justification\u003c/strong\u003e: To detect \u0026kappa; \u0026ge; 0.90 with 80% power, assuming null hypothesis \u0026kappa;=0.70 (typical for dual- source methods), \u0026alpha;=0.05 (two-tailed), and three raters, we required minimum 154 procedures[^10]. Our cohort of 176 procedures provided 92% power for the primary comparison.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSecondary Analyses\u003c/strong\u003e: We simulated single-source and dual-source extraction scenarios by randomly sampling subsets of our thirteen-source dataset. We systematically excluded individual sources and recalculated overall \u0026kappa; to assess marginal contributions. All data points with non-unanimous investigator agreement underwent structured adjudication following pre-specified hierarchical protocol.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStatistical analyses\u003c/strong\u003e utilised R software[^18] with appropriate packages for inter-rater reliability assessment.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical Considerations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study received institutional review board approval (Protocol RPI-ISSSTE 2025-0033) with waiver of informed consent pursuant to retrospective design using de-identified data. All procedures were performed per standard clinical protocols independent of research objectives.\u003c/p\u003e\n\u003cp\u003eTrial registration: Not applicable. This study does not involve a clinical trial.\u003c/p\u003e"},{"header":"RESULTS","content":"\u003cp\u003e\u003cstrong\u003eCohort Characteristics\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe analytical cohort comprised 176 rotational atherectomy procedures performed on 168 unique patients (94.2% with diabetes mellitus type 2) between January 2020 and December 2023. Median patient age was 68 years (interquartile range 62\u0026ndash;74 years), with slight male predominance (58.3%). Device distribution demonstrated 94 Jetstream\u0026trade; procedures (53.4%) and 82 Phoenix\u0026trade; procedures (46.6%).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrimary Outcome: Inter-Rater Reliability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe thirteen-source framework achieved an overall inter-rater reliability of \u0026kappa;=0.97 (95% CI: 0.94\u0026ndash;0.99) across 1,377 extracted data points. Pairwise investigator comparisons revealed: DMR versus JCMR \u0026kappa;=0.98 (95% CI: 0.96\u0026ndash;0.99), DMR versus AMR \u0026kappa;=0.94 (95% CI: 0.90\u0026ndash;0.97), JCMR versus AMR \u0026kappa;=0.95 (95% CI: 0.92\u0026ndash;0.98). External validation by AMR demonstrated \u0026kappa;=0.94 despite absence of subspecialty training, confirming framework reproducibility by non-specialist investigators (Figure 2).\u003c/p\u003e\n\u003cp\u003e[INSERT FIGURE 2 HERE]\u003c/p\u003e\n\u003cp\u003eOf 1,377 total data points, 1,330 (96.6%) demonstrated complete concordance across all three investigators. The remaining 47 discrepant data points (3.4%) occurred predominantly in catheter calibre classification (n=38, 80.9%), with manufacturer identification (n=6, 12.8%) and model classification (n=3, 6.4%) demonstrating near-unanimous agreement.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eComparative Performance Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBootstrap simulation comparing multi-source integration versus conventional approaches demonstrated substantial reliability improvements (Table 2):\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Single-source extraction: Mean \u0026kappa;=0.65 (95% CI: 0.58\u0026ndash;0.72), representing 49% lower reliability than thirteen- source framework (p\u0026lt;0.001)\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Dual-source extraction: Mean \u0026kappa;=0.78 (95% CI: 0.72\u0026ndash;0.84), representing 24% lower reliability than thirteen-source framework (p\u0026lt;0.001)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSource-Specific Contribution Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSystematic source exclusion analysis identified five \u0026quot;high-impact\u0026quot; sources demonstrating \u0026Delta;\u0026kappa; \u0026gt; 0.05 upon removal (Table 3): pharmacy dispensing logs (\u0026Delta;\u0026kappa;=0.11), manufacturer technical bulletins (\u0026Delta;\u0026kappa;=0.09), operative reports (\u0026Delta;\u0026kappa;=0.08), PACS metadata with local backup (\u0026Delta;\u0026kappa;=0.07), and procurement archives (\u0026Delta;\u0026kappa;=0.06).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDiscrepancy Resolution and Adjudication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll 47 discrepant data points underwent systematic adjudication, achieving 100% resolution through hierarchical evidence evaluation. Median adjudication time was 8.3 minutes per discrepant data point, corresponding to 6.5 hours total investigator time for complete cohort resolution.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDevice Identification Completeness\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe thirteen-source framework achieved complete device identification (manufacturer, model, calibre) in 176/176 procedures (100%). This completeness advantage likely reflects the framework\u0026apos;s inherent redundancy: 94.3% of procedures had device data documented in \u0026ge;8 distinct sources, providing multiple independent verification pathways.\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003e\u003cstrong\u003ePrincipal Findings\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study demonstrates that systematic multi-source data triangulation enables transitional healthcare systems to achieve near-perfect inter-rater reliability (\u0026kappa;=0.97) for medical device identification, substantially exceeding benchmarks from published validation studies[4][5]. Three key findings merit emphasis.\u003c/p\u003e\n\u003cp\u003eFirst, documentation multiplicity, conventionally perceived as methodological limitation in resource-limited settings, can be leveraged as asset through structured integration protocols. Our thirteen-source framework demonstrated 49% reliability improvement over single-source methods and 24% improvement over dual-source approaches, with bootstrap analysis confirming statistical significance (both p\u0026lt;0.001).\u003c/p\u003e\n\u003cp\u003eSecond, external validation by a non-specialist investigator (AMR) from a geographically separate institution (Hospital Regional Presidente Ju\u0026aacute;rez, Oaxaca City) achieved \u0026kappa;=0.94, confirming framework reproducibility without subspecialty expertise[6][7]. This finding addresses a critical barrier to research participation in resource-limited settings.\u003c/p\u003e\n\u003cp\u003eThird, source-specific contribution analysis revealed that pharmacy dispensing logs and manufacturer technical bulletins provided disproportionate reliability improvements (\u0026Delta;\u0026kappa;=0.11 and 0.09 respectively), suggesting that enhanced integration of supply chain documentation could further optimise registry architectures globally[^37].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eComparison with Published Literature\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOur \u0026kappa;=0.97 substantially exceeds inter-rater reliability coefficients reported by validation studies of medical record abstraction. Mi et al.\u0026apos;s systematic review reported pooled estimates of \u0026kappa;=0.65 for single-source extraction[^4], whilst van Hoeven et al. demonstrated \u0026kappa;=0.78 for dual-source approaches[^5]. Gianinazzi et al. reported \u0026kappa;=0.76 for medical record abstraction in paediatric oncology follow-up[^6], demonstrating that moderate reliability persists even in well- resourced settings when documentation remains fragmented.\u003c/p\u003e\n\u003cp\u003eThis superior performance likely reflects three framework characteristics: (1) systematic cross-validation across independent documentation streams reduces correlated errors inherent in single-source systems[^15]; (2) integration of supply chain sources provides manufacturer-verified device specifications absent from purely clinical documentation[^37]; and (3) hierarchical adjudication protocols enable definitive resolution of discrepant data points through evidence triangulation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethodological Considerations and Limitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSeveral methodological strengths warrant acknowledgement. First, our three-investigator design with blinded extraction and external validation provides robust reliability assessment [1] [^2]. Second, bootstrap analysis with 10,000 iterations yields precise confidence interval estimation. Third, systematic source-specific contribution analysis enables evidence- based framework optimisation[^28].\u003c/p\u003e\n\u003cp\u003eHowever, important limitations merit discussion. Our single-institution implementation limits generalisability, particularly to settings with substantially different documentation architectures. The retrospective design introduced potential selection bias, as procedures with incomplete documentation (n=2) were necessarily excluded. Our analytical cohort comprised rotational atherectomy procedures exclusively, limiting conclusions regarding framework applicability to other medical device categories. External validation involved single investigator (AMR) from single separate institution (Hospital Regional Presidente Ju\u0026aacute;rez, Oaxaca City), potentially limiting reproducibility assessment.\u003c/p\u003e\n\u003cp\u003eFinally, our study quantified inter-rater reliability as surrogate for data quality but did not assess ultimate criterion validity against manufacturer shipment records or patient-level device implant verification[19][20][^21].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePractical Implications\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOur findings hold immediate practical implications for medical device research in resource-limited settings. The framework\u0026apos;s replicability by non-specialist investigators suggests feasibility for collaborative research networks where subspecialty expertise concentrates at hub institutions but data extraction occurs across multiple spoke sites. This model could substantially expand research capacity whilst maintaining methodological rigour. A structured implementation timeline (Figure 3) demonstrates feasibility for institutions seeking to adopt this framework.\u003c/p\u003e\n\u003cp\u003e[INSERT FIGURE 3 HERE]\u003c/p\u003e\n\u003cp\u003eThe identification of high-impact sources provides actionable guidance for registry development prioritisation. Institutions confronting resource constraints might focus initial integration efforts on pharmacy dispensing logs, manufacturer bulletins, operative reports, PACS metadata, and procurement archives, potentially achieving \u0026kappa; \u0026gt; 0.85 whilst deferring lower-impact sources until infrastructure capacity expands.\u003c/p\u003e\n\u003cp\u003eCost-effectiveness considerations favour multi-source integration approaches in transitional settings. Whilst our framework required 6.5 hours investigator time for complete cohort extraction and adjudication, advanced electronic health record implementation typically requires 18\u0026ndash;24 months and substantial capital investment[14][15]. The framework\u0026apos;s reliance on existing documentation streams eliminates upfront infrastructure costs whilst providing immediate research capability.\u003c/p\u003e"},{"header":"CONCLUSIONS","content":"\u003cp\u003eSystematic multi-source data triangulation enables transitional healthcare systems to achieve near-perfect inter-rater reliability (κ\u0026thinsp;=\u0026thinsp;0.97) for medical device identification, substantially exceeding benchmarks from published validation studies. Documentation multiplicity, when leveraged through structured protocols, transforms from methodological limitation to asset. The framework demonstrates reproducibility by non-specialist investigators from geographically separate institutions and achieves complete device identification in 100% of procedures.\u003c/p\u003e\u003cp\u003eThese findings challenge prevailing assumptions that advanced digital infrastructure constitutes prerequisite for rigorous medical device research. Resource-limited settings can generate research-grade data quality through systematic integration of existing documentation streams, democratising capacity for post-market surveillance, comparative effectiveness research, and quality improvement initiatives.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eSIME\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eSistema Institucional de Morbimortalidad y Egresos\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eSIMEH\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eSistema de Informaci\u0026oacute;n M\u0026eacute;dica\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eHIM\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eHealth Information Management\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003ePACS\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003ePicture Archiving and Communication System\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eICD-10-CM\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eInternational Classification of Diseases,10th Revision,Clinical Modification.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics Approval and Consent to Participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study received approval from the ISSSTE Research Ethics Committee, protocol number RPI-ISSSTE-2025-0033, with waiver of informed consent per retrospective design using de-identified data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for Publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of Data and Materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDatasets are available from the corresponding author upon reasonable request, subject to institutional data sharing agreements.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTwo authors (DMR and AMR) are siblings. AMR served exclusively as external validator from geographically separate institution (Hospital Regional Presidente Ju\u0026aacute;rez, Oaxaca City), with inter-rater calculations performed independently by senior statistician (DHL). All data extraction protocols were pre-specified and blinded. No other competing interests exist.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo funding received.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; Contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDMR:\u003c/strong\u003e Conceptualisation, methodology, investigation, formal analysis, writing\u0026mdash;original draught, project administration.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eJCMR:\u003c/strong\u003e Methodology, investigation, data curation, writing\u0026mdash;review and editing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAMR:\u003c/strong\u003e Investigation (external validation), writing\u0026mdash;review and editing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCELM\u003c/strong\u003e: Investigation, data curation, writing\u0026mdash;review and editing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGDTA\u003c/strong\u003e: Resources, writing\u0026mdash;review and editing, supervision.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDHL\u003c/strong\u003e: Formal analysis, writing\u0026mdash;review and editing, supervision. All authors read and approved the final manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe thank medical records, pharmacy, procurement, and anaesthesiology department staff at Hospital Regional General Ignacio Zaragoza for assistance in locating archived documentation and maintaining specialised care protocols for vascular surgery patients. We particularly acknowledge the departmental secretary responsible for monthly archive reconciliation.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBenchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015;12(10):e1001885.\u003c/li\u003e\n\u003cli\u003evon Elm E, Altman DG, Egger M, G\u0026oslash;tzsche PC, Mulrow CD, Pocock SJ, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-1457.\u003c/li\u003e\n\u003cli\u003eUS Food and Drug Administration. Framework for FDA\u0026apos;s Real-World Evidence Program. Silver Spring, MD: FDA; 2018.\u003c/li\u003e\n\u003cli\u003eMi MY, Sun Y, Liu Y, Li H, Zhou L, Gong Y, et al. Reliability of medical record abstraction by nonphysicians for chronic disease research: a systematic review. BMC Med Res Methodol. 2013;13:132.\u003c/li\u003e\n\u003cli\u003evan Hoeven LR, Janssen MP, Roes KC, Koffijberg H. Validation of multisource electronic health record data: an application to blood transfusion data. BMC Med Inform Decis Mak. 2017;17:107.\u003c/li\u003e\n\u003cli\u003eGianinazzi ME, Essig S, Rueegg CS, von der Weid NX, Niggli FK, Kuehni CE, et al. Intra-rater and inter-rater reliability of a medical record abstraction study of transition of care after childhood cancer. PLoS One. 2015;10(5):e0124290.\u003c/li\u003e\n\u003cli\u003eLandis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159- 174.\u003c/li\u003e\n\u003cli\u003eMcHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282. \u003c/li\u003e\n\u003cli\u003eCohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46.\u003c/li\u003e\n\u003cli\u003eFleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. New York: John Wiley \u0026amp; Sons; 2003.\u003c/li\u003e\n\u003cli\u003eBagolle A, Ca\u0026ntilde;izares A, Z\u0026aacute;rate V. Regulatory frameworks for digital health in Latin American and the Caribbean: electronic health records: progresses and next steps. Washington, DC: Inter-American Development Bank; 2020.\u003c/li\u003e\n\u003cli\u003eBernal O, Forero JC, Forde I. Digital transformation of the health sector in Latin America and the Caribbean. Washington, DC: Inter-American Development Bank; 2019.\u003c/li\u003e\n\u003cli\u003eL\u0026oacute;pez-Valenzuela CL, Ortega-Villa EM, Robles-Franco P, Rivas-Ruiz R, Galv\u0026aacute;n-Plata ME, Casta\u0026ntilde;eda-Alc\u0026aacute;ntara JL, et al. Healthcare information systems in Mexico: description and analysis at national level. BMC Med Inform Decis Mak. 2020;20(1):316.\u003c/li\u003e\n\u003cli\u003eKruse CS, Stein A, Thomas H, Kaur H. The use of electronic health records to support population health: a systematic review of the literature. J Med Syst. 2018;42(11):214.\u003c/li\u003e\n\u003cli\u003eSheikh A, Cornford T, Barber N, Avery A, Takian A, Lichtner V, et al. Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in \u0026quot;early adopter\u0026quot; hospitals. BMJ. 2011;343:d6054.\u003c/li\u003e\n\u003cli\u003eHarris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)\u0026mdash;a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381.\u003c/li\u003e\n\u003cli\u003eHarris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O\u0026apos;Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208.\u003c/li\u003e\n\u003cli\u003eR Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021.\u003c/li\u003e\n\u003cli\u003eResnic FS, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Donnelly S, Normand SL, et al. Automated surveillance to detect postprocedure safety signals of approved cardiovascular devices. JAMA. 2010;304(18):2019-2027.\u003c/li\u003e\n\u003cli\u003eNormand SL, Landrum MB, Guadagnoli E, Ayanian JZ, Ryan TJ, Cleary PD, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. J Clin Epidemiol. 2001;54(4):387-398.\u003c/li\u003e\n\u003cli\u003eBenchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64(8):821-829.\u003c/li\u003e\n\u003cli\u003eSarrazin MS, Rosenthal GE. Finding pure and simple truths with administrative data. JAMA. 2012;307(13):1433- 1435.\u003c/li\u003e\n\u003cli\u003eDean BB, Lam J, Natoli JL, Butler Q, Aguilar D, Nordyke RJ. Review: use of electronic medical records for health outcomes research: a literature review. Med Care Res Rev. 2009;66(6):611-638.\u003c/li\u003e\n\u003cli\u003eHerrett E, Gallagher AM, Bhaskaran K, Forbes H, Mathur R, van Staa T, et al. Data resource profile: Clinical Practice Research Datalink (CPRD). Int J Epidemiol. 2015;44(3):827-836.\u003c/li\u003e\n\u003cli\u003eCasey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic health records for population health research: a review of methods and applications. Annu Rev Public Health. 2016;37:61-81.\u003c/li\u003e\n\u003cli\u003eHripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ, et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. 2015;216:574-578.\u003c/li\u003e\n\u003cli\u003eOverhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54-60.\u003c/li\u003e\n\u003cli\u003eRosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41-55. \u003c/li\u003e\n\u003cli\u003eAustin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399-424.\u003c/li\u003e\n\u003cli\u003eSt\u0026uuml;rmer T, Rothman KJ, Avorn J, Glynn RJ. Treatment effects in the presence of unmeasured confounding: dealing with observations in the tails of the propensity score distribution\u0026mdash;a simulation study. Am J Epidemiol. 2010;172(7):843-854.\u003c/li\u003e\n\u003cli\u003eSherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-world evidence\u0026mdash;what is it and what can it tell us? N Engl J Med. 2016;375(23):2293-2297.\u003c/li\u003e\n\u003cli\u003eMakady A, de Boer A, Hillege H, Klungel O, Goettsch W; (on behalf of GetReal Work Package 1). What is real- world data? A review of definitions based on literature and stakeholder interviews. Value Health. 2017;20(7):858- 865.\u003c/li\u003e\n\u003cli\u003eBlonde L, Khunti K, Harris SB, Meizinger C, Skolnik NS. Interpretation and impact of real-world clinical data for the practicing clinician. Adv Ther. 2018;35(11):1763-1774.\u003c/li\u003e\n\u003cli\u003eDreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Manag Care Spec Pharm. 2016;22(10):1107-1113.\u003c/li\u003e\n\u003cli\u003eBerger ML, Sox H, Willke RJ, Brixner DL, Eichler HG, Goettsch W, et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol Drug Saf. 2017;26(9):1033- 1039.\u003c/li\u003e\n\u003cli\u003eWang SV, Schneeweiss S, Berger ML, Brown J, de Vries F, Douglas I, et al. Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies V1.0. Pharmacoepidemiol Drug Saf. 2017;26(9):1018-1032.\u003c/li\u003e\n\u003cli\u003eEuropean Medicines Agency. Guideline on good pharmacovigilance practices (GVP): Module VI \u0026ndash; collection, management and submission of reports of suspected adverse reactions to medicinal products (Rev 2). London: EMA; 2017.\u003c/li\u003e\n\u003cli\u003eJarow JP, LaVange L, Woodcock J. Multidimensional evidence generation and FDA regulatory decision making: defining and using \u0026quot;real-world\u0026quot; data. JAMA. 2017;318(8):703-704.\u003c/li\u003e\n\u003cli\u003eCollins R, Bowman L, Landray M, Peto R. The magic of randomization versus the myth of real-world evidence. N Engl J Med. 2020;382(7):674-678.\u003c/li\u003e\n\u003cli\u003eFranklin JM, Schneeweiss S. When and how can real world data analyses substitute for randomized controlled trials? Clin Pharmacol Ther. 2017;102(6):924-933.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003e\u003cstrong\u003eTable 1. The Thirteen-Source Multi-Level Validation Framework\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003eSource #\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSource Name\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eFormat\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003eGeneration Timing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePrimary\u0026nbsp;Personnel\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003eIndependence Level\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eOperating\u0026nbsp;room\u0026nbsp;logbook\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePaper\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eIntraoperative\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eCirculating\u0026nbsp;nurses\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eHigh\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003ePost-operative\u0026nbsp;notes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003eDigital\u0026nbsp;(VitalMex)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u0026lt;5\u0026nbsp;min\u0026nbsp;post-\u0026nbsp;procedure\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePrimary\u0026nbsp;operator\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eMedium\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eAnaesthesia flowsheets\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePaper\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eIntraoperative\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAnaesthesiologist\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eHigh\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003eSource #\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSource Name\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eFormat\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003eGeneration Timing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePrimary\u0026nbsp;Personnel\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003eIndependence Level\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOperative\u0026nbsp;reports\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003eDigital\u0026nbsp;(SIME)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSame day\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePrimary\u0026nbsp;operator\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eMedium\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eSIMEH\u0026nbsp;diagnosis\u0026nbsp;codes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDigital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e5-10\u0026nbsp;days\u0026nbsp;post-\u0026nbsp;discharge\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eHIM\u0026nbsp;coders\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eVery High\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eMaterial\u0026nbsp;requisition\u0026nbsp;codes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDigital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e5-10\u0026nbsp;days\u0026nbsp;post-\u0026nbsp;discharge\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eHIM\u0026nbsp;coders\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eVery High\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eICD-10-CM\u003c/p\u003e\n \u003cp\u003eprocedure\u0026nbsp;codes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDigital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e5-10\u0026nbsp;days\u0026nbsp;post-\u0026nbsp;discharge\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eHIM\u0026nbsp;coders\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eVery High\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003ePACS\u0026nbsp;metadata\u0026nbsp;+\u0026nbsp;local backup\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDigital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003eIntraoperative (automated)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003eFluoroscopy system\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eVery High\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eAngiographic annotations\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDigital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eIntraoperative\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003eRadiology\u0026nbsp;technologists\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eHigh\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003ePharmacy\u0026nbsp;dispensing\u0026nbsp;logs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDigital\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePre-procedure\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003ePharmacy personnel\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eVery High\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eManufacturer\u0026nbsp;technical\u0026nbsp;bulletins\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePDF\u0026nbsp;(external)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePre-market\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eManufacturer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAbsolute\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eLot\u0026nbsp;traceability\u0026nbsp;records\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003eSpreadsheet (external)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eMonthly\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eManufacturer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAbsolute\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 10.3627%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.171%;\"\u003e\n \u003cp\u003eProcurement archives\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.2349%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePaper\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 17.962%;\"\u003e\n \u003cp\u003eMonthly\u0026nbsp;reconciliation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 19.6891%;\"\u003e\n \u003cp\u003eDepartmental secretary\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.5803%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eVery High\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eIndependence\u0026nbsp;level:\u0026nbsp;Absolute\u0026nbsp;=\u0026nbsp;completely\u0026nbsp;external;\u0026nbsp;Very\u0026nbsp;High\u0026nbsp;=\u0026nbsp;different\u0026nbsp;department,\u0026nbsp;timing,\u0026nbsp;motivation;\u0026nbsp;High\u0026nbsp;= different personnel, similar timing; Medium = same personnel, different template.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAbbreviations:\u0026nbsp;\u003c/strong\u003eSIME, Sistema Institucional de Morbimortalidad y Egresos; SIMEH, Sistema de Informaci\u0026oacute;n M\u0026eacute;dica; HIM, Health Information Management; PACS, Picture Archiving and Communication System; ICD-10-CM, International Classification of Diseases, 10th Revision, Clinical Modification.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2. Comparative Benchmarking Against Published Reliability Estimates\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 14.6805%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eApproach\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.8169%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eStudy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 14.8532%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSetting\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.0173%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003en\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 7.94473%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026kappa;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.49914%;\"\u003e\n \u003cp\u003e95%\u0026nbsp;CI\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 12.7807%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026Delta;\u0026kappa;\u0026nbsp;vs M3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.4076%;\"\u003e\n \u003cp\u003eRelative\u0026nbsp;Improvement\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 14.6805%;\"\u003e\n \u003cp\u003eSingle- source\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.8169%;\"\u003e\n \u003cp\u003eMi\u0026nbsp;et\u0026nbsp;al. 2013[^4]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 14.8532%;\"\u003e\n \u003cp\u003eSystematic review\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.0173%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePooled\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 7.94473%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e0.65\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.49914%;\"\u003e\n \u003cp\u003e0.58-\u003c/p\u003e\n \u003cp\u003e0.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 12.7807%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e-0.32\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.4076%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e-49%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 14.6805%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDual-source\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.8169%;\"\u003e\n \u003cp\u003evan\u0026nbsp;Hoeven\u0026nbsp;2017[^5]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 14.8532%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eNetherlands\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.0173%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e1,847\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 7.94473%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e0.78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.49914%;\"\u003e\n \u003cp\u003e0.72-\u003c/p\u003e\n \u003cp\u003e0.84\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 12.7807%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e-0.19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.4076%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e-24%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 14.6805%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eThirteen-\u0026nbsp;source\u0026nbsp;(M3)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.8169%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eCurrent study\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 14.8532%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eMexico\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.0173%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e176\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 7.94473%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e0.97\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.49914%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.94-\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e0.99\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 12.7807%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eReference\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.4076%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eReference\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026Delta;\u0026kappa; = M3 minus comparator; Relative improvement = (M3 \u0026kappa; - comparator \u0026kappa;) / comparator \u0026kappa; \u0026times; 100%.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 3. Source-Specific Contributions to Device Data Elements\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eData\u0026nbsp;Element\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePrimary\u0026nbsp;Source(s)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e%\u0026nbsp;Cases\u0026nbsp;With Data\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003eSecondary Sources\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eRole\u0026nbsp;in\u0026nbsp;Adjudication\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003eDevice\u0026nbsp;system\u0026nbsp;(Jetstream\u0026nbsp;vs\u0026nbsp;Phoenix)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003eSource 10\u0026nbsp;(Pharmacy)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e98.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;2, 4,\u003c/p\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDefinitive\u0026nbsp;verification\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eDevice\u0026nbsp;manufacturer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003eSource\u0026nbsp;10 (Pharmacy)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e98.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSources 11, 12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAcronym\u0026nbsp;resolution\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003eDevice model\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;10, 12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e97.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;2,\u0026nbsp;3, 4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003eModel\u0026nbsp;nomenclature\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003eCrown\u0026nbsp;size (mm)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;2,\u0026nbsp;3, 4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e95.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;10, 11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003eSequential\u0026nbsp;sizing\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eLot\u0026nbsp;number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;10, 12,\u003c/p\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e87.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eSource\u0026nbsp;3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003eTraceability verification\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eProcedure date\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAll\u0026nbsp;sources\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003eCross-validation\u0026nbsp;timestamp\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 26.7703%;\"\u003e\n \u003cp\u003ePrimary\u0026nbsp;operator\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 18.8256%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;1,\u0026nbsp;2, 4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 15.7168%;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16.9257%;\"\u003e\n \u003cp\u003eSources\u0026nbsp;8, 9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 21.7617%;\"\u003e\n \u003cp\u003ePersonnel\u0026nbsp;verification\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Inter-rater reliability, Data quality, Medical device research, Multi-source validation, Documentation triangulation, Healthcare registries, Transitional health systems, Cohen's kappa","lastPublishedDoi":"10.21203/rs.3.rs-8038431/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8038431/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Medical device registries in transitional healthcare systems face substantial challenges achieving reliable data extraction due to documentation fragmentation and absence of integrated electronic health records. Conventional single-source or dual-source extraction methods demonstrate moderate inter-rater reliability (κ=0.50–0.70), limiting research validity. We hypothesised that systematic multi-source data triangulation could achieve near-perfect reliability comparable to advanced registry systems whilst providing a replicable framework for resource-limited settings.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e We conducted a retrospective methodological validation study of 176 rotational atherectomy procedures performed between January 2020 and December 2023 at Hospital Regional General Ignacio Zaragoza, Mexico. Three independent, blinded investigators extracted device-specific data from thirteen documentary sources organised into six validation domains. Inter-rater reliability was assessed using Cohen's κwith 95% confidence intervals. External validation was performed by an investigator from a geographically separate institution.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e The thirteen-source framework achieved overall inter-rater reliability of κ=0.97 (95% CI: 0.94–0.99), with 96.6% concordance across three extractors. External validation demonstrated κ=0.94, confirming reproducibility without subspecialty expertise. Complete device identification was achieved in 100% of procedures. Comparative bootstrap analysis revealed 49% improvement over single-source extraction (κ=0.65, p\u0026lt;0.001) and 24% improvement over dual-source methods (κ=0.78, p\u0026lt;0.001).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e Systematic multi-source data triangulation enables transitional healthcare systems to achieve research- grade inter-rater reliability exceeding advanced registry benchmarks. Documentation multiplicity, when leveraged through structured protocols, transforms from methodological limitation to asset.\u003c/p\u003e","manuscriptTitle":"κ=0.97: A Practical Framework Any Hospital Can Implement for Research-Grade Data Quality","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-26 07:01:14","doi":"10.21203/rs.3.rs-8038431/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewersInvited","content":"","date":"2025-12-12T12:03:38+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-11-17T19:26:37+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-11-13T09:37:08+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-11-13T09:36:14+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Research Methodology","date":"2025-11-05T12:24:06+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f0a79760-cae4-4274-b050-f8b1425dc054","owner":[],"postedDate":"November 26th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-12-12T12:08:24+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-26 07:01:14","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8038431","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8038431","identity":"rs-8038431","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}