{"paper_id":"a35cf8ec-6667-4198-a34f-de947dd8b970","body_text":"Augmenting the Clinical Data Sources for Enigmatic\nDiseases: A Cross-Sectional Study of Self-Tracking Data\nand Clinical Documentation in Endometriosis\nIpek Ensari 1 Adrienne Pichon 2 Sharon Lipsky-Gorman 2 Suzanne Bakken 2,3 Noémie Elhadad 2\n1 Data Science Institute, Columbia University, New York, New York,\nUnited States\n2 Department of Biomedical Informatics, Columbia University Irving\nMedical Center, New York, New York, United States\n3 Columbia School of Nursing, Columbia University, New York,\nNew York, United States\nAppl Clin Inform 2020;11:769 –784.\nAddress for correspondence I p e kE n s a r i ,P h D ,C o l u m b i aU n i v e r s i t y ,\nData Science Institute, 475 Riverside Dr, Room 320, New York,\nNY 10115, United States (e-mail: ie2145@columbia.edu).\nKeywords\n► mobile platforms\n► quantified self\n► chronic illness or\nspecial needs\n► electronic health\nrecords and systems\n► personal health\nrecords\n► smart phone\nAbstract Background Self-tracking through mobile health technology can augment the\nelectronic health record (EHR) as an additional data source by providing direct patient\ninput. This can be particularly useful in the context of enigmatic diseases and further\npromote patient engagement.\nObjectives This study aimed to investigate the additional information that can be\ngained through direct patient input on poorly understood diseases, beyond what is\nalready documented in the EHR.\nMethods This was an observational study including two samples with a clinically\nconfirmed endometriosis diagnosis. We analyzed data from 6,925 women with\nendometriosis using a research app for tracking endometriosis to assess prevalence\nof self-reported pain problems, between- a nd within-person variability in pain over\ntime, endometriosis-affected tasks of daily function, and self-management strategies.\nWe analyzed data from 4,389 patients identi fied through a large metropolitan hospital\nEHR to compare pain problems with the self-tracking app and to identify unique data\nelements that can be contributed via patient self-tracking.\nResults Pelvic pain was the most prevalent problem in the self-tracking sample\n(57.3%), followed by gastrointestinal-r elated (55.9%) and lower back (49.2%) pain.\nUnique problems that were captured by self-tracking included pain in ovaries (43.7%)\nand uterus (37.2%). Pain ex perience was highly variable both across and within\nparticipants over time. Within-person vari ation accounted for 58% of the total variance\nin pain scores, and was large in magnitude, based on the ratio of within- to between-\nperson variability (0.92) and the intraclass correlation (0.42). Work was the most\naffected daily function task (49%), and there was signi ficant within- and between-\nperson variability in self-management effectiveness. Prevalence rates in the EHR were\nsignificantly lower, with abdominal pain being the most prevalent (36.5%).\nConclusion For enigmatic diseases, patient self-tracking as an additional data source\ncomplementary to EHR can enable learning from the patient to more accurately and\ncomprehensively evaluate patient health history and status.\nreceived\nMay 17, 2020\naccepted\nJuly 14, 2020\nDOI https://doi.org/\n10.1055/s-0040-1718755.\nISSN 1869-0327.\n© 2020 Georg Thieme Verlag KG\nStuttgart · New York\nTHIEME\nResearch Article 769\nArticle published online: 2020-11-18\n\nBackground and Signi ficance\nThe Learning Health System (LHS) framework proposes an\nimproved health care system to accelerate the lengthy ( /C24 17\nyears) learning health cycle from health care data to research\nfindings that inform clinical decisions by learning and shar-\ning information from patients, care delivery systems, and\nclinical research over time.\n1,2 This framework relies on the\nelectronic health record (EHR) as a primary source for gain-\ning information about patients and generating evidence to\ninform care,3 and aims to make the research-to-clinical care\npipeline more ef ficient4 through use of big data, health\ntechnologies, and participatory design. 5,6\nThe EHR constitutes a rich, convenient source of data for\nclinical decision-making, 7 conducting biomedical re-\nsearch,8,9 and generating relevant and actionable knowledge\nthat can inform the tailoring of treatments. 10 The problem\nlists within the EHR are particularly useful as they provide a\nstructured, comprehensive, and accessible list of patient\nproblems.11 Problem lists include diagnoses, illnesses, inju-\nries, and other factors that affect the health of the patient,\ndocumented by the provider at the point of care and stored in\nstandard encoding systems. 12 They facilitate data sharing,\ninformation retrieval, and communication throughout the\nhealth care continuum, and further provide data source for\nresearch studies and other secondary data uses. 12\nYet, sole reliance on the EHR and data collected at point of\ncare can be insuf ficient,13 particularly when studying condi-\ntions that lack adequate documentation, 14 conditions that\nhave a fluctuating course over time, and in instances where\nthe clinic assessment of an outcome might differ from its\nambulatory assessment.15 Health providers may not log diag-\nnosis codes for observed conditions that do not require treat-\nment, diagnostic testing, or referral.8 Previous studies further\nreport incompleteness and signi ficant variability in provider\nagreement on problem lists,16 and significant burnout due to\nEHR use among providers17–19 resulting in less time-treating\npatients, incomplete chart notes, and duplicate records. 17\nThese factors can collectively contribute to de ficiencies in\nEHR data accuracy, quality, and level of detail. 20 This can\nsubsequently pose challenges for not only clinical care, 21 but\nalso research including disease case identi fication14,22 and\nlongitudinal analyses.20 In the context of enigmatic diseases,\nthese de ficiencies can further challenge attempts to under-\nstand and describe (i.e., “characterize”) the disease, derive\naccurate and comprehensive evaluations of patient ’ s disease\nstatus and subsequently target their needs.\nOne approach to circumvent these limitations is to aug-\nment the EHR data with direct patient input obtained\nthrough self-tracking.\n23–25 Self-tracking is de fined as the\npractice of repeatedly collecting data and re flecting on it\nto acquire knowledge or achieve a goal such as behavior\nchange.\n26–29 Since these data capture patients ’ day-to-day\nexperiences outside of formal clinical settings, 28,30 they can\naid in differential diagnosis 31 and monitoring of disease by\nproviding additional information not obtained from the EHR.\nIn addition to contributing to better patient care and shared\nclinical decision-making, 24,32–34 including the patient in the\ndata generation and sharing process further circumvents\nplacing additional documentation burden on the provider\nat point of care. As such, this approach could be an attractive\nalternative to efforts to increase provider EHR documenta-\ntion responsibility.\n35 The potential impact of self-tracked\nhealth data through mHealth technology for informing evi-\ndence-based care is acknowledged by many, 36,37 including\nthe World Health Organization (WHO) stating that it can help\naddress the growing burden of chronic disease.\n38 These\nfactors collectively provide impetus for investigating the\ndata elements that can be elicited from mobile patient\ntracking data for describing enigmatic diseases.\nOne such enigmatic condition is endometriosis: a systemic,\nestrogen-dependent inflammatory condition with a range of\ndebilitating symptoms,\n39 comorbidities,40 and associated with\nsignificant impact on daily function and quality of life\n(QoL).41,42 Chronic pelvic pain is its most frequent symptom43;\nothers include pain with sexual intercourse (dyspareunia),\npainful urination (dysuria), and ovulation pain. 44,45 Endome-\ntriosis is associated with a productivity loss of 6.3 hours/week46\nand an estimated $69.4 billion per year in excess health\nexpenditures in the United States,42 and is the second leading\nindication for hysterectomy.47 Despite its prevalence rate of 10%\namong women of reproductive age, 44 endometriosis is an\nunderfunded disease48 that is still considered “clinically con-\nfusing” and “surprising,”44,49 with no cure or adequate treat-\nment to date. There further is substantial between-patient\nvariation in its clinical manifestations, 44,49 which likely con-\ntributes to the approximately 6.7 ( /C6 6.3) year delay between\nsymptom-onset and its surgical diagnosis,50 and the difficulty\nof understanding its heterogeneous pathophysiology and iden-\ntification of possible subtypes (i.e.,“phenotypes”)o ft h ed i s e a s e .\nAs a disease demographic, endometriosis patients further\nface inequities with respect to access to clinical care, 40,51\nincluding inadequate insurance coverage of endometriosis-\nrelated treatments,\n40,52 and gender bias and attitudes in flu-\nencing physician ’ s diagnostic and treatment decisions, 53,54\nwhich could be contributing to the underdocumentation of\nthe patient ’ s disease experience. An investigation 55 of the\npatterns of prevalence, incidence, and frequency of endome-\ntriosis reported that no data are available across the past\n30 years to attempt to document changes in presenting\nsymptom severity, life impact, or short- or long-term prog-\nnosis among women with endometriosis. Failure to obtain\nthe right data about these patients is a missed opportunity\nfor effectively targeting their unique needs,\n56 and we pro-\npose that this lack of proper documentation can be augment-\ned through patient self-report. Improving documentation\ncan provide the foundation to address the inequities in\ndiagnosis and care for those with endometriosis and that\nincluding the patient in the generation of such documenta-\ntion can be a starting point in this endeavor.\nOur inquiry is in line with the goal of LHS to create patient-\ncentered care systems and is further supported by previous\nrecommendations\n57 for leveraging health information tech-\nnologies to support and implement this goal. It is proposed\nthat patient-centered information technologies can be lever-\naged to democratize the health care system within the LHS\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.770\n\n\nthrough patient engagement and inclusion in knowledge co-\ncreation. As such, our study addresses a timely question by\nexploring the potential of patient self-tracking for augment-\ning the currently relied upon clinical data sources.\nObjectives\nUsing endometriosis as a grounding example, we investigate\nthe additional information that can be gained through mHealth\nself-tracking on conditions that are poorly understood and\ndocumented. Accordingly, this study aims to (1) describe\nendometriosis pain problems documented through self-track-\ning and in the EHR to investigate the potential of self-tracking\nfor augmenting the EHR documentation, (2) quantify the\nbetween- and within-personfluctuations in pain to character-\nize the dynamic nature of endometriosis, and (3) asses endo-\nmetriosis impact on daily function and its self-management\ndocumented through self-tracking as additional data elements\nthat can provide information about the burden of disease and\npossible points of intervention for disease management.\n42,43\nMethods\nStudy design and protocols were approved by the Columbia\nUniversity Irving Medical Center (CUIMC) Institutional Review\nBoard (AAAQ9812). This was a retrospective study conducted\nwith data from two independent samples of women with a\nclinician or surgery confirmed diagnosis of endometriosis.\nSelf-Tracking Sample\nWe obtained self-tracking data from Phendo, an observational\nresearch app for tracking endometriosis symptoms and avail-\nable for iOS\n1 and Android. 2 Phendo allows anyone with\nendometriosis around the world to download the app at any\ntime to characterize their endometriosis experience by track-\ning symptoms, activities, and self-management techniques.\nUsers are free to track as much or as sporadically as they wish,\nand they do not receive any prompts or requests to track a\nspecific variable from the research team. Endometriosis diag-\nnosis is determined through self-report based on participants’\nresponse to a question asking whether the participant has\nendometriosis and if so, how it was diagnosed (i.e., surgery or\nclinician confirmation). Phendo was designed using a partici-\npatory design approach through a series of qualitative and\nquantitative studies with endometriosis patients, with the\ngoal of creating a patient-centered tool that engages the user\nas an active participant in the research on better understand-\ning endometriosis.\n58–60 As such, users of Phendo self-track as a\nform of participatory research to contribute to creation of\nbetter documentation of the patient disease experience.\nAligned with goals of the LHS for promoting patient engage-\nment,\n6,61 these aspects of Phendo make it particularly suitable\nfor undertaking this investigation.\nElectronic Health Record Cohort Extraction\nThe Observational Health Data Sciences and Informatics\n(OHDSI) de-identi fied instance of the CUIMC Data Ware-\nhouse62,63 was queried to select a cohort of endometriosis\ncases for description of pain problems documented in the\nEHR. The CUIMC EHR system is used across the medical\ncenter university hospitals, clinics, and doctors’ offices across\nthe City of New York, which has a population of approxi-\nmately 8.44 million, including 43% White, 24% Black, 14%\nAsian, 15% other race, and 3.5% two or more races.\n64 The\nclinical data warehouse includes both inpatient and outpa-\ntient records and contains 36,578 “concepts” (i.e., clinical\nentities such as findings, diagnoses, drugs, and procedures)\nincluding 11,952 conditions, 12,334 drugs, and 10,816 pro-\ncedures from 5,364,781 patients. 65\nWe extracted the data using the OHDSI Observational\nMedical Outcomes Partnership (OMOP) Common Data Model\n(CDM)62,66,67 and SNOMED CT as the standardized EHR\nvocabulary.68 The CDM formally speci fies the encoding and\nrelationships among EHR concepts in a consistent and stan-\ndardized way using a hierarchy of subtypes. 62,68 This har-\nmonizes disparate coding systems with minimal information\nloss to a standardized vocabulary by mapping source con-\ncepts from different systems (e.g., ICD-9, ICD-10, RxNorm,\nCPT4, NDC, etc.) onto a standard concept ID during the\nextract-transform-load (ETL) process. SNOMED CT is the\ndesignated U.S. standard terminology for EHR diagnosis\nand problem lists, and has distinct advantages over ICD\ncodes for capturing discrete patient care diagnoses.\n69,70 It\nhierarchically organizes concepts, which allows aggregation\nof information based on subtype classi fication. More specific\n(i.e., descendent or child) concepts have more granularity\nand detail, whereas more general (i.e., ascendent or parent)\nconcepts have less granularity and clinical detail, but they\naggregate similar subtype concepts (e.g., “abdominal tender-\nness of left lower quadrant” and “right lower quadrant pain”)\ninto logical groups (e.g., “abdominal pain ”) which can be a\ndesirable feature when conducting statistical analyses.\nElectronic Health Record Sample\nA cohort of women with endometriosis in the EHR was\ncreated using the following set of criteria that have previ-\nously been demonstrated 71 to have a speci ficity of 93% and\nprecision of 85% for correctly selecting endometriosis\npatients in the EHR: women 15 to 49 years old with an\nendometriosis-related or endometriosis-prevalent proce-\ndure, an endometriosis concept code 30 days prior or post\neither of these procedures and documentation of an endo-\nmetriosis concept code after the initial procedure. For endo-\nmetriosis condition codes, we used the SNOMED concepts\n“endometriosis” (129103003), “endometriosis of the ovary ”\n(266589005), “endometriosis of the pelvic peritoneum ”\n(198251001), and “endometriosis of the uterus ” (76376003).\nOutcomes\nSelf-Tracked Pain Problems\nThe Phendo app includes four pain-related questions\n(►Table 1 ). Item “Where is the pain? ” allows users to select\nresponses (e.g., “cervix,”“ vagina,” and “chest”) using a visual\npain scale ( ►Supplementary Figure S1 , available in the\nonline version), similar to the McGill Pain Questionnaire. 72\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 771\n\n\nItem “any gastrointestinal/urine symptoms? ” allows partic -\nipants to report painful bowel movements (i.e., dyschezia),\ndysuria, and epigastric (e.g., gas) pain.\nPain Problems in the Electronic Health Record\nWe selected all pain-related problems documented in the\npatients’ records and computed their prevalence rates to\ndescribe documentation of endometriosis-related pain in the\nEHR. We chose to investigate the problem lists in the EHR due\nto their relevance for both research and clinical practice.\nProblem lists provide a structured, systematic way to contain\nclinician notes, facilitating access to data and conducting\nresearch.\n11 From a clinical standpoint, they improve efficien-\ncy for multiple providers to coordinate patient management,\nand for those who are taking care of new patients to\nfamiliarize themselves with a patient ’ s problems quickly. 73\nPain Variability\nInvestigations of within- and between-individual variability\nin endometriosis pain were conducted using self-tracked\ndata from Phendo participants. While signi ficant pain vari-\nability has previously been reported in other conditions,\n74\nbetween- versus within-individual variability in pain over\ntime in endometriosis has not been quanti fied. We under-\ntook this investigation because disease course over time and\nacross patients can have important implications for deriving\ncomprehensive profiles of patient disease history and status,\nidentification of disease phenotypes,\n74 and for predicting\nchange in disease course or functional capacity over time. 75\nTo jointly assess variability for both dimensions of area\nand severity of pain, we created a composite day-level pain\nscore by adding the severity scores reported for each body\narea (e.g., moderate pain in abdomen, mild pains in chest and\nleg would yield 2 þ 1 þ 1 ¼ 4 as the total score), removing any\nproblem-severity duplicate for the day. Removing day-level\nduplicates for pain area-severity pairs circumvents several\nissues: (1) the possibility of rumination/catastrophizing,\nwhich might look like, for example, a participant tracking\nthe same intensity of pain in the same area multiple times\nwithin a span of several hours, (2) to account for the day-level\ntracking habits of the participants (e.g., some participants\nmight track a pain area once at the end of the day even if they\nexperience it multiple times during the day), and (3) any\npossible errors during data transfer from the app to the data\nwarehouse (e.g., if there is no reception or wi- fi at the time of\ntracking, there might be a delay in data dumping, potentially\nthough infrequently leading to duplicates of the entry into\nthe data warehouse).\nAdditional Data Elements\nPrevalence of affected daily function tasks and self-manage-\nment techniques were assessed for the Phendo sample.\nParticipants can track free-text responses to these two items,\nwhich then get mapped onto common terms for standardi-\nzation (e.g., “go to a family gathering, ”“ hang out with\nfriends” get mapped onto “socialize”). This standardization\nprocess has been used in other previous research from our\ngroup,71,76 which we maintain for this analysis, and rely on\npublished literature 43,52 for generalizability and compara-\nbility of the prevalence estimates.\nData Analytic Approach\nPrevalence of Pain Problems\nIn Phendo, we computed total counts and percentages of pain\nproblems across the entire sample by including the total\nsample size in the denominator instead of limiting to the\nnumber of participants to those who ever tracked a pain\nproblem, as a more conservative approach. In the EHR\nsample, we computed counts and percentages of pain prob-\nlems, aggregating at the ascendent concept level, as this\nallows for logical groupings of pain problems.\nFor pain problems that exist in both samples, we con-\nducted two-sample tests for equality of proportions.\n77 We\nfurther computed the standardized z scores and the con fi-\ndence intervals around the difference between the two\nTable 1 Phendo pain-related variables and tracking statistics ( n ¼ 6,925)\nQuestion Sample\nsize\nTracked instances\n(median/mean/SD/range)\nTracking frequency in\ndays (mean/SD/range)\nWhere is the pain?\nHow severe is the pain? a\n5,205 42/105.5/162.5/1 –965 6.3/26.4/1 –1,147\nAny other pain symptoms? b\nHow severe is the symptom? a\n4,413 25/66.6/103.8/1 –536 8.2/30.7/1 –1,147\nAny gastrointestinal/urine issues?\nHow severe is the symptom? b\n4,578 30/71.9/106.8/1 –573 7.4/29.1/1 –1,147\nHow was sex? (dyspareunia) 676 50/85.0/87.5/1 –344 4.7/19.7/1 –609\nWhich activities are hard to do? 5,764 36/96.5/142.1/1 –708 5.5/28.2/1 –999\nWhat did you do to self-manage?\nDid it help? ( “helpful ” or “not helpful/no effect ”)\n6,025 40/106.0/164.9/1 –958 5.3/27.4/1 –1,019\nAbbreviation: SD, standard deviation.\naSeverity of the pain is tracked as “mild” (1), “moderate ” (2), and “severe” (3).\nbResponses to item “any other pain symptoms? ” were included to capture pain reports in instances where a participant might have tracked this item\nbut not item 1 or 3.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.772\n\n\nproportions.78 We a priori chose to compute prevalence rates\nbased on the aggregate parent concepts due to reasons\nrelated to the EHR data architecture described earlier (See\nElectronic Health Record Cohort Extraction ).\nVariability of Pain\nTo describe the distribution of severity levels across pain\nproblems at the sample level, we computed frequencies of\nthree severity levels for each body area (i.e., each pain\nlocation-severity combination is summed and divided by\nthe participant’ s total counts of tracks to account for tracking\nfrequency). To quantify pain variability, we first computed\nthe ratio of within-person to between-person variability\n75\n(i.e., average within-person standard deviation (SD) divided\nby the SD of the overall mean of all the participant-level\nmean scores). This ratio provides a standardized expression\nof the temporal variability relative to the distribution of the\nsample scores.\n75 This is analogous to computing an effect size\n(e.g., Cohen’ s d or a t-score) where the magnitude of effect is\nexpressed in SDs, and provides two advantages that make it\nparticularly suitable when there is both substantial average\nwithin- and between-person variability in the data. First,\nbecause change is expressed as a standardized score, change\nand its statistical signi ficance can be evaluated for an indi-\nvidual without referring to scores from other individuals.\nSecond, it incorporates between-person differences in mag-\nnitude of variability by adjusting the change in terms of each\nindividual’ s magnitude of variability.\n75\nNext, we estimated the between- and within-person var-\niances using a linear mixed-effects model (LMM) where day-\nlevel pain was regressed on the grouping variable of partici-\npant as a random effect. LMMs are commonly used in repeated\nmeasures designs and provide estimates of variance in the\noutcome (i.e., pain) explained by the predictor variables (i.e.,\nparticipant).\n79 The predictor term for participant in the model\nis included as a random effect, which handles nonindepen-\ndence in the data and estimates a separate intercept for each\nparticipant. This provides an estimate of the between-partici-\npant variance, that is, variation in pain explained by the\n“grouping structure” where, group here refers to each partici-\npant. In this type of model where the only predictor is the\nparticipant variable and the intercept, the total variance is\nassumed to equal to the sum of between-participant and\nwithin-participant (i.e., the residual) variances. Using these\nestimates, we computed the intraclass correlation (ICC), which\nis calculated as the variance among participant means over the\nsum of participant-level and data-level (residual) variances.\n80\nThe ICC (also referred to as “repeatability”) is a common and\nrecommended measure for assessing stability or fluctuation\nover time, and represents the fraction of the total variance in\nthe population of interest that can be attributed to variation\namong groups. 80 It further provides an estimation of the\nexpected correlation between two randomly drawn units\nthat are in the same group81 (e.g., two repeated measurements\nof pain for one individual). Larger ICC values indicate larger\nwithin-group correlation or repeatability and larger magni-\ntude of the variance explained by the between-group varia-\ntion. Finally, we estimated the confidence intervals for the ICC\nusing parametric bootstrapping and the statistical significance\nof the ICC in the LMM using a likelihood ratio test (LRT).\n80,82\nThe LRT assesses goodness-of-fit between two competing (e.g.,\nnull and alternative) models by comparing their log\nlikelihoods.\nAdditional Data Elements\nUsing the Phendo sample, we computed prevalence rates for\ndaily function tasks, self-management techniques, and fre-\nquency of effectiveness (and its SD) of each self-management\ntechnique.\nResults\nSample Characteristics\nPhendo sample characteristics are provided in ►Table 2 .\nThere were 6,925 participants with a surgical- or clinician-\nconfirmed diagnosis of endometriosis who responded to at\nleast one of the questions pertaining to pain, daily function\ntasks, or self-management techniques. The EHR query iden-\ntified 4,389 endometriosis patients, which translates to a\nprevalence of 0.4% documented endometriosis cases (i.e., out\nof 1,097,250 females in the same age bracket documented\nwithin the same time frame) in the clinical database. The\nsample consisted of women aged 14 to 49, with a mean age of\n36.4 (SD ¼ 7.4) and median age of 37 (mean absolute devia-\ntion ¼ 7.4). With respect to race, 1,227 (56.0%) were white,\n431 (9.8%) were Black/African-American, 202 (4.6%) were\nAsian, 18 (0.4%) were other Paci fic -Islander, 23 (0.5%) were\nAsian Indian, 3 (0.06%) were American Indian or Alaska\nNative, 20 (0.4%) were of other race, and 2,462 (56.0%) did\nnot have race data. Ethnicity data were unavailable on more\nthan half the patients; 651 (14.8%) were Hispanic, 1,420\n(32.3%) were non-Hispanic, and no data were available from\n2,318 patients. We further searched for data on other demo-\ngraphic factors (e.g., such as employment and education\nstatus); however, no further information was available.\nOutcomes\nD e s c r i p t i o no fP a i nP r o b l e m si nP h e n d o\nA visual depiction of the prevalence of pain problems in Phendo\nare provided in ►Figure 1 . Pelvic pain was the most prevalent\nproblem (57.3%), followed by lower back pain (49.2%) and\novarian pain (43.7%). Gastrointestinal (i.e., epigastric and in-\ntestinal) problems collectively were prevalent in 55.9% of the\nsample, comparable to that of pelvic pain. Unique pain prob-\nlems identified through self-tracking included pain in ovaries,\nuterus, vagina, cervix, dyschezia, dyspareunia, and rectal pain\n(\n►Supplementary Table S1 , available in the online version).\nD e s c r i p t i o no fP a i nP r o b l e m si nt h eE l e c t r o n i cH e a l t h\nRecord\nPain problem counts and prevalence rates in the EHR sample\nare provided in ►Supplementary Table S2 (available in the\nonline version). We identi fied 19 parent pain concepts and\n101 unique descendent concepts mapped onto one or more\nparent concepts. Abdominal pain was the most prevalent\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 773\n\n\npain problem (36.5%), followed by pain in pelvis (29.8%). Rest\nof the pain problems were documented for less than 10% of\nthe patients, including chest pain (9%), dysuria (4.3%), and\nbackache (6.8%). Of note, 856 pelvic pain concepts were also\nmapped under “abdominal pain,” which included dysmenor-\nrhea, renal colic, and bladder pain. Excluding these repeated\ncases, there were 1,147 unique “abdominal pain ” concepts\ndocumented in the EHR sample (26.1%). Upon further in-\nspection, we identi fied 101 instances of breast pain under\nthe “chest pain” concept (2.3%), and 128 instances of epigas-\ntric pain under “abdominal pain ” (2.9%).\nTwo Samples Proportion Tests\nResults from the two-sample proportion tests are provided\nin ►Table 3 . All pain problems were signi ficantly more\nprevalent in the Phendo sample ( p ¼ 0.01 for abdominal\npain, p < 0.001 for all others). Largest differences in propor-\ntions were observed for back pain, headache/migraines, and\npelvic pain ( ►Table 3 ).\nVariability in Pain\nQualitative Description of Variability\nVisualization of self-tracked pain reports at different levels of\nseverity and through time indicated large variability in the\naverage intensities reported across participants ( ►Figures 2\nand 3). Moderate pain was the most prevalent severity across\nbody areas (►Figure 2 top), in line with previous literature on\npain typology in endometriosis.45 However, tracking frequen-\ncies of all three severity levels signi ficantly varied for all pain\nproblems based on the SDs ( ►Figure 2 bottom). The trajecto-\nries of pain frequencies in a subsample of 194 participants over\na 4-week duration are depicted using a Sankey ’ s diagram\n(\n►Figure 3 top), which is a type of flow diagram for showing\nfactors’ states and transitions over time. 83 For easier visual\nrepresentation of the differential path transitions represented\nby the “links” across 4 weeks, weekly frequencies are catego-\nrized as low ( <5 times), moderate (5– 10 times), or high ( >10\ntimes). The cut-off points were selected such that each 5-point\ninterval corresponds to approximately half a SD, based on the\nsub-sample’ s mean (9.0) and SD (11.2) for weekly pain fre-\nquency. The links connecting the bars ( “nodes”)d e p i c tt h e\nvariability in the pain frequency through time, both within and\nbetween participants. The tile grid diagram (\n►Figure 3 bot-\ntom) shows dailyfluctuations in pain location and severity over\ntime for a single Phendo participant. Darker colors indicate\nlarger number of pain locations reported by the participant. For\nexample, the depicted participant reported severe pain in three\nbody locations on March 19, while on March 20, they reported\nmoderate and severe pain in six body locations each.\nQuantitative Description of Variability\nAll participants who provided at least 7 days of tracking data\non pain were included in the variability analysis, yielding a\ntotal of 43,539 person-level days from 1,534 Phendo partic -\nipants. Mean average (i.e., “mean of means ”) day-level pain\nscore was 7.75 (SD ¼ 5.61). Median average day-level pain\nscore was 6.21 (mean absolute deviation ¼ 3.79). Within-\nTable 2 Descriptive statistics for the Phendo sample\nParticipant characteristics Mean (SD)/Frequency (%)\nAge (y) ( N ¼ 5,915) 29.6 (6.9)\nMedian ¼ 28.9 (MAD ¼ 7.2)\nBody mass index ( N ¼ 5,522)a 26.5 (7.0)\nMedian ¼ 24.7 (MAD ¼ 5.6)\nEndometriosis diagnosis\nSurgically con firmed\nClinician con firmed\n5,224 (75.4)\n1,701 (24.5)\nEmployment status\nEmployed 3,876 (55.9)\nNot employed 856 (12.3)\nStudent 844 (12.1)\nUnknown 1,349 (19.4)\nRace/Ethnicity\nNon-Hispanic White 4,861 (70.1)\nNon-Hispanic Black 160 (2.3)\nHispanic (White or non-White) 327 (4.7)\nAsian 156 (2.2)\nNative American 45 (0.6)\nOther (including mixed\nrace/ethnicity)\n371 (5.3)\nUnknown 1,005 (14.5)\nMarital status\nMarried/in a domestic\npartnership\n3,298 (47.5)\nSeparated/divorced 191 (2.7)\nSingle, never married 2,086 (30.1)\nWidowed 3 (0.04)\nUnknown 1,016 (14.6)\nLiving environment\nUrban 2,428 (35.0)\nSuburban 2,489 (35.9)\nRural 992 (14.3)\nUnknown 17 (0.38)\nPrimary work location\nHome 1,481 (21.3)\nOutside home 4,084 (58.9)\nUnknown 1,360 (19.6)\nEducation level\nHigh school or less 831 (2)\nSome college 1,828 (26.3)\nCollege or higher 3,251 (46.9)\nUnknown 1,015 (14.6)\nAbbreviations: BMI, body mass index; MAD, mean absolute deviation;\nSD, standard deviation.\nNote: Unknown indicates not tracked by the participant in the Phendo\napp pro file, and therefore, data are not available.\naValues below 12 and over 70 are excluded from BMI summary statistics.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.774\n\n\nperson SD in day-level pain was 5.17 (SD ¼ 3.45). Expressed\nin SDs to obtain a standardized measure of the within-person\nvariability75 (5.17/5.61), this yields a ratio of 0.92 and similar\nto other effects sizes, can be interpreted as large. 75\nResults of the LMM analysis are provided in ►Table 4 .T h e\nICC was 0.42 (95% con fidence interval ¼ 0.40– 0.44) and was\nstatistically signi ficant (LRT Chi-square ¼ 22,600, p < 0.001).\nThis indicates that the 42% of the variance in daily pain is\nattributed to between-participant variation and that pain\nscores are signi ficantly correlated from day to day for each\nparticipant. Within-person variance estimate was 38.72,\nindicating that approximately 58% of the total variance in\ndaily pain is attributed to within-participant variation\n(►Table 4 ). A plot of each participant ’ s estimated mean\nscores and con fidence intervals are provided in ►Figure 4 ,\ndemonstrating the variability that occurs across participants\n(x-axis), and within-participant (y-axis). Taken together,\nthese suggest that substantial variability occurs not just\nbetween participants but also within-participant over\ntime. Of note, number of tracked days was not a signi ficant\nFig. 1 Prevalence of pain problems reported in the Phendo sample.\nTable 3 Results of the two-sample test for equality of proportions\nPain problem\n(SNOMED ID)\nEHR counts\n(proportion)\nPhendo counts\n(proportion)\nChi-square\nstatistic\n95% CI z statistic\nAbdominal a (200219) 1,603 (0.36) 2,690 (0.38) 6.14 ( p ¼ 0.01) /C0 0.04 to 0.004 /C0 2.47\nAbdominal\n(200219)\n885 (0.20) 2,690 (0.38) 433.74 b /C0 0.20 to 0.17 /C0 20.82\nHeadache/Migraines\n(378253/318736)\n480 (0.05) 2,644 (0.38) 1,494 b /C0 0.33 to 0.31 /C0 38.65\nBackache (134736) 299 (0.06) 3,414 (0.49) 2,198.30 b /C0 0.43 to 0.41 /C0 46.88\nChest (77670) 405 (0.09) 1,026 (0.14) 75.93 b /C0 0.06 to 0.04 /C0 8.71\nJoint (77074) 334 (0.07) 733 (0.10) 27.83 b /C0 0.04 to 0.02 /C0 5.27\nDysuria (197684) 191 (0.04) 944 (0.13) 256.33 b /C0 0.10 to 0.08 /C0 16.01\nLower limb (4024561) 261 (0.05) 1,998 (0.28) 882 b /C0 0.24 to 0.21 /C0 29.69\nUpper limb (4009890) 149 (0.03) 419 (0.06) 39.73 b /C0 0.03 to 0.06 /C0 6.30\nHip (200219) 35 (0.003) 570 (0.08) 338.49 b /C0 0.08 to 0.07 /C0 18.39\nBone (4129418) 16 (0.03) 570 (0.08) 338.49 b /C0 0.003 to 0.08 /C0 18.39\nPelvis (4147829) a 1,312 (0.29) 3,969 (0.57) 811.57 b /C0 0.29 to 0.25 /C0 28.48\nAbbreviations: CI, con fidence interval; EHR, electronic health record.\nNote: 95% con fidence interval built around the difference in proportions.\naIncludes the 885 conditions that are mapped under both “abdominal pain ” and “pain in pelvis. ”\nbp < 0.0001.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 775\n\n\ncovariate in the variance estimation model (B ¼ 0.001, stan-\ndard error ¼ 0.003, t ¼ 0.485, p ¼ 0.62), or correlated with\npain scores (Pearson ’ s r ¼ 0.01, t ¼ 0.41, p ¼ 0.68). Therefore,\nit was not included in the final model as a covariate.\nAdditional Data Elements\nPrevalence rates of reported affected daily function tasks are\nprovided in ►Table 5 . We identified a total of 22 distinct tasks\nreported by the participants, who on average reported three\nunique affected tasks (SD ¼ 2.2, range ¼ 1– 14). Working\n(53.1%), getting out of bed (52.2%), standing (47.4%), and using\nthe toilet (47.3%) were the most frequently reported tasks. Of\nnote, 43% of the participants reported sleeping as an affected\ntask. Similarly, there were multiple physical mobility-related\ntasks reported including sitting, walking, and climbing the\nstairs, reported by at least 30% of the participants.\nPrevalence rates of all self-management techniques are\nprovided in ►Figure 5 and ►Supplementary Table S3 (avail-\nable in the online version). Participants reported a median of\nthree unique techniques (mean ¼ 3.21, SD ¼ 2.21, range ¼ 1–\n14). Rest and using a heat pack were the most prevalent (52.4\nand 50.7%, respectively). Assessment of effectiveness frequen-\ncy indicated considerable variability in the reported helpful-\nness of the techniques. For example, breathing exercises\n(reported as the third most frequently used technique by\n38%) and ice packs were tracked as unhelpful 51 and 52% of\nFig. 2 Top: Prevalence of mild, moderate, and severe pains by body locati on as reported in the Phendo cohort. Each body area-intensity pair\ncounted once per participant. For all body locations, participants tracked predominantly moderate pain, with severe pain as second most\ncommon, and mild as least common. Bottom: Mean proportion of severity reported for each body area normalized by total tracks of pain reports\nand averaged across the sample. One-sided error bars were used for visual display purposes and represent standard deviations.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.776\n\n\nthe time on average. Moreover, even though rest was the most\nprevalent technique, it was reported as helpful only 55.3% of\nthe time on average. On the other hand, acupuncture and\nmedical marijuana were the least prevalent (2.8 and 4.6%,\nrespectively) but most frequently effective techniques on\naverage (78.4 and 74.6% of the time, respectively).\nDiscussion\nIn this study, we investigate the potential of using direct\npatient input via mobile technology to identify additional\ndata elements that can be obtained for contributing to\nexisting data sources in the context of enigmatic disease:\n(1) improve prevalence estimates and characterization of\ntraditionally underdocumented diseases, (2) provide multi-\ndimensional data (e.g., symptom severity, location, and\ntemporal fluctuations84) on the disease for generating infor-\nmation that can aid in the personalization of care, and (3)\nprovide contextual information to characterize burden of\ndisease under real life circumstances and identify possible\npoints of intervention. To our knowledge, this study is the\nfirst to provide a quantitative analysis to demonstrate how\nparticipatory research-based self-tracking data compare\nwith EHR and what additional information could be\nextracted for investigation of underdocumented diseases.\nOur finding that pelvic pain is the most prevalent symp-\ntom in our sample is in line with previous studies that use\nself-reported questionnaires to assess pain symptomology in\nendometriosis.\n45 We extend this by reporting signi ficantly\nfewer instances of pelvic pain and lack of any documentation\nof several other endometriosis-related pain problems (e.g.,\ndyschezia\n85) in the EHR. Similarly, epigastric problems were\nsignificantly underdocumented in the EHR, even though this\nwas the fourth most frequently reported problem in the\nPhendo sample. This could be due to various reasons. There\ncould have been a higher number of patients presenting with\nepigastric pain, but these instances might have been docu-\nmented as “abdominal pain ” by the provider in the EHR\nwithout providing further details. From the patient ’ s\nFig. 3 Top: Sankey diagram of pain frequencies over time in a sample of 194 Phendo participants over the course of a 4-week period. Weekly\nfrequencies are categorized into low ( <5t i m e s ) ,m o d e r a t e( 5–10 times), and high ( >10 times), depicted by the different colors. The links depict\nthe variability in the pain frequency through time, both within participant, and between participants. Bottom: Daily fluctuations in pain intensity\nover time for a sample individual in the Phendo cohort by severity and nu mber of pain locations. Darker color indicates larger number of pain\nlocations reported by the participant. For example, the depicted participant reported severe pain in three body locations on March 19, while on\nMarch 20, they reported moderate and severe pain in six body locations each.\nTable 4 Between- and within-person variance estimates\nobtained from the linear mixed-effects model ( n ¼ 1,534)\nBetween-person\nvariance (t ^2)\nWithin-person\nvariance ( σ^2)\nVariance SD\n(95% CI)\nVariance SD\n(95% CI)\nPain\nscore\n28.7 5.3\n(5.1–5.5)\n39.2 6.2\n(6.1–6.2)\nAbbreviations: CI, con fidence interval; ICC, intraclass correlation coef-\nficient; SD, standard deviation; SE, standard error.\nNote: Phendo participants who provided at least 7 days of data were\nincluded in the analysis. Number of days was not a signi ficant predictor\nin the model (B ¼ 0.001, SE ¼ 0.003, t ¼ 0.485, p ¼ 0.62) and therefore\nexcluded from the final models for estimating variances and the ICC.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 777\n\n\nperspective, previous studies report patient resistance to\ndisclosing pain, 86 and that patient ’ s physical pain can nega-\ntively affect patient– provider communication.87 It is possible\nthat patients feel less stigmatized when sharing their symp-\ntoms through a mobile app rather than directly sharing with\ntheir physician.\n53,88 Another possibility could be lack of\nadequate health insurance for access to point of care, which\nhas been linked to not only delays in diagnosis, but also\ndisparities in primary and/or specialist care and treat-\nment.\n40,89 A similar study 90 comparing patient self-reported\nsurveys to EHR problem lists and chart notes for common\nmedical conditions reported that fewer than half of the 380\nproblems were shared across the data sources, and that 22%\nof all diagnoses were found only in patient self-report. Our\nfindings reinforce these findings in demonstrating direct\npatient input provides an opportunity to bring these differ-\nent data sources together to improve our understanding of\npatient health history and disease status.\n90 We further\ndemonstrate the added value of mHealth tools designed\nbased on participatory research to promote patient engage-\nment and self-tracking for improving our understanding of\ndiseases that are underdocumented and clinically not well\nunderstood. Within the LHS framework, these have been\nidentified as opportunities for patient inclusion to democra-\ntize the health care system.\n57\nAssessment of variation in pain indicated substantial\nbetween- and within-individual variability in our sample,\nwhich is in line with findings from other chronic pain\nsamples reported in the literature. 74 It has been long alluded\nto that endometriosis is a heterogeneous disease that fluc -\ntuates in symptoms within individual over time, and that the\nmagnitude of this fluctuation is moreover variable across\nindividuals, yet this has not been quanti fied. Our results\nprovide quantitative support for the dynamic nature of\nFig. 4 Plot of the person-level pain scores e stimated from the multilevel model ( n ¼ 1,534). Y-axis ( “effect range ”) represents pain scores. Each\nblack dot represents one participant and gray lines indicate 95% con fidence intervals. Distribution of points across the x-axis indicate large\nvariability across individuals (i.e., between-group variance). D ark gray lines indicate random effects that are statistically signi ficant.\nTable 5 Prevalence of daily function tasks reported (%) in the\nPhendo sample\nDaily function task Count Prevalence (%)\nWorking 3,681 53.1\nGetting out of bed 3,619 52.2\nStanding 3,287 47.4\nUsing the toilet 3,281 47.3\nSitting 3,273 47.2\nWalking 3,214 46.4\nSocializing 3,122 45.1\nSleeping 2,950 42.6\nDressing 2,706 39.0\nClimbing stairs 2,334 33.7\nRunning 2,294 33.1\nEating 2,292 33.1\nPreparing food 2,230 32.2\nStretching 2,191 31.6\nLying down 1,917 27.6\nJumping 1,812 26.1\nSex 1,778 25.6\nHousework 1,755 25.3\nShopping 1,638 23.6\nLifting 1,469 21.2\nKneeling 1,309 18.9\nBathing 854 12.3\nNote: Out of the total 6,925, all 5,764 participants tracked at least one\ndaily function task in Phendo. Denominator of 6,925 is used to compute\nprevalence rates.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.778\n\n\nendometriosis pain and quantifying this variability is a\nstarting point for being able to derive clinically relevant\nassociations. That a patient ’ s pain symptoms over time\nfluctuate almost as much as the variability in overall pain\nexperience observed across a sample of patients indicates, at\nminimum, that information gathered about a patient at\nsingle contact time point (e.g., doctor of fice visit) might\nnot be representative of the person ’ s overall disease experi-\nence as it unfolds over time. Similarly, attempts to character-\nize endometriosis through data collection designs relying\nsolely on EHR problem lists might yield incomplete or biased\nresults, findings supported by previous studies conducting in\nother conditions.\n14,16,90 From a clinical standpoint, pain\nvariability has been demonstrated to be predictive of treat-\nment response and disease outcomes. 79,91 For example, a\nstudy79 reported that higher within-person pain variability\nat baseline in a randomized controlled trial for a fibromyalgia\npain drug was predictive of placebo response and nonspecific\ntreatment effects. In another study with fibromyalgia\npatients, authors demonstrated that between- and within-\nperson variability predicted disease symptomology clusters\nand help identify disease phenotypes. In sum, frequent\nmonitoring can capture this dynamic nature of chronic\npain, yielding a more comprehensive and accurate disease\nprofile than what can be estimated through EHR which relies\non contact incidence.\nOur observations on prevalence of impaired daily func -\ntion are in line with others who report substantial disease\nimpact on QoL in endometriosis.\n43,50,52 For example, Phendo\nparticipants reported work as their most affected task and\nalmost half of the sample report their sleep being affected by\ntheir endometriosis, findings previously reported in a survey\nwith 107 Puerto Rican women (mean age ¼ 34.5) with en-\ndometriosis.\n43 In addition, our findings on the prevalence of\nsocializing and walking as affected daily function tasks are\nclosely aligned with those reported in another sample of 193\nPuerto Rican women with endometriosis (45.0 vs. 48% and\n46.4 vs. 41.4%, respectively).\n52 With respect to self-manage-\nment techniques, acupuncture was self-reported as effective\nmost of the time ( /C24 87%), though only 2.8% of the Phendo\nparticipants reporting its use. Similarly, physical therapy was\nreported by only 4.6% of the participants. These low frequen-\ncies might indicate lack of access to such treatments, based\non prior evidence that insurance coverage, costs and changes\nin insurance policy have been identi fied as signi ficant bar-\nriers to receiving musculoskeletal (e.g., physical and pelvic)\ntherapy and rehabilitation services.\n92 The ability to identify\nthese nuances in patient experiences with respect to their\ndisease management is another strength of using direct\npatient input via mobile technology. Though beyond the\nscope of the aims of this study, between-individual variabili-\nty in the effects of self-management techniques merits\nfurther investigation. Taken together with our observations\non the self-management technique effect variability, these\nfindings not only underscore the burden of endometriosis\nand its impact on daily living, but also the potential of using\nFig. 5 Average effect of self-management techniques reported by the Phendo participants (n ¼ 6,025). One-sided error bars were used for visual display\nand indicate standard deviations. Proportions of each technique tracked as helpful or unhelpful (i.e.,“did not help” or “no effect”) out of the total number of\ntimes tracked was averaged across all participants. There was significant variability as indicated by the standard deviations. For example, for the average user,\nacupuncture was reported to be helpful approximately 75% of the time, and unhelpful approximately 25% of the time.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 779\n\n\ndirect input from the patient for identifying suitable targets\nfor intervention and effective strategies for symptom\nmanagement.\nWhile mHealth and self-tracking for self-management\nand improving health care have been widely advocated\nboth by the scienti fic community\n93,94 and public health\nagencies (e.g., WHO), 38 their use to generate better evi-\ndence about chronic diseases is still rare. 95,96 We demon-\nstrate that self-tracking data can be valuable particularly\nfor conducting observational research on pain-related dis-\neases that are poorly understood and/or studied. Another\nstrength of our study is the participatory design aspect of\nPhendo, which ensures that the items included in the app\nare relevant to the patient population. This has been\nidentified as a way to improve study retention and long-\nterm adherence to tracking,\n97 which is particularly impor-\ntant when the goal is to increase documentation base for an\nenigmatic disease.\n60 Indeed, the prevalence rate of endo-\nmetriosis cases in our EHR database was 0.4%, which is\nsubstantially lower than the reported 10% prevalence esti-\nmated in the population. This provides further rationale for\nfocusing efforts to augment the typically relied upon EHR\ndatabases through direct patient input to promote patient\nengagement and inclusion in the data generation process,\nwhich have been recommended as approaches for accom-\nplishing the patient-centered-care system promoted within\nthe LHS framework.\n57 Self-tracking facilitates documenta-\ntion of not only severe pain, but also mild and moderate\npain instances. This can reduce the likelihood of over-\nrepresenting severe cases, a potential limitation of EHR.\n41\nDaily tracking allows for monitoring acute pain, which can\nhave distinctive differences in physiology compared with\nchronic pain,\n98 but still be associated to subsequent risk for\ndeveloping chronic pain in clinical populations, 99,100 par-\nticularly in women. 101 As such, information on temporal\nfluctuations can improve our understanding of the dynamic\ncourse of the disease and subsequently making timely\nclinical decisions.\nWe note the potential of patient self-recording for im-\nproving the completeness of demographic data in the health-\ncare systems. This paucity of data on common demographic\nfactors (e.g., race, ethnicity and education level)\n102 and the\nexclusive reliance on ICD-9 coding 103 in the EHR is a com-\nmonly reported issue in the literature. 104 In a recent\nstudy,105 among the 2.4 million patients in New York Pres-\nbyterian Hospital healthcare system ’ s EHR (used also for\nextracting the EHR sample in this study), race, or ethnicity\nwas unknown for 57%, compared with 86% when patients\ndirectly recorded themselves. Accomplishing this is critical\nfor supporting precision medicine initiatives and reducing\nhealth care disparities and inequities.\nWe note several limitations in our study. First, we had\nindependent samples of women with endometriosis, and\nas such there might have been differences in characteristics\nof the two samples that in fluence the proportion estimates.\nNext, we were unable to conduct statistical tests of signi fi-\ncance for pain intensity, frequency of their fluctuations,\nand self-management techniques and daily function tasks\nreported, as these data were not provided in the EHR for\nour sample. Third, EHR data from the clinical warehouse\nwere available from January 1996 onward; therefore, our\nsearch precludes identi fication of cases prior to that year.\nDemographic factors were substantially missing in the\nEHR, an observation that has been consistently reported\nby others in the literature.\n102,103,105 Likewise, some of the\ndemographic information was also missing in the Phendo\nsample. The study samples therefore might not be repre-\nsentative of endometriosis patients of some socioeconomic\nand ethnic/racial backgrounds. Though we are not aware of\nrace/ethnicity as a predictive risk factor for endometriosis\ndiagnosis, we cannot ascertain if these results would have\nbeen different with a more diverse sample. Approximately,\n60% of the Phendo participants self-reported residing in the\nUnited States and approximately 30% reported residing in\nother countries, whereas EHR data were obtained from a\nsingle (though ethnoracially and linguistically diverse)\nmetropolitan city and we do not have data on the patients ’\nliving or working environments. As such, there could be\ndifferences between the two samples in comparison with\nrespect to environmental factors that could have in flu-\nenced the outcomes. Similarly, we do not know the exact\nbreakdown of specialist versus general medicine practi-\ntioners using the EHR system. Though beyond the scope of\nthe present study, future studies could investigate pain\nproblem documentation by practitioner specialty area.\nFinally, we note the challenges of investigations in the\nEHR, a topic extensively discussed in the literature,\n106,107\nwhich might have in fluenced our results. The ETL process\nthat converts the clinical data into the OMOP data tables is a\ndynamic process that improves over time, but is not\ncompletely error free. As such, the EHR search might have\nmissed some pain-related conditions. A recent study of EHR\nproblem list completeness and accuracy reports that pro-\nviders struggle more with maintaining complete problem\nlists for patients whose diseases status are less severe and\nless symptomatic.\n16 Accordingly, some of the patients iden-\ntified in the EHR cohort might have reported additional pain-\nrelated findings, but these might have not been documented\nby the provider.\nConclusion\nPatient self-tracking has been linked to increased self-ef fica-\ncy, patient adherence, engagement as advocates in their\npersonalized care, and improved health outcomes.\n108–110\nThis approach can enhance the research process by increas-\ning inclusion of those prone to health inequities,111 including\ndisease-level inequities as demonstrated herein using endo-\nmetriosis as a grounding example. Direct patient input via\nself-tracking can facilitate collecting and sharing informa-\ntion in a timely manner without increasing provider docu-\nmentation responsibility and associated burden. In the same\nway, the EHR are both the source of information and the\ntarget of interventions; mHealth applications could play a\ncritical role by promoting patient engagement in aspects of\nthe health care system.\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.780\n\n\nClinical Relevance Statement\nUse of mHealth technology for obtaining direct patient input\ncan supplement clinical documentation captured during\npoint of care to more accurately and comprehensively evalu-\nate patient health history and status. Patient inclusion in the\ndata generation and sharing process can further alleviate\nburden due to EHR use, and enable patient as an active\nparticipant in their care. This approach can be applied to\nother conditions beyond endometriosis, especially those that\nare chronic and fluctuating in symptoms over time.\nMultiple Choice Questions\n1. Which of the following is a primary endometriosis-related\nproblem detected in the self-tracking sample in line with\nprevious literature, but was found to be signi ficantly missing\nin the electronic health record (EHR)?\na. Hip pain\nb. Pelvic pain\nc. Breast pain\nd. Headache\nCorrect Answer:The correct answer is option b. Pelvic pain is a\nprimary characteristic symptom in endometriosis. It was the\nmost prevalent problem in the Phendo sample (57.3%, com-\npared with 29.8% in the EHR), a finding in line with previous\nstudies that assess pain symptomology in endometriosis.\n45\nUpon comparison, we further report significantly fewer instan-\nces of epigastric pain and dysuria in the EHR, as well as other\nendometriosis-related pain problems (e.g., dyschezia85).\n2. Which contributory aspect of direct patient input via\nmobile technology is investigated in this study?\na. Increase the number of patients we can add into the\nLearning Health System (LHS)\nb. Increase the mHealth use among stakeholders of LHS\nc. Expand upon the characterization of burden of disease in\nthe EHR\nd. Increase documentation on diseases that are\nunderdocumented\nCorrect Answer: The correct answer is option d. While there\nare numerous potential bene fits of including direct patient\ninput to complement information gathered from EHR for\nconducting research and informing medical decisions, this\nstudy focuses on how these data could help generate addi-\ntional information on underdocumented diseases (e.g., en-\ndometriosis) that would not be possible to obtain from EHR.\n3. In addition to clinical conditions, what other type of\ninformation has been indicated to be signi ficantly under-\ndocumented in the EHR?\na. Patient preferences for treatment\nb. Diagnosis dates\nc. Demographic factors\nd. Date of documentation\nCorrect Answer: The correct answer is option c. Information\non common demographic factors (e.g., race, ethnicity, and\neducation level) were substantially missing in the EHR in our\nanalyses, an observation that has been also reported by\nothers in the literature. For example, a study\n105 reported\nthat among the 2.4 million patients in a large New York City\nuniversity health care system ’ s EHR, race, or ethnicity was\nunknown for 57%, compared with 86% when patients directly\nrecorded themselves.\n4. This study investigates endometriosis pain variability based\non patient self-reported data collected frequently over time.\nWhat do the results of the linear mixed model (LMM) with\nrespect to the estimated intercepts (\n►Figure 4 ) indicate?\na. The ratio of within-participant to group-level pain vari-\nability is comparable.\nb. The mean day-level pain scores substantially vary across\nparticipants.\nc. There is signi ficant intraclass correlation of pain scores\nover time.\nd. Within-participant variation in daily pain accounts for\n58% of the total model variance.\nCorrect Answer: The correct answer is option b. The use\nof LMM with a random intercept for participant allows\nestimation of a separate person-level mean in daily pain\nscores, represented by the intercept. This random effect\nwas statistically signi ficant in the model, indicating that\nthere is substantial variability across participants in their\ndaily pain levels. This is depicted in\n►Figure 4 where\nvariability across participants is apparent across the x-axis,\nwhile within-participant variability is apparent across the y-\naxis, represented by the variable 95% con fidence intervals.\nProtection of Human and Animal Subjects\nAll procedures performed in studies involving human\nparticipants were in accordance with the ethical stand-\nards of the institutional and/or national research commit-\ntee and with the 1964 Helsinki Declaration and its later\namendments or comparable ethical standards. All proce-\ndures followed were in accordance with ethical standards\nof the responsible committee on human experimentation\n(institutional and national) and with the Helsinki Decla-\nration of 1975, as revised in 2000. Informed consent\nwas obtained from all patients for being included in the\nstudy.\n• Available at https://itunes.apple.com/us/app/phendo/\nid1145512423\n• Available at https://play.google.com/store/apps/details?\nid¼com.appliedinformaticsinc.phendo\nFunding\nThis study received financial support from Columbia\nUniversity Data Science Institute Postdoctoral Fellowship,\nEndometriosis Foundation of America, National Science\nFoundation (grant number: 1344668), and from National\nInstitutes of Health, U.S. National Library of Medicine\n(grant number: R01 LM013043).\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 781\n\n\nConflict of Interest\nNone declared.\nReferences\n1 Institute of Medicine (US) Roundtable on Evidence-Based Medi-\ncine Olsen L, Aisner D, McGinnis JM, eds. The Learning Health-\ncare System: Workshop Summary. Washington, DC: National\nAcademies Press; 2007\n2 McGinnis JM, Powers B, Grossmann C. Digital Infrastructure\nfor the Learning Health System: The Foundation for Continu-\nous Improvement in Health and Health Care: Workshop\nSeries Summary. Washington, DC: National Academies Press;\n2011\n3 Friedman CP, Wong AK, Blumenthal D. Achieving a nationwide\nlearning health system. Sci Transl Med 2010;2(57):57cm29\n4 Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, re-\nsponsive, relevant (R3) research: a call for a rapid learning health\nresearch enterprise. Clin Transl Med 2013;2(01):10\n5 Greene SM, Reid RJ, Larson EB. Implementing the learning health\nsystem: from concept to action. Ann Intern Med 2012;157(03):\n207– 210\n6 Green LW. Making research relevant: if it is an evidence-based\npractice, where ’ s the practice-based evidence? Fam Pract 2008;\n25(Suppl 1):i20 – i24\n7 Cortez A, Hsii P, Mitchell E, Riehl V, Smith P. Conceptualizing a\ndata infrastructure for the capture, use, and sharing of patient-\ngenerated health data in care delivery and research through\n2024. Accenture. . Available at: https://www.healthit.gov/sites/\ndefault/files/onc_pghd_practical_guide.pdf?plat-\nform¼hootsuite. Accessed January, 2018\n8 Casey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic\nhealth records for population health research: a review of methods\nand applications. Annu Rev Public Health 2016;37:61– 81\n9 Lin KJ, Singer DE, Glynn RJ, Murphy SN, Lii J, Schneeweiss S.\nIdentifying patients with high data completeness to improve\nvalidity of comparative effectiveness research in electronic\nhealth records data. Clin Pharmacol Ther 2018;103(05):\n899– 905\n10 Jensen PB, Jensen LJ, Brunak S. Mining electronic health records:\ntowards better research applications and clinical care. Nat Rev\nGenet 2012;13(06):395 – 405\n11 Acker B, Bronnert J, Brown T, et al. Problem list guidance in the\nEHR. J AHIMA 2011;82(09):52 – 58\n12 Blondeau C. Pocket Glossary of Health Information Management\nand Technology. 3rd Edition Chicago, IL: AHIMA; 2011\n13 Daskivich TJ, Abedi G, Kaplan SH, et al. Electronic health record\nproblem lists: accurate enough for risk adjustment? Am J Manag\nCare 2018;24(01):e24 – e29\n14 Wang M, Cyhaniuk A, Cooper DL, Iyer NN. Identi fication of\npatients with congenital hemophilia in a large electronic health\nrecord database. J Blood Med 2017;8:131 – 139\n15 Wang YC, Shimbo D, Muntner P, Moran AE, Krakoff LR, Schwartz\nJE. Prevalence of masked hypertension among US adults with\nnonelevated clinic blood pressure. Am J Epidemiol 2017;185\n(03):194– 202\n16 Wang EC -H, Wright A. Characterizing outpatient problem list\ncompleteness and duplications in the electronic health record.\nJ Am Med Inform Assoc 2020;27(08):1190 – 1197\n17 Ommaya AK, Cipriano PF, Hoyt DB, et al. Care-centered clinical\ndocumentation in the digital environment: solutions to alleviate\nburnout. NAM Perspectives . Available at: https://nam.edu/wp-\ncontent/uploads/2018/01/Care-Centered-Clinical-Documenta-\ntion.pdf. Accessed January 29, 2018\n18 Collier R. Electronic health records contributing to physician\nburnout. Can Med Assoc . Available at: https://europepmc.org/\narticle/med/29133547. Accessed 2017\n19 Gardner RL, Cooper E, Haskell J, et al. Physician stress and\nburnout: the impact of health information technology. J Am\nMed Inform Assoc 2019;26(02):106 – 114\n20 West SL, Johnson W, Visscher W, Kluckman M, Qin Y, Larsen A.\nThe challenges of linking health insurer claims with electronic\nmedical records. Health Informatics J 2014;20(01):22 – 34\n21 Adane K, Gizachew M, Kendie S. The role of medical data in\nefficient patient care delivery: a review. Risk Manag Healthc\nPolicy 2019;12:67 – 73\n22 Rhodes ET, Laffel LM, Gonzalez TV, Ludwig DS. Accuracy of\nadministrative coding for type 2 diabetes in children, adoles-\ncents, and young adults. Diabetes Care 2007;30(01):141 – 143\n23 Valikodath NG, Newman-Casey PA, Lee PP, Musch DC, Niziol LM,\nWoodward MA. Agreement of ocular symptom reporting be-\ntween patient-reported outcomes and medical records. JAMA\nOphthalmol 2017;135(03):225 – 231\n24 Cahill S, Makadon H. Sexual orientation and gender identity data\ncollection in clinical settings and in electronic health records: a key\nto ending LGBT health disparities. LGBT Health 2014;1(01):34– 41\n25 Bagley SC, Altman RB. Computing disease incidence, prevalence\nand comorbidity from electronic medical records. J Biomed\nInform 2016;63:108 – 111\n26 Almalki M, Gray K, Sanchez FM. The use of self-quanti fication\nsystems for personal health information: big data management\nactivities and prospects. Health Inf Sci Syst 2015;3(Suppl 1):S1\n27 Swan M. Emerging patient-driven health care models: an exam-\nination of health social networks, consumer personalized medi-\ncine and quanti fied self-tracking. Int J Environ Res Public Health\n2009;6(02):492– 525\n28 Khovanskaya V, Baumer EPS, Cosley D, Voida S, Gay G. Everybody\nknows what you’ re doing: a critical design approach to personal\ninformatics. Proceedings of the SIGCHI Conference on Human\nFactors in Computing Systems . Available at: https://stephen.\nvoida.com/uploads/Publications/Publications/khovanskaya-\nchi13.pdf. Accessed 2013\n29 Figueiredo M, Caldeira C, Chen Y, Zheng K. Routine self-tracking\nof health: reasons, facilitating factors, and the potential impact\non health management practices. AMIA Annu Symp Proc 2018;\n2017:706– 714\n30 Shapiro M, Johnston D, Wald J, Mon D. Patient-generated health\ndata. RTI International . Available at: https://www.healthit.gov/\nsites/default/files/rti_pghd_whitepaper_april_2012.pdfApril.\nAccessed April 2012\n31 Boyer GS, Templin DW, Goring WP, et al. Discrepancies between\npatient recall and the medical record. Potential impact on\ndiagnosis and clinical assessment of chronic disease. Arch Intern\nMed 1995;155(17):1868 – 1872\n32 Sampson UKA, Kaplan RM, Cooper RS, et al. Reducing health\ninequities in the U.S.: recommendations from the NHLBI ’ sh e a l t h\ninequities think tank meeting. J Am Coll Cardiol 2016;68(05):\n517– 524\n33 Ahmed S, Berzon RA, Revicki DAInternational Society for Quality\nof Life Research. , et al; . The use of patient-reported outcomes\n(PRO) within comparative effectiveness research: implications\nfor clinical practice and health care policy. Med Care 2012;50\n(12):1060– 1070\n34 Pichon A, Schiffer K, Horran E, Bakken S, Mamykina L, Elhadad N.\nDivided We Stand: The Collaborative Work of Patients and Pro-\nviders in an Enigmatic Chronic Disease. [under anonymous re-\nview]. . Available at: http://www.columbia.edu/ /C24 ab3886/docs/\nAPichon_CV_Jan2020.pdf. Accessed 2020\n35 Payne TH, Corley S, Cullen TA, et al. Report of the AMIA EHR-2020\ntask force on the status and future direction of EHRs. J Am Med\nInform Assoc 2015;22(05):1102 – 1110\n36 Rycroft-Malone J, Seers K, Titchen A, Harvey G, Kitson A, McCor-\nmack B. What counts as evidence in evidence-based practice?\nJ Adv Nurs 2004;47(01):81 – 90\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.782\n\n\n37 Dullabh P, Hovey L, Heaney-Huls K, Rajendran N, Wright A, Sittig\nDF. Application programming interfaces in health care: findings\nfrom a current-state sociotechnical assessment. Appl Clin Inform\n2020;11(01):59– 69\n38 Executive B. mHealth: use of appropriate digital technologies for\npublic health: report by the Director-General. GenevaWorld\nHealth Organization; Available at: https://apps.who.int/iris/bit-\nstream/handle/10665/274134/B142_20-en.pdf?sequence ¼1&\nisAllowed¼y. Accessed November 27, 2017\n39 Tamaresis JS, Irwin JC, Goldfien GA, et al. Molecular classification\nof endometriosis and disease stage using high-dimensional\ngenomic data. Endocrinology 2014;155(12):4986 – 4999\n40 Fourquet J, Zavala DE, Missmer S, Bracero N, Romaguera J, Flores\nI. Disparities in healthcare services in women with endometri-\nosis with public vs private health insurance. Am J Obstet Gynecol\n2019;221(06):623.e1 – 623.e11\n41 De Graaff AA, D ’ Hooghe TM, Dunselman GA, Dirksen CD, Hum-\nmelshoj L, Simoens SWERF EndoCost Consortium. The signi fi-\ncant effect of endometriosis on physical, mental and social\nwellbeing: results from an international cross-sectional survey.\nHum Reprod 2013;28(10):2677 – 2685\n42 Simoens S, Dunselman G, Dirksen C, et al. The burden of\nendometriosis: costs and quality of life of women with endome-\ntriosis and treated in referral centres. Hum Reprod 2012;27(05):\n1292– 1299\n43 Fourquet J, Gao X, Zavala D, et al. Patients ’ report on how\nendometriosis affects health, work, and daily life. Fertil Steril\n2010;93(07):2424– 2428\n44 Rogers PA, D ’ Hooghe TM, Fazleabas A, et al. Priorities for endo-\nmetriosis research: recommendations from an international\nconsensus workshop. Reprod Sci 2009;16(04):335 – 346\n45 Schliep KC, Mumford SL, Peterson CM, et al. Pain typology and\nincident endometriosis. Hum Reprod 2015;30(10):2427 – 2438\n46 Soliman AM, Coyne KS, Gries KS, Castelli-Haley J, Snabes MC,\nSurrey ES. The effect of endometriosis symptoms on absenteeism\nand presenteeism in the workplace and at home. J Manag Care\nSpec Pharm 2017;23(07):745 – 754\n47 Whiteman MK, Hillis SD, Jamieson DJ, et al. Inpatient hysterec -\ntomy surveillance in the United States, 2000-2004. Am J Obstet\nGynecol 2008;198(01):34.e1 – 34.e7\n48 Health NIo Estimates of funding for various research, condition,\nand disease categories (RCDC). Available at: https://report.nih.gov/\ncategorical_spending.aspx. Published 2019. Accessed March 15,\n2020\n49 Garry R. Is insulin resistance an essential component of PCOS?:\nthe endometriosis syndromes: a clinical classi fication in the\npresence of aetiological confusion and therapeutic anarchy. Hum\nReprod 2004;19(04):760 – 768\n50 Nnoaham KE, Hummelshoj L, Webster PWorld Endometriosis\nResearch Foundation Global Study of Women ’ s Health consor-\ntium. , et al; . Impact of endometriosis on quality of life and work\nproductivity: a multicenter study across ten countries. Fertil\nSteril 2011;96(02):366 – 373\n51 Farland LV, Horne AW. Disparity in endometriosis diag-\nnoses between racial/ethnic groups. BJOG 2019;126(09):1115 –\n1116\n52 Fourquet J, Báez L, Figueroa M, Iriarte RI, Flores I. Quanti fication\nof the impact of endometriosis symptoms on health-related\nquality of life and work productivity. Fertil Steril 2011;96(01):\n107– 112\n53 Chen EH, Shofer FS, Dean AJ, et al. Gender disparity in analgesic\ntreatment of emergency department patients with acute ab-\ndominal pain. Acad Emerg Med 2008;15(05):414 – 418\n54 Leresche L. Defining gender disparities in pain management. Clin\nOrthop Relat Res 2011;469(07):1871 – 1877\n55 Ghiasi M, Kulkarni MT, Missmer SA. Is endometriosis more\ncommon and more severe than it was 30 years ago? J Minim\nInvasive Gynecol 2020;27(02):452 – 461\n56 Budrionis A, Bellika JG. The learning healthcare system: where are\nwe now? A systematic review. J Biomed Inform 2016;64:87– 92\n57 Galvin HK, Petersen C, Subbian V, Solomonides A. Patients as\nagents in behavioral health research and service provision:\nrecommendations to support the learning health system. Appl\nClin Inform 2019;10(05):841 – 848\n58 McKillop M, Mamykina L, Elhadad N. Designing in the dark:\neliciting self-tracking dimensions for understanding enigmatic\ndisease. Proceedings of the 2018 CHI Conference on Human\nFactors in Computing Systems . Available at: http://people.dbmi.\ncolumbia.edu/noemie/papers/18chi.pdf. Accessed 2018\n59 McKillop M, Voigt N, Schnall R, Elhadad N. Exploring self-\ntracking as a participatory research activity among women\nwith endometriosis. J Particip Med 2016;8:e17\n60 Ensari I, Elhadad N. mHealth For Research: Participatory\nResearch Applications to Gain Disease Insights. In Digital Health:\nMobile and Wearable Devices for Participatory Health Applica-\ntions. Eds. Syed-Abdul S, Zhu X, Fernandez-Luque L. Elsevier,\n2020. Forthcoming. ISBN: 978-0128200773\n61 Friedman C, Rubin J, Brown J, et al. Toward a science of learning\nsystems: a research agenda for the high-functioning Learning\nHealth System. J Am Med Inform Assoc 2015;22(01):43 – 50\n62 Hripcsak G, Duke JD, Shah NH, et al. Observational health data\nsciences and informatics (OHDSI): opportunities for observational\nresearchers. Stud Health Technol Inform 2015;216:574– 578\n63 Hripcsak G, Albers DJ. Correlating electronic health record con-\ncepts with healthcare process events. J Am Med Inform Assoc\n2013;20(e2):e311– e318\n64 United States Census Bureau QuickFacts. New York City, New\nYorkUnited States Census Bureau. Available at: https://data.\ncensus.gov/cedsci/all?q¼new%20york%20cit-\ny&g¼1600000US3651000&hidePreview ¼false&tid¼ACSDP1-\nY2018.DP05&vintage¼2018. Accessed 2020\n65 Ta CN, Dumontier M, Hripcsak G, Tatonetti NP, Weng C. Columbia\nOpen Health Data, clinical concept prevalence and co-occurrence\nfrom electronic health records. Sci Data 2018;5:180273 – 180273\n66 Awad E, Ahmed HAH, Yousef A, Abbas R. Ef ficacy of exercise on\npelvic pain and posture associated with endometriosis: within\nsubject design. J Phys Ther Sci 2017;29(12):2112 – 2115\n67 The Observational Medical Outcomes Partnership (OMOP) Com-\nmon Data Model. Available at: https://ohdsi.github.io/Common-\nDataModel/background.html. Accessed 2020\n68 International Health Terminology Standards Development Or-\nganization SNOMED CT Worldwide. Available at: http://www.\nihtsdo.org/snomed-ct/snomed-ct-worldwide. Accessed 2016\n69 Zini EM, Lanzola G, Bossi P, Quaglini S. An environment for\nguideline-based decision support systems for outpatients mon-\nitoring. Methods Inf Med 2017;56(04):283 – 293\n70 Ramakrishnan N, Hanauer D, Keller B. Mining electronic health\nrecords. Computer 2010;(10):77 – 81\n71 McKillop MM. Phenotyping endometriosis from observational\nhealth data. Columbia University . Available at: https://academ-\niccommons.columbia.edu/doi/10.7916/d8-1est-dh56. Accessed\nMarch 18, 2019\n72 Melzack R. The McGill Pain Questionnaire: major properties and\nscoring methods. Pain 1975;1(03):277 – 299\n73 Simborg DW, Star field BH, Horn SD, Yourtee SA. Information\nfactors affecting problem follow-up in ambulatory care. Med\nCare 1976;14(10):848 – 856\n74 Bartley EJ, Robinson ME, Staud R. Pain and fatigue variability\npatterns distinguish subgroups of fibromyalgia patients. J Pain\n2018;19(04):372– 381\n75 Salthouse TA. Implications of within-person variability in cog-\nnitive and neuropsychological functioning for the interpretation\nof change. Neuropsychology 2007;21(04):401 – 411\n76 Urteaga I, McKillop M, Elhadad N. Learning endometriosis\nphenotypes from patient-generated data. NPJ Digit Med 2020;\n3:88\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al. 783\n\n\n77 Wilson EB. Probable inference, the law of succession, and statis-\ntical inference. J Am Stat Assoc 1927;22(158):209 – 212\n78 Newcombe RG. Interval estimation for the difference between\nindependent proportions: comparison of eleven methods. Stat\nMed 1998;17(08):873 – 890\n79 Harris RE, Williams DA, McLean SA, et al. Characterization and\nconsequences of pain variability in individuals with fibromyal-\ngia. Arthritis Rheum 2005;52(11):3670 – 3674\n80 Stoffel MA, Nakagawa S, Schielzeth H. rptR: Repeatability estima-\ntion and variance decomposition by generalized linear mixed ‐\neffects models. Methods Ecol Evol 2017;8(11):1639– 1644\n81 Hox JJ, Moerbeek M, Van de Schoot R. Multilevel Analysis:\nTechniques and Applications. Routledge 2010\n82 Nakagawa S, Schielzeth H. Repeatability for Gaussian and non-\nGaussian data: a practical guide for biologists. Biol Rev Camb\nPhilos Soc 2010;85(04):935 – 956\n83 Huang C -W, Lu R, Iqbal U, et al. A richly interactive exploratory\ndata analysis and visualization tool using electronic medical\nrecords. BMC Med Inform Decis Mak 2015;15:92 – 92\n84 Bougie O, Healey J, Singh SS. Behind the times: revisiting endome-\ntriosis and race. Am J Obstet Gynecol 2019;221(01):35.e1– 35.e5\n85 Seracchioli R, Mabrouk M, Guerrini M, et al. Dyschezia and\nposterior deep in filtrating endometriosis: analysis of 360 cases.\nJ Minim Invasive Gynecol 2008;15(06):695 – 699\n86 Cagle J, Bunting M. Patient reluctance to discuss pain: under-\nstanding stoicism, stigma, and other contributing factors. J Soc\nWork End Life Palliat Care 2017;13(01):27 – 43\n87 Norouzinia R, Aghabarari M, Shiri M, Karimi M, Samami E.\nSamami EJGjohs. Communication barriers perceived by nurses\nand patients. Glob J Health Sci 2015;8(06):65 – 74\n88 Hoffmann DE, Tarzian AJ. The girl who cried pain: a bias against\nwomen in the treatment of pain. J Law Med Ethics 2001;29(01):\n13– 27\n89 As-Sanie S, Black R, Giudice LC, et al. Assessing research gaps and\nunmet needs in endometriosis. Am J Obstet Gynecol 2019;221\n(02):86– 94\n90 Weiskopf NG, Cohen AM, Hannan J, Jarmon T, Dorr DA. Towards\naugmenting structured EHR data: a comparison of manual chart\nreview and patient self-report. AMIA Annu Symp Proc 2020;\n2019:903– 912\n91 Farrar JT, Troxel AB, Haynes K, et al. Effect of variability in the 7-\nday baseline pain diary on the assay sensitivity of neuropathic\npain randomized clinical trials: an ACTTION study. Pain 2014;\n155(08):1622– 1631\n92 Carvalho E, Bettger JP, Goode AP. Insurance coverage, costs, and\nbarriers to care for outpatient musculoskeletal therapy and\nrehabilitation services. N C Med J 2017;78(05):312 – 314\n93 Genes N, Violante S, Cetrangol C, Rogers L, Schadt EE, Chan YY.\nFrom smartphone to EHR: a case report on integrating patient-\ngenerated health data. NPJ Digit Med 2018;1(01):23\n94 Alexander S. mHealth Technologies for the Self-management of\nDiabetes in the Older Population. ACM SIGACCESS Accessibility\nand Computing 2015;(111):14 – 18\n95 Steinhubl SR, Muse ED, Topol EJ. The emerging field of mobile\nhealth. Sci Transl Med 2015;7(283):283rv3\n96 Faurholt-Jepsen M, Munkholm K, Frost M, Bardram JE, Kessing\nLV. Electronic self-monitoring of mood using IT platforms in\nadult patients with bipolar disorder: a systematic review of the\nvalidity and evidence. BMC Psychiatry 2016;16(01):7\n97 Perski O, Blandford A, West R, Michie S. Conceptualising engage-\nment with digital behaviour change interventions: a systematic\nreview using principles from critical interpretive synthesis.\nTransl Behav Med 2017;7(02):254 – 267\n98 Honoré P, Menning PM, Rogers SD, Nichols ML, Mantyh PW.\nNeurochemical plasticity in persistent in flammatory pain. In: .\nProgress in brain research. Vol 129 Elsevier; 2000:357 – 363\n99 Sinatra R. Causes and consequences of inadequate management\nof acute pain. Pain Med 2010;11(12):1859 – 1871\n100 Tasmuth T, Kataja M, Blomqvist C, von Smitten K, Kalso E.\nTreatment-related factors predisposing to chronic pain in\npatients with breast cancer – a multivariate approach. Acta Oncol\n1997;36(06):625– 630\n101 Poleshuck EL, Katz J, Andrus CH, et al. Risk factors for chronic\npain following breast cancer surgery: a prospective study. J Pain\n2006;7(09):626– 634\n102 Fremont A, Lurie N. Eliminating health disparities: measurement\nand data needs. Appendix D, The role of racial and ethnic data\ncollection in eliminating disparities in health care Washington\n(DC): National Academies Press (US); 2004\n103 Birkhead GS, Klompas M, Shah NR. Uses of electronic health\nrecords for public health surveillance to advance public health.\nAnnu Rev Public Health 2015;36(01):345 – 359\n104 Douglas MD, Dawes DE, Holden KB, Mack D. Missed policy\nopportunities to advance health equity by recording demo-\ngraphic data in electronic health records. Am J Public Health\n2015;105(Suppl 3):S380 – S388\n105 Polubriaginof FCG, Ryan P, Salmasian H, et al. Challenges with\nquality of race and ethnicity data in observational databases.\nJ Am Med Inform Assoc 2019;26(8-9):730 – 736\n106 Kiepek W, Sengstack PP. An evaluation of system end-user\nsupport during implementation of an electronic health record\nusing the model for improvement framework. Appl Clin Inform\n2019;10(05):964– 971\n107 Thyvalikakath TP, Duncan WD, Siddiqui ZNational Dental PBRN\nCollaborative Group. , et al; . Leveraging electronic dental record\ndata for clinical research in the national dental PBRN practices.\nAppl Clin Inform 2020;11(02):305 – 314\n108 Charmel PA, Frampton SB. Building the business case for patient-\ncentered care. Healthc Financ Manage 2008;62(03):80 – 85\n109 Chiauzzi E, Rodarte C, DasMahapatra P. Patient-centered activity\nmonitoring in the self-management of chronic health conditions.\nBMC Med 2015;13(01):77\n110 Korotitsch WJ, Nelson-Gray RO. An overview of self-monitoring\nresearch in assessment and treatment. Psychol Assess 1999;11\n(04):415\n111 Chase HS, Radhakrishnan J, Shirazian S, Rao MK, Vawdrey DK.\nUnder-documentation of chronic kidney disease in the electron-\nic health record in outpatients. J Am Med Inform Assoc 2010;17\n(05):588– 594\nApplied Clinical Informatics Vol. 11 No. 5/2020\nSelf-Tracking Data and Clinical Documentation in Endometriosis Ensari et al.784","source_license":"CC0","license_restricted":false}