Refining chronic pain phenotypes: A comparative analysis of sociodemographic and disease-related determinants using electronic health records.

This study was approved by the Institutional Review Board at the Stanford School of Medicine (IRB# 53750). Informed consent was not required, as this study involved the use of de-identified electronic health records (EHR) data, ensuring patient privacy and compliance with ethical standards for research involving human subjects. EHR data fields available for analysis include demographics, healthcare utilization, diagnostic codes, flowsheets, medications, investigations, procedures, encounters, and clinical notes. 9 We accessed clinical EHR via the Stanford Research Repository (STARR) database to identify study participants. We extracted, de-identified, and recorded data from STARR in REDCap, a secure, browser-based software, for analysis by the study team. We restricted the recruitment of the study population to adults who visited any Stanford outpatient rheumatology clinic in 2019. We randomly selected patients with at least two mentions, spaced three or more months apart, using ICD-10 codes for the following rheumatologic conditions: ankylosing spondylitis, psoriatic arthritis, Sjogren’s syndrome, systemic lupus erythematosus, and systemic sclerosis. Pain intensity scores are patient-reported, with scores collected at the clinic visit by a medical assistant who inputs the values into the EHR using the Visual Analog Scale ranging from 0 (no pain) to 10 (worst pain imaginable). To categorize patients as meeting the unimodal chronic pain criterion, we identified patients with two or more pain scores rated at ≥4 points, recorded at least three months apart. We applied a chronic pain diagnosis code phenotyping algorithm to identify patients with chronic pain. We defined a list of ICD-10 codes covering various painful conditions, including abdominal pain, chest pain, joint pain, limb pain, cervicalgia, temporomandibular disorder, chronic low back pain, fibromyalgia, irritable bowel syndrome, urologic chronic pelvic pain syndrome, vulvodynia, endometriosis, migraine, chronic tension-type headache, and myalgic encephalomyelitis/chronic fatigue syndrome. The labeling criteria required the presence of ≥2 ICD codes from the defined list recorded at visits separated by at least 3 months. To identify patients with chronic pain, we applied the pain prescription criterion, which required receipt of ≥2 pain-related prescriptions during 2019, with prescriptions recorded at visits separated by at least 3 months. We extracted pain-related prescriptions, including opioids, non-steroidal anti-inflammatory drugs (NSAIDs), anticonvulsants, antidepressants, and muscle relaxants, using previously established definitions ( Supplementary Table 1 ). 9 We applied the pain intervention phenotyping algorithm to categorize patients as having chronic pain. We classified patients if they underwent any procedures listed in the identified CPT codes during 2019. We identified patients undergoing common Current Procedural Terminology (CPT) codes in pain interventions or treatments through consultations with rheumatologists and pain clinicians. These CPT codes include procedures such as physical therapy, acupuncture, radiofrequency ablation, carpal tunnel injections, and nerve blocks ( Supplementary Table 2 ). The combinations of the structured data fields yielded fifteen unique phenotyping algorithms: unimodal (composed of one structured data element, n=4); bimodal (composed of two data elements, n=6); trimodal (composed of three data elements, n=4); and quadrimodal (composed of four data elements, n=1). During manual chart reviews for each algorithm, we confirmed chronic pain if one of the following criteria was met: (i) Documentation in Clinic Notes: Any indication in the clinic notes by the treating clinician describing the complaint of pain as chronic. (ii) Multiple encounters: two or more encounters, separated by ≥3 months, where the same or similar pain complaint was noted as present and continuous. To ensure accuracy, two clinicians (TB and BV) reviewed the patient’s chart, history, and physical notes for phrases defining pain as constant, recurrent, persistent, or ongoing in the same body area for ≥3 months. A chronic pain classification was assigned if the same pain was consistently documented in visit notes for ≥3 months. In cases where there was disagreement between the two reviewers regarding the presence of chronic pain, a third reviewer (OC) acted as a “tiebreaker.” Exclusions were made for pain attributed to other clinical conditions unrelated to ARDs, such as cancer, fracture-dislocation, and degenerative bone diseases supported by radiological and pathological evidence in the EHR. Demographic information included age (18–40, 41–64 and ≥65 years), sex (male versus female), race (Asian/Pacific Islander, Black, Others, White, and Unknown), ethnicity (Hispanic/Latino, Non-Hispanic, and others), and health insurance (public, private, and others). We examined the baseline characteristics of the sample by assessing sociodemographic factors (age, sex, race, ethnicity, and insurance status) to determine their distribution within each of the five rheumatic diseases: ankylosing spondylitis, psoriatic arthritis, Sjogren’s syndrome, systemic lupus erythematosus, and systemic sclerosis. We calculated the prevalence of each unimodal and multimodal phenotyping algorithm among participants across different sociodemographic factors such as age group, sex, race-ethnicity, health insurance status, and disease status. We then assessed each algorithm component for its predictive ability to accurately identify chronic pain documented in clinic notes. We calculated various probability and accuracy metrics for the completed algorithm, including positive predictive value (PPV), negative predictive value (NPV), sensitivity, specificity, accuracy, and F1-score. PPV and NPV were determined based on the count of true-positive (TP) and true-negative (TN) chronic pain patients identified by the algorithm. We calculated sensitivity as the proportion of TP chronic pain patients identified as positive for chronic pain by the algorithm, while specificity represented the proportion of TN patients among those identified as negative for chronic pain. F1-score was calculated as the harmonic mean of precision (PPV) and recall (sensitivity), which is expressed as two-times the product of precision and recall and divided by the sum of the precision and recall. The project has been uploaded to the Open Science Framework. 6

802 patients meeting the inclusion criteria were selected; 79% were female ( Table 1 ). The median age was 61.0 years; most patients were aged 41–64 years (45%). Most patients were White (65%) and non-Hispanic (82%). Approximately half (52%) of patients were publicly insured. More than a quarter (28%) of patients were being managed for Sjogren’s syndrome. There were no major differences in demographic distribution by sex. However, half of male patients were receiving treatment for psoriatic arthritis and another 29% were being managed for ankylosing spondylitis, while the most prevalent (34%) condition in females was Sjogren’s syndrome, followed by systemic sclerosis (23%). The evaluation of various phenotyping algorithms based on different combinations of clinical data sources revealed significant insights into the trade-offs between sensitivity, specificity, PPV and NPV ( Table 2 ). The sensitivity of the algorithms varied from ~18% to ~67%, while specificity ranged from ~55% to ~96%. Notably, combining multiple structured data elements generally enhanced specificity and PPV, but often at the expense of sensitivity. For example, the quadrimodal algorithm that combined pain scores, ICD codes, prescriptions, and interventions (#15) achieved the highest specificity (96%) and PPV (86%), indicating a high true positive rate and a low false positive rate, making it particularly useful for applications where minimizing false positives is crucial. On the other hand, the highest sensitivity was observed with the use of ICD codes alone, #2 (67%), suggesting that ICD codes may be more effective at identifying true positive cases compared to other algorithms. This highlights the importance of ICD codes in contexts where capturing all potential cases is more critical than excluding false positives. The combination of pain scores and ICD (#5) codes also performed well, with a PPV of 80% and specificity of 86%, providing a good balance for situations requiring high precision and reasonable recall. F1-scores were highest for pain scores alone (67), indicating that pain scores are a strong single-source option for phenotyping. However, combinations of data sources generally resulted in better overall performance metrics, suggesting that integrating multiple types of data can lead to more reliable and comprehensive phenotyping. For example, the combination of pain scores, ICD Codes, and prescriptions (#11) yielded an F1-score of 0.36, despite having lower sensitivity, showcasing the robustness of multi-source algorithms in specific contexts. PPV was generally have higher in females, with values like 89% for “Pain scores + ICD codes + Prescriptions + Interventions, #15” versus 71% for males ( Figure 1 ). Individuals aged 18–40 had the highest PPV, with values such as 100% for “Pain scores + Prescriptions + Interventions, #13”. Racial differences reveal that PPV was lower in Asian/PI patients, and was higher among Black patients (e.g., 100% for “Prescriptions + Intervention #10.”) Ethnicity-based PPV data showed that Hispanic individuals generally have higher values compared to non-Hispanics. Insurance status also influenced PPV, with private insurance holders demonstrating the highest values. Disease-specific PPV variations highlight that conditions such as ankylosing spondylitis and psoriatic arthritis have high PPV values, while Sjogren’s syndrome shows particularly high PPV (95% for “Pain scores + ICD codes + Prescriptions + Interventions, #15”). However, males, Asian/PI, older patients, those without public/private insurance, patients with systemic sclerosis had the lowest PPV. In contrast, we observed generally low NPV across subgroups. Males generally exhibited higher NPV, such as 73% for “Pain scores, #1” compared to females’ 55% ( Figure 2 ). Age-based differences highlight that individuals aged 41–64 have notably higher NPVs, such as 73% for “ICD codes + Prescriptions + Interventions, #14” compared to younger and older age groups. Race/ethnic differences show that Asian/PI individuals had higher NPVs, while Black and Hispanic individuals had lower values. Insurance status also impacted NPV, with private insurance holders showing higher values. NPV was high among patients with ankylosing spondylitis and systemic sclerosis, reflecting effective predictive capabilities, while systemic lupus erythematosus shows mixed results. We observed low sensitivity across subgroups. We observed generally higher sensitivity in females compared to males ( Figure 3 ). For example, using pain scores (#1), the sensitivity for females is 69% versus 55% for males. Age-based sensitivity peaks in individuals aged 41–64, particularly with ICD codes (#2) at 73% and combined algorithms involving pain scores and ICD codes (#5) at 52%. In contrast, those aged 18–40 and 65+ showed generally lower sensitivities, suggesting potential areas for algorithm adjustment. Racial disparities were also evident as sensitivity was highest for Black patients for pain scores (#1) at 77% and Asian/PI exhibited the lowest (48%). In addition, sensitivity was highest for ICD codes (#2) for White and Asian/PI compared to other algorithms. The sensitivity of algorithms was generally higher in Hispanic individuals (75%) than in non-Hispanics (61%). Insurance status also impacted sensitivity, as private insurance holders showed the highest values, especially with ICD codes (87%). Disease-specific analysis revealed that patients with systemic sclerosis generally had higher sensitivity (69% for pain scores), while sensitivity was lower in patients with psoriatic arthritis and Sjogren’s syndrome. In contrast, males tended to have higher specificity values compared to females, such as 80% versus 63% for “Pain scores” ( Figure 4 ). Specificity also varied by age, with individuals aged 18–40 years exhibiting high values (e.g., 98% for “Pain scores + ICD codes + Interventions”, #12). Specificity was generally highest in Asian/PI individuals in general, while specificity was also exceptionally high in Black patients for certain combinations (e.g., 100% for “Pain scores + ICD codes + Interventions”, #12). Specificity was lower in Hispanic patients compared to non-Hispanics. Insurance status also appeared to influence specificity, with private insurance holders demonstrating the highest values (e.g., 98% for “Pain scores + ICD codes + Interventions”, #12). Specificity was high in ankylosing spondylitis (98% for “Pain scores + ICD codes + Prescriptions + Interventions”, #15), while conditions such as psoriatic arthritis and systemic lupus erythematosus exhibited more varied results. F1-scores in females were higher than in males across all pain algorithms, indicating better performance in identifying true positives and minimizing false positives ( Figure 5 ). For example, with pain scores (#1), the F1-score for females was 0.69 versus 0.59 for males. Racial differences are considerable as F1-scores in Black patients (0.72) were higher than Asian or Pacific Islanders (0.53), suggesting variability in algorithm performance across racial groups. F1-scores were also higher among Hispanic or Latino patients (0.73) than non-Hispanics (0.65), indicating more reliable phenotyping for this group. Individuals aged 18–40 tended to have the highest F1-scores, especially with algorithms combining multiple data sources. For instance, the F1-score for this age group with ICD codes (#2) was 0.74, while it is lower for those aged 65+ years (0.62). Insurance status revealed that private insurance holders had higher F1-scores (0.77) compared to those with public insurance (0.68), reflecting differences in data quality or availability. Patients with Sjogren’s Syndrome had the highest F1-scores (0.75), whereas those with psoriatic arthritis generally had lower F1-scores (0.55), indicating a need for improvements in algorithms for this condition.

Our study found substantial heterogeneity in several metrics between and within chronic pain phenotyping algorithms. PPV improved as more structured data elements were included. For example, the quadrimodal phenotyping algorithm had a PPV of 86%, while the unimodal phenotyping algorithm that included only prescriptions had a PPV of 58%. Specificity was also highest in the quadrimodal phenotyping algorithm. We also observed generally low sensitivities, with the ICD code-only phenotyping algorithm showing the highest sensitivity at 67%. The optimal case definitions identified in this study were two unimodal algorithms that used either pain scores or ICD codes, demonstrating moderate sensitivity levels of 60% and 67%, respectively, and similarly modest specificity levels of 67% and 59%. These generally modest values suggest that while these data elements can contribute to identifying chronic pain, they may not be sufficient on their own and require further refinement or complementary data sources for more accurate phenotyping. It is important to note that pain scores are usually recorded in EHRs, while administrative claims data may not include this information. The use of EHR for chronic pain phenotyping has gained significant attention in recent years, with various algorithms being developed to enhance accuracy. 9 , 11 , 14 , 16 , 18 A notable example is the study conducted by Tian et al . in 2013, which set a precedent for evaluating chronic pain phenotyping algorithms using structured data elements using a multisite community health center EHR dataset 16 . Similar to our study, the Tian et al . study used manual chart review by clinicians as the gold standard for validating alternative chronic pain phenotyping algorithms 16 . The authors initially evaluated the performance of several unimodal, bimodal and trimodal data elements combining pain scores, ICD-9 codes and opioid medications. The unimodal phenotyping algorithm with pain scores had a PPV of 72% (the lowest), similar to what we observed in our study. In the Tian study, the highest PPVs were observed in bimodal and trimodal algorithms. Specifically, the “ICD-9 + Opioid” combination yielded a PPV of 96%, and the “Pain score + Opioid + ICD-9” combination yielded a PPV of 98%. In our study, the PPVs of similarly constructed algorithms were 76% and 84%, respectively. Our study’s lower PPVs compared to the Tian study could be due to key differences in study design and patient populations. For example, the patient population in the study by Tian et al., drawn from primary care settings, may have a lower prevalence of common chronic pain conditions. Additionally, we included a broader range of medications—such as opioids, anticonvulsants, and antidepressants—now commonly used off-label for pain management. 7 These factors, along with evolving clinical practices, likely contributed to the variations in algorithm performance. We found that males, older patients, and those identifying as Asian/Pacific Islander had the lowest PPV and sensitivity. This phenomenon is not unique to our study. Younger patients (ages 17–54), patients of color (Asian, Hispanic, Black, multi-racial, Middle Eastern, Hawaiian), male patients, and those on government insurance are classified as “data-lacking” groups. 10 This classification is based on the expectation that their medical records may contain more missing data due to potentially reduced access to care or less frequent healthcare utilization. Conversely, older patients (ages 55+), female patients, those with private insurance, and White patients are categorized as “data-rich” groups, as these individuals are likely to have greater access to healthcare or seek care more frequently. 10 In our study, we hypothesized that differences in PPV across subgroups might be influenced by variations in the prevalence of chronic pain and the likelihood of chronic pain documentation associated with sociodemographic and disease-related factors. However, our dataset did not capture detailed information on documentation practices specific to each subgroup, limiting our ability to assess this directly. Future research should consider investigating subgroup-specific documentation patterns to better understand their impact on phenotyping algorithm accuracy and to identify potential areas for improving documentation consistency across diverse patient populations. PPV is influenced by the prevalence of a condition in the population; thus, groups with higher documented rates of chronic pain or more frequent healthcare interactions might exhibit higher PPV. 2 , 3 Women are more likely to experience chronic pain conditions (e.g., fibromyalgia, migraine and osteoarthritis) compared to men. 1 Private insurance holders might have higher reported prevalence of chronic pain conditions due to greater access to healthcare services and diagnostic testing. This access can lead to more diagnoses of chronic pain conditions compared to those with limited insurance coverage. 10 While older adults do experience significant chronic pain 13 , the PPV might be lower in this group due to differences in healthcare access or variations in how pain is reported or managed among older populations. In addition, Sjogren’s syndrome is associated with high prevalence of chronic pain, particularly due to joint pain, muscle pain, and dryness-related discomfort. 8 , 17 On the other hand, sensitivity and specificity reflect the phenotyping algorithm’s ability to correctly identify true cases and non-cases, which may be less affected by these subgroup characteristics. The consistent sensitivity across subgroups suggests that the phenotyping algorithms performed uniformly in identifying chronic pain across different populations, while the differences in PPV indicate that the accuracy of the positive predictions varied depending on the specific characteristics of the subgroups. An important consideration is the potential application of the validated computable phenotype in recruiting participants for clinical trials, particularly pragmatic clinical trials. Leveraging this phenotype could allow for more efficient, data-driven identification of eligible participants based on chronic pain criteria across diverse patient populations. This capability is especially valuable for large-scale studies where traditional recruitment may be challenging, offering a scalable solution that enhances both accuracy and inclusivity in clinical research. Standardizing documentation of chronic pain across structured fields would ensure more consistent data capture, particularly regarding pain intensity, duration, and management strategies. Additionally, implementing standardized prompts for clinicians to document specific pain characteristics could improve data completeness and reliability. Furthermore, there is significant opportunity to employ natural language processing (NLP) techniques to enhance the chronic pain data obtained from unstructured EHR fields. By extracting nuanced pain-related information from clinical notes, NLP can complement structured data, enabling more comprehensive phenotyping that captures the complexity of patient experiences. Our study has limitations. First, the reliance on EHR data, which may be incomplete or inaccurate, could impact the validity of the phenotyping algorithms. Second, using clinic notes as the gold standard for comparing information recording with structured EHR data presents challenges due to their inherent subjectivity and variability in documentation practices. Third, we could not determine the specific indications for pain medications, such as whether they were prescribed solely for pain management or for other conditions like depression. This limitation could affect the accuracy of categorizing patients with chronic pain based on our extracted prescription data. Future research should consider incorporating additional data sources or methods to clarify the indications for these medications to enhance the specificity of pain-related assessments. Fourth, we included patients from a single academic center, focusing on a specific population with ARD. Although 25% of our cohort was diagnosed with Sjogren’s disease, this distribution does not indicate a specific specialty focus of the center. 9 Instead, it may reflect natural variation in patient diagnoses within our sample. This specific composition of our cohort may limit the generalizability of our findings to broader rheumatology populations, particularly those with more prevalent rheumatic diseases such as rheumatoid arthritis or osteoarthritis. Fifth, we also included acupuncture or physical therapy as indicators of chronic pain without requiring these interventions to occur at two separate time points. Although these treatments are often employed in chronic pain management, they are also used for acute pain care. This overlap could introduce misclassification and affect the accuracy of our phenotyping approach. Incorporating a two-time-point criterion for these indicators could improve the consistency and validity of the classification. Future analyses should address this adjustment to enhance methodological rigor and alignment across chronic pain criteria. Additionally, while we identified patients with chronic pain within this cohort, we did not provide a detailed breakdown of chronic pain prevalence across each of the five rheumatic diseases studied. Future studies should consider sampling from multiple clinics to better capture the full spectrum of rheumatic disease populations and enhance generalizability. Our study has several strengths. First, to our knowledge, this was the first evaluation of heterogeneity in sociodemographic and disease-related correlates of agreement between structured data elements and clinical notes in the documentation of chronic pain. This granular approach helps identify and address potential biases and variations in algorithm performance across diverse patient populations. Second, the development and evaluation of fifteen distinct phenotyping algorithms offer a nuanced assessment of chronic pain identification, enhancing the reliability and comprehensiveness of the findings. Third, using manual chart reviews by two experienced clinicians as the reference standard for validating the phenotyping algorithms provides high accuracy and credibility to the study’s findings, ensuring that the chronic pain classification is based on detailed clinical judgments. In summary, while EHR-based phenotyping holds significant potential for identifying chronic pain phenotypes and could improve clinical and research applications in the future, it is not yet widely implemented within EHR systems. Our study contributes to the exploration of this approach, supporting its potential application as a reliable method for identifying chronic pain in diverse patient populations. Further development and validation are needed before phenotyping algorithms are integrated into routine EHR use. Future research should focus on integrating broader data sources, including novel classification systems like ICD-11, 12 to advance the precision and utility of EHR-based phenotyping. Ultimately, improving these algorithms will enhance real-world chronic pain surveillance and contribute to more personalized, effective care across healthcare settings.

Chronic pain is a pervasive and complex health condition affecting millions of individuals worldwide, with significant implications for quality of life, healthcare utilization, and economic burden. 15 As the prevalence of chronic pain continues to rise, effective management and documentation of patient information are crucial for optimizing clinical care and outcomes. 5 , 19 , 20 In chronic pain management, electronic health records (EHR) are important for documenting patient encounters, treatment modalities, and outcomes. 9 Structured data fields (e.g., pain intensity, treatment modalities, diagnosis codes, and interventions) offer standardized templates for capturing specific chronic pain phenotypes. 9 We recently characterized the differences in chronic pain case definitions in patients with autoimmune rheumatic diseases (ARD) such as systemic lupus erythematosus, Sjogren’s syndrome, and psoriatic arthritis. 4 , 9 We compared the prevalence of chronic pain using various phenotyping algorithms derived from structured data fields, including pain scores, diagnostic codes, analgesic medications, and pain interventions. We found significant variability in chronic pain prevalence estimates based on these different definitions, highlighting potential biases and misclassifications in EHR. There is a need to examine heterogeneity in measures of phenotyping algorithms and potential variability in accuracy across different subgroups for a few reasons. First, by evaluating algorithm performance across diverse groups, researchers can identify and correct algorithmically-maintained biases, thus ensuring that all patients receive equitable care. Second, identifying heterogeneity in subgroups may also reveal how patients with the greatest pain care needs experience biases that impede access to appropriate care. Algorithms validated across diverse subgroups are more likely to be ecologically valid, robust and generalizable, thus leading to better overall performance and reliability. Third, ensuring that algorithms perform well across subgroups can increase clinicians’ trust in these tools, thus leading to better integration into clinical practice and improved patient outcomes. Finally, by highlighting where algorithms underperform, researchers can focus on developing more inclusive models and explore factors contributing to algorithm performance disparities and downstream disparities in care. This study aims to determine which chronic pain case definitions derived from structured data elements achieve the best accuracy, and how these validation metrics vary by sociodemographic and disease-related factors. This inquiry will ensure that the strengths and limitations of these case definitions are well understood and appropriately addressed in their applications in other real-world data. We hypothesize that factors such as sex, age, disease status, race, and insurance status may influence documentation practices and contribute to variability in the consistency of information recording within EHRs.