{"paper_id":"4be8fa5a-2ac4-4b35-bf00-3557381ac736","body_text":"Enhancing and Personalising Endometriosis Care\nwith Causal Machine Learning ⋆\nAriane Hine1[0000−0002−7447−9371], Thais Webber2[0000−0002−8091−6021], and\nJuliana Bowles1,3[0000−0002−5918−9114]\n1 School of Computer Science, University of St Andrews, KY16 9SX St Andrews, UK\n{aah4,jkfb}@st-andrews.ac.uk\n2 School of Computer Science and Digital Tech., Aston University, B4 7ET, UK\nt.webber@aston.ac.uk\n3 Software Competence Centre Hagenberg, Softwarepark, 4232 Hagenberg, Austria\njuliana.bowles@scch.at\nAbstract. Endometriosis poses significant challenges in diagnosis and\nmanagement due to the wide range of varied symptoms and systemic\nimplications. Integrating machine learning into healthcare screening pro-\ncesses can significantly enhance and optimise resource allocation and\ndiagnostic efficiency, and facilitate more tailored and personalised treat-\nment plans. This paper discusses the potential of leveraging patient-\nreported symptom data through causal machine learning to advance en-\ndometriosis care and reduce thelengthy diagnostic delays associated with\nthis condition. The goal is to propose a novel personalised non-invasive\ndiagnostic approach that understands the underlying causes of patient\nsymptoms and combines health records and other factors to enhance\nprediction accuracy, providing an approach that can be utilised globally.\nKeywords: Female reproductive health· Endometriosis · Artificial In-\ntelligence · Prediction models · Diagnosis · Menstrual health\n1 Introduction\nEndometriosis is a prevalent, chronic, inflammatory condition affecting approx-\nimately 10% of individuals assigned female at birth during their reproductive\nyears and beyond. Historically characterised as a gynaecological disease [7], re-\ncent research has revealed its systemic implications [16]. Endometriosis can affect\nvirtually all organs in the human body, with symptoms ranging from asymp-\ntomatic cases to severe, life-altering conditions [14,26].\nCommon symptoms include menstrual irregularities, heavy menstrual flow\n(menorrhagia), painful periods (dysmenorrhea), pain during sexual intercourse\n(dyspareunia), chronic pelvic pain unrelated to menstruation, tenderness, ad-\nnexal mass (growths near the uterus/ovaries), infertility or subfertility, depres-\nsion, anxiety, abdominal bloating, nausea, and restricted mobility [10,26]. In\n⋆ BowlesispartiallysupportedbytheAustrianFundingCouncil(FWF)underMeitner\nM 3338-N.\n\nA. Hine et al.\nmore advanced cases, patients often experience bowel and bladder-related symp-\ntoms such as painful bowel movements (dyschezia), loss of bladder control (dy-\nsuria), blood in the urine or stool during menstruation, painful urination, and\nchronic fatigue [24]. Given the multitude of symptoms, overlapping conditions,\nand the complexity of the disease, the root cause of endometriosis has not yet\nbeen conclusively determined [1,10]. It has been hypothesised that one cause\nmay be retrograde menstruation, where menstrual blood flows backwards into\nthe pelvis during menstruation. Some of the menstrual blood contains endome-\ntrial tissue, which is believed to implant within a woman’s abdomen, leading to\npatches of endometriosis [18]. This is not the sole cause of the condition, however,\nas 90% of all menstruating women are thought to experience this phenomenon.\nOn the contrary, other sources suggest that endometriosis may start at birth,\nwith symptoms not triggering until puberty [23]. There are many varying theo-\nries on the source of the condition, with no one definitive cause. The aetiology\nof endometriosis remains medically undetermined, complicated by its complex\nmultifactorial nature. This complexity likely contributes to the challenges in un-\nderstanding and diagnosing the condition, as multiple variables seem to influence\nits development and progression in patients [4,15].\nFurther complexity arises when patients with endometriosis, whether sus-\npected or diagnosed, have comorbidities that contribute to pelvic-related symp-\ntoms. These additional conditions must be considered within a comprehensive\ndiagnostic algorithm. Common comorbidities include Irritable Bowel Syndrome\n(IBS), Interstitial Cystitis, and Adenomyosis [22], all of which present with simi-\nlar pelvic symptoms. Their presence can complicate the clinical scenario, making\naccurate diagnosis more challenging and often resulting in significant delays in\nidentifying the condition and starting appropriate treatment and management.\nThis complexity emphasises the need for thorough diagnostic protocols and tech-\nnological support for clinical decisions that account for conditions with overlap-\nping symptoms. Integrating such considerations into diagnostic tools is crucial\nfor enhancing diagnostic accuracy, supporting and ensuring that the diagnostic\nprocess reflects the multifaceted approach taken by clinicians in practice.\nWe believe that leveraging multiple diverse sources of health data (including\npatient-reported data) and cutting-edge machine learning (ML) techniques for\nanalysis can reduce diagnostic delays by accurately identifying the underlying\ncauses of patient symptoms [11]. Causal machine learning, which focuses on\nunderstanding and identifying the causal relationships between variables rather\nthan just correlations, holds particular promise in this area. By applying causal\nML techniques, we can move beyond traditional correlation-based models to\nuncover the underlying mechanisms of endometriosis. This can lead to more\naccurate diagnostic tools that not only predict the likelihood of the disease but\nalso provide insights into the potential causes of patient symptoms.\nThe goal of the EndoML Project is to develop a novel personalised non-\ninvasive diagnostic approach that utilises diverse health data, including patient-\nreported information, and applies advanced ML techniques to reduce diagnostic\ndelays. By understanding the underlying causes of patient symptoms, combining\n\nEndometriosis Care with Causal Machine Learning\nhealth records with other relevant factors, and enhancing prediction accuracy, we\naim to create a globally applicable diagnostic tool. Even if the EndoML Project\nhas been specifically developed for targeting endometriosis, and the data col-\nlected focuses on that, the underlying (causal) machine learning approach can\npotentially shed light on more generic solutions to address the diagnosis of other\nconditions. Overall, such approaches can optimise healthcare resource allocation,\nfor example, reducing unnecessary tests and procedures, improving the diagnos-\ntic process through timely and accurate identification of the condition (in this\ncase, endometriosis), and facilitating the creation of shareable computational\nmethods for personalised treatment.\nThis paper is structured as follows. Section 2 provides the context of our\nwork and the need for digital solutions to address diagnostic delays in general,\nand endometriosis in particular. We discuss related work and research lever-\naging AI/ML techniques, including the shortcomings of existing attempts for\nalgorithmic-based diagnosis of endometriosis. Section 3 presents the EndoML\nProject and the technical details underlying the proposed causal ML model,\nfocusing on the prediction flow and introduction to the wide data collection\nthrough a comprehensive survey. Section 4 presents data collection approach and\nfindings from initial data analysis, including data preprocessing and descriptive\nstatistics. Finally, Section 5 concludes the paper with our aims for future work.\n2 Endometriosis Care and Diagnosis Delays\nOne of the most challenging aspects clinicians face when seeing patients is to\nensure the correct diagnosis of their conditions. This can be challenging when\nsymptomatic manifestations can vary considerably between individuals, partic-\nularly in complex conditions such as endometriosis [1]. There are complex care\npathways and diagnosis scenarios for endometriosis with delays evident from\nboth patients’ and clinicians’ viewpoints (refer to Fig. 1 and Table 1). It is im-\nportant to acknowledge that the list of delays is not exhaustive and is subject\nto further investigation [1,11].\nFrom clinicians’ viewpoint, the complex nature of endometriosis, presenting\nvaried symptoms and potential overlap with other conditions, brings challenges\nsuch as the lack of effective guidelines and integrated holistic health history. For\ninstance, in this context, the average time to diagnosis in the UK is reported\nto range from 4 to 11 years [1]. From patients’ viewpoint [1], for some individ-\nuals, endometriosis is accompanied by a barrage of symptoms that significantly\nimpair their quality of life on a daily basis, including pain (through a variety\nof manifestations), and often infertility. Other patients may exhibit few to no\nsymptoms,andasaresult,diagnosismayoccurunexpectedlyduringasecondary,\nunrelated medical procedure, leading to further tests to better understand the\nstate of the patient’s endometriosis. Some asymptomatic individuals are likely\nto remain undiagnosed but the same holds true for symptomatic sufferers, given\nthe intricacies involved in the diagnostic process and potential misinterpretation\n\nA. Hine et al.\nFig.1: Complex scenario of delays in endometriosis care (refer to Table 1).\nPatient viewpoint\np1 Lack of awareness\np2 Normalisation of symptoms\np3 (Limited) Access to care\np4 Fear of judgement/being a burden\np5 Downplaying symptoms\np6 Difficulty recalling & communicating symptom history\np7 Hormonal birth control masking symptoms\np8 Long waiting list times for referrals from Primary Care\np9 Reluctance to have invasive procedures performed\np10 Long waiting list times in Secondary and Tertiary Care (Surgical interventions)\nClinician viewpoint\nc1 Lack of guidelines & holistic health history\nc2 Poor recognition of symptoms\nc3 Normalisation of symptoms\nc4 Somatisation of symptoms\nc5 Patient has non-specific symptom profile\nc6 Need to test for differential conditions\nc7 Poor collaboration between healthcare providers\nc8 Non-diagnostic imaging tests\nc9 First-line management of symptoms vs. diagnostic testing\nc10 Other factors masking symptoms\nc11 Reluctance to perform invasive procedures\nTable 1: Contrasting viewpoints on diagnostic delays (in Fig. 1).\n\nEndometriosis Care with Causal Machine Learning\nof clinical signs (refer to the clinicians’ viewpoint misdiagnoses occurrences in\nFig. 1).\nIn summary, the significant diagnosis delay is not solely due to clinical com-\nplexities but also stems from broader systemic challenges within healthcare de-\nlivery and patient awareness [6]. Both clinician and patient sources of delays\nare subject to societal-induced biases stemming from the historical minimisa-\ntion, normalisation, and somatisation of sufferers’ pain [1]; whether this delay\ndirectly or indirectly influences the outcome. Both are also impacted by the re-\nluctance to perform invasive procedures, resulting in a longer waiting time for\nlaparoscopic (gold standard) diagnosis.\nLaparoscopic surgery is invasive, costly, and presents risks of potential dam-\nage to internal organs during surgical exploration. This is probably why, from\nthe onset of symptoms and their progressive presentation, clinicians traditionally\nstart the diagnosis process with physical examinations, periodic symptom track-\ning and management, followed by imaging modalities, tests (or assessments) for\ndifferential conditions, but mostly considering first-line management iterations\nmodified over time based on individual’s response or circumstance to suppress or\nalleviate the symptoms [22] (refer to management interventions illustration on\nFig. 1). Moreover, even laparoscopic surgery has the potential for false negative\nresults as illustrated on the right side of Fig. 1, culminating in more cycles of\nmanagement interventions and further potential diagnostic surgeries. This risk is\nparticularly high if surgery is performed by a general surgeon or a non-specialist\nin endometriosis. Such inaccuracies can lead to further patient interventions and\nmore costly actions, as the underlying condition remains unidentified and inad-\nequately managed (or insufficiently treated).\nThere is hence a need for digital solutions, algorithms, and tools that address\nthe burden surrounding diagnostic difficulties, can support and reduce diagnos-\ntic delays and improve the quality of life for both confirmed and suspected en-\ndometriosis sufferers, especially those who are severely affected. Ultimately, these\nsolutionswouldbehighlybeneficialwhenintegratedintoprimary,secondary,and\ntertiary healthcare settings, potentially avoiding risky and costly invasive pro-\ncedures [23,27]. A personalised non-invasive diagnostic approach can become an\nessential tool to effectively triage patients for referrals to tertiary care, such as\ngynaecology, ensuring that those in need of specialised care are identified and\ntreated more promptly.\n2.1 Machine learning in endometriosis diagnosis and treatment\nMachine learning (ML) has the potential to transform traditional approaches\nin healthcare diagnosis and treatment [27]. However, the integration of ML in\nhealthcare remains challenging due to stringent requirements for data quality,\nprivacy, and regulatory compliance [17]. As in any critical domain, developing\nhealthcare AI/ML tools requires meticulous attention to detail and adherence\nto rigorous standards. In the last decade, research on algorithmic detection and\ntreatment of endometriosis has gained attention due to the growing awareness\n\nA. Hine et al.\nand understanding of the condition among healthcare professionals, the general\npublic, and increased patient advocacy [3,8].\nML is set and suitable to optimise and increase efficiency in many areas of\nmedical diagnostics and treatment, especially for conditions with complex or un-\nknown etiologies such as endometriosis [23]. These areas range from improving\nthe core understanding of a condition’s manifestations, progression, and patient\nphenotypes to enhancing research and development towards improving processes\nand guidelines used by clinicians in their daily practice. This continuous improve-\nment is supported by a feedback loop inherent in ML applications; the more data\nand results these systems integrate and analyse, the more refined and effective\nthey increasingly become. This cyclical process of learning and adapting helps\ndrive advancements in both theoretical knowledge and practical applications,\neffectively closing the loop of improvement. Fig. 2 lists our understanding of the\nvaried focus and impact of AI/ML tools throughout the years in the context of\nendometriosis such as topics mentioned in [1,11,22,25,27], among other sources.\nFig.2: Focus and impact of AI/ML algorithms and tools for endometriosis.\nItisessentialtoremarkthereareafew relevantpoints relatedtotheadequate\nchoice of methods and approaches that focus on symptom information and a\n\nEndometriosis Care with Causal Machine Learning\nbetter understanding of endometriosis. A common theme within the published\nresearch on endometriosis diagnosis is the use of ML methods such as Regression\nanalysis [5,8], which look for data correlations without understanding deeper\ncausal relationships between variables. Some approaches can have significant\ndrawbacks, for instance, limited or inappropriate cohort selection (which can\nresult in skewed outcomes and limited generalisability) and selection bias (due\nto strict inclusion/exclusion criteria, which can result in under-representation of\ncertain subsets), failing to capture the disease’s full heterogeneity [3,5,8]. These\nexamples of limitations underscore the need for more sophisticated techniques\nthat can handle diverse and complex data while providing deeper insights into\ncausal mechanisms [19,21].\nIn summary, capturing the full diversity of the endometriosis population is\nessential for developing accurate diagnostic tools that can be applied globally. By\nincluding a more representative sample of patients, these tools can better reflect\nthe variety of symptoms and disease manifestations, leading to more precise and\neffective diagnostics and treatments. This comprehensive approach is relevant\nfor improving patient outcomes and ensuring that diagnostic tools are effective\nacross different demographics and healthcare settings [1].\nWhile statistical prediction models for diagnosis offer considerable potential\n[20], it is crucial to advance our understanding of endometriosis and its nuanced\nbehaviour through more advanced algorithmic techniques; moving beyond mere\nanalysis of simple correlations [11]. In contrast to traditional ML methods (e.g.,\nDecision trees, Gradient boosting, and AdaBoost) commonly applied in the lit-\nerature [13,20], causality-based techniques [19,21] aim to uncover the diseases\nthat cause symptoms, not just the statistical relationships to symptoms, which\nmay be erroneous or unable to capture the complex existing causal relationships.\nCausal ML specifically seeks to model and test hypothetical interventions,\ndetermining the direct effect of one variable over another through do-calculus\nperformed on causal Bayesian networks [19]. This allows researchers to experi-\nment with “what-if\" scenarios, with insights on how altering certain factors could\nhypothetically impact the presence and severity of symptoms [11]. In a diagnos-\ntic context, the modelled scenario would be as follows: ‘what if a patient did not\nhave endometriosis; would their symptoms still be expected to persist?’. This\nshift towards prioritising causality over correlation is an important advancement\nin endometriosis research, where multiple overlapping symptoms and factors\ncomplicate diagnosis and treatment. Consequently, by distinguishing between\ncorrelation and causation, causal ML can help identify risk factors and imple-\nment effective early interventions, rather than merely associating symptoms with\nthe disease. One predictor approach could prioritise interventions that identify\nwhich symptoms would persist if the condition were absent in a patient, hence\naccurately determining which symptoms are directly caused by endometriosis.\nAdditionally, this approach holds promise to increase model accuracy and offer\nmore targeted, effective diagnostic interventions.\n\nA. Hine et al.\n3 The EndoML Project and Data Collection\nIncorporating algorithmic tools to assist clinicians in diagnosing and prioritising\ntreatment based on symptom profiles can decrease the number of supplementary\ninvestigative appointments and diagnostic tests, thereby alleviating diagnostic\ndelays [1,4,9,18,27]. A significant portion of the delay in diagnosing endometrio-\nsis stems from healthcare professionals’ uncertainty about a definitive diagnosis,\nwhich is particularly concerning given the condition’s progressive nature [2].\nThe EndoML Project aims to minimise the need for invasive diagnostic pro-\ncedures, such as laparoscopic surgery, blending patient-generated data from mul-\ntiple sources and (causal) machine learning (ML) techniques. In addition, estab-\nlishing the most likely underlying cause of a patient’s symptoms can increase\nclinicians’ confidence in promptly referring patients to specialised care teams,\npromoting efficient healthcare provisioning and optimised resource allocation.\nThere is a noticeable gap in clinical workflows that ML tools could strengthen to\nenhance diagnostic efficiency, for instance, by automating patient data analysis,\nrecognising patterns and abnormalities in patient data, and providing informed\nclinical decision support.\nThe ultimate goal is to develop an ML tool that is easily interpretable, and\nadaptable to new and evolving knowledge, providing comprehensive insights into\nvariables and their relationships, rather than solely focusing on correlations. The\npremise is that causal ML techniques can result in more accurate diagnostic tools\nthat can also provide valuable insights into the potential causes of a patient’s\nsymptoms [19,21]. In practice, causal ML modelling involves key steps:\n– Data collection:patient data must be collected, including variables related\nto the condition of interest, potential causal and risk factors, and treatment\noutcomes. The data collection phase should be comprehensive, aiming to in-\ntegrate diverse and detailed patient information. It is important to balance\ndata input from clinical assessments with self-reported patient data to cap-\nture a full picture of the condition’s impact. This dual approach facilitates\na deeper understanding of the condition from both medical and patient-\noriented perspectives. Fig. 3 depicts the aspects related to data collection,\nwith both clinician data, patient self-reported data, and their potential over-\nlap. It stresses the characteristics, benefits, challenges and limitations related\nto different data collection perspectives. This overview presents the various\ndata collection methods, illustrating examples of clinician data and poten-\ntial patient self-reports, along with the common variables each source offers.\nIt also highlights a few key advantages and disadvantages of each method,\nproviding insights to consider when selecting data sources.\n– Graph construction:Directed Acyclic Graph (DAG) construction, where\npotential causal variables are identified based on the collected data, domain\nknowledge and prior research. This step is crucial as it lays the groundwork\nfor understanding the underlying mechanisms of the condition. This step\nalso involves the integration of expert knowledge into the DAG structure to\n\nEndometriosis Care with Causal Machine Learning\nFig.3: Comparing clinical and patient self-reported data.\nensure that it accurately represents the hypotheses about causal pathways\nand reflects both direct and indirect relationships among variables. Through\niterative refinement and updating, the DAG becomes a dynamic tool upon\nwhich we perform simulated interventions based on established causal links.\n– Causal Inference:causal inference techniques are employed to estimate\nthe causal effects of variables on the outcome of interest. Structures such\nas causal Bayesian networks [21] can model these relationships. This step\ninvolves the examination of hypothetical scenarios through counterfactual\nreasoning, which helps predict what would happen to the outcome if a vari-\nable were altered while controlling for other factors. This provides a solid\n\nA. Hine et al.\nfoundation for making informed decisions about treatment strategies and\ndiagnostic outcomes when used in a clinical setting.\n– Evaluation:causal ML model must undergo rigorous evaluation and results\ninterpretation. Appropriate metrics, such as accuracy, precision, recall, and\nF1-score, are used to evaluate the model’s ability to make effective and reli-\nable predictions for new data. Cross-validation techniques are also employed\nto validate the model’s performance and to prevent overfitting. Involving\nclinicians in the validation process is crucial to ensure the practical appli-\ncability and clinical relevance of the model. This step incorporates expert\nfeedback to fine-tune the model.\n– Updating:causal ML model is continuously updated with new data and\nfindings. As more patient data becomes available, the model parameters and\nstructures are revised to incorporate the latest information. The updating\nprocess ensures that the model remains current and accurate over time. Reg-\nular updates also involve re-evaluating the model’s performance metrics and\nmaking necessary adjustments to improve prediction accuracy. This step is\nvital for maintaining the model’s relevance and for adapting to new infor-\nmation. In addition to routine model updates, ongoing surveys and novel\ndata collection methods should be explored to constantly enrich the dataset.\nFuture efforts may focus on integrating diverse health data streams, includ-\ning real-time patient monitoring and digital health records, to continuously\nrefine and enhance the predictive capabilities. These initiatives are crucial\nfor capturing a broader spectrum of patient experiences and for tailoring\ninterventions more effectively.\n– Deployment:once validated, the model can be deployed in a clinical set-\nting, where it can assist healthcare providers in making informed diagnos-\ntic and treatment decisions. The deployment phase involves integrating the\nmodel into existing healthcare systems, training clinicians on its use, and\nestablishing protocols for its application in clinical practice.\n– Maintenance:post-deployment,thecausalMLmodelmustbecontinuously\nmonitored for performance and accuracy, leveraging ongoing feedback from\nclinicians and patients to identify areas for improvement. Regular mainte-\nnance will ensure the model adapts to new data and evolves according to\nclinical practice and emerging research findings, sustaining its effectiveness\nand utility in real-world settings.\nFinally, Fig. 4 shows the proposed prediction modelling flow for endometrio-\nsis. The model input is global patient-collected survey data, which undergoes a\nseries of stages, from data preparation and cleaning to their utilisation within\nthe causal ML algorithms via the causal graph. The prediction modelling flow\nleads to probabilities as output that can be leveraged to estimate the likelihood\n(either positive or negative) of endometriosis or differential diagnoses in new\n\nEndometriosis Care with Causal Machine Learning\nFig.4: Overview of EndoML Project and its proposed prediction modelling flow.\npatients exhibiting symptoms. Future refinements could involve tailoring output\npredictions to further identify the most effective treatment pathways to match\nthe specific manifestations of symptoms in patients.\nCausal machine learning emerges as a promising research direction, upgrad-\ning existing Bayesian Network-based solutions by incorporating causal reason-\ning into the modelling process [21]. The hypothesis is that we can overcome the\nlimitations of traditional statistical-based ML approaches, which are primarily\ncorrelative, integrating causal inference methodologies. Unlike traditional meth-\nods, the Noisy-OR Twin Bayesian Networks [19,21] allow us to model complex\ncausal relationships among disease and symptom variables, providing a deeper\nunderstanding of patient-reported data. The term “noisy-OR” refers to the ex-\ntension of traditional Bayesian Networks (BNs) by introducing uncertainty via\n‘noisy’ nodes and logical OR operations. The term ‘twin’ refers to the two dif-\nferent versions of the world that the model encapsulates, the factual world and\nthe counterfactual world (where ‘interventions’ are performed). Introducing ad-\nditional uncertainty and complexity allows us to model more realistic scenarios\nwith incomplete or uncertain data [21]. Furthermore, causal inference uncovers\nthese causal relationships by simulating interventions on diseases via the math-\nematical ‘do(x)’ operator [21]. These interventions demonstrate the strength of\nthe relationship between a condition being ‘cured’ or ‘switched off’ and the like-\n\nA. Hine et al.\nlihood of a symptom also being ‘switched off’ as a result [19,21]. This logic leads\nto typically stronger predictions that account for temporal knowledge and under-\nlying mechanisms of associations between variables, facilitating more accurate\nand reliable predictions, thus possibly increasing chances of clinician adoption.\nIt should be emphasised that the EndoML Project prioritises a patient-\ncentric approach, highlighting the importance of understanding and addressing\nthe unique experiences and needs of individuals affected by endometriosis. Util-\nising patient-reported symptom data allows the patient to feel empowered by\nproactively participating in their healthcare journey, despite one of the most\nprominent challenges being data quality. Examples are the impact of recall bias\nand the reduced ability to control confounders in causal ML techniques [12,21].\nThe potential benefits of the approach still outweigh these risks, as healthcare\nproviders can customise interventions based on individual symptom profiles for\nfaster and more effective treatment outcomes.\nOur data collection methodology involves a thorough questionnaire applied\nacross three countries (UK, Brazil, Austria) based on the EHP-30 survey and\ninfluenced by the intake form used in an Austrian Fertility clinic. This survey\ncovers an array of variables spanning demographics, menstrual history, family\nbackground, fertility, sexual health, surgical history, and contraceptive usage,\nto cover the various symptoms experienced by women. The selection of a de-\nveloped country with a national healthcare system (UK), a developed country\nwith a predominant reliance on private healthcare (Austria), and a developing\ncountry (Brazil) was strategic. This choice aimed at capturing socioeconomic\nand geographical differences to include diversity in the input data and, hence,\nbuild a potentially stronger predictor. It is important to note that although we\nselected social media support groups in the countries of interest (UK and Brazil),\nwe encouraged the distribution of this survey link outside of these groups to gain\na larger respondent pool and capture a larger subset of the population. From\nthe UK Survey collection, we reached a total of 475 entries with 227 complete\nvalid responses from the UK specifically but we are aware of respondents from\noutwith the UK (except from Brazil/South America).\nFig. 5 details the geographical distribution of respondents of the UK Survey\n(2024) per country. Although a vast majority of respondents were from the UK,\na large proportion of responses (91) came from the United States. Australia was\nthe third most frequent country of response with 14 valid questionnaires being\nsubmitted. This distribution indicates not only a wide interest in the topic but\nalso reflects the effectiveness of our outreach strategy in engaging a diverse inter-\nnational audience even though this was not the original intended population. In\ntotal, including the UK, the survey reached 28 countries so far which highlights\nthe global interest in this work and the willingness of sufferers to share their\nexperience to improve endometriosis research. Once collected, the survey data\nis cleaned and treated, automatically analysed by a Python script, which gener-\nates probability distributions for each variable, through Kernel Density Estima-\ntion (KDE) for continuous variables, and builds custom predictions for distinct\nvariables. The resulting distributions are then transformed into a Conditional\n\nEndometriosis Care with Causal Machine Learning\nFig.5: Geographical distribution of respondents of the UK Survey per country.\nProbability Table (CPT) to weight the BN (please refer to Fig. 4). Free text\ninputs may undergo analysis via NLP techniques to uncover relevant responses\nthat can also add knowledge to the ML prediction model. However, in practical\napplications, addressing challenges during data preprocessing becomes crucial,\nespecially when managing the diverse quality of free text inputs and formulating\nrobust strategies to tackle ambiguous or context-dependent language [12].\nFinally, the output of the causal inference algorithm, determined by the ex-\npected disablement and expected sufficiency calculations [19,21], is instrumen-\ntal in determining whether a patient’s symptoms likely indicate endometriosis,\nbased on diverse data (i.e., from the three countries) and the causal ML model\npredictions. Expected disablement quantifies the estimated degree of impairment\ncaused by endometriosis-associated symptoms. This involves the causal inference\nalgorithm’s analysis of collected data, considering the severity of symptoms and\ntheir impact on the individual’s well-being. Expected sufficiency reflects the like-\nlihood that the observed symptoms suggest a positive endometriosis diagnosis\nby evaluating symptom patterns and their associations. Both calculations can\nbe used interchangeably to predict an individual’s probability of diagnosis [21].\nTherefore, in addition to upgrading existing Bayesian network-based solu-\ntions, this approach focuses on collecting diverse data from multiple sources.\nThis strategy is essential for designing a highly accurate diagnostic tool capa-\nble of understanding more comprehensively the intricacies of endometriosis. The\ndata diversity ensures that the resulting tool remains applicable, shareable, and\nusable across various geographical regions and demographic groups. This also\nbrings the benefit of bias mitigation that may arise from limited or homoge-\nneous datasets [2], thereby enhancing the generalisability and reliability of our\ndiagnostic tool, increasing the probability of adoption into clinical workflows.\n\nA. Hine et al.\n4 Preliminary Insights\nThe EndoML Project data collection and preprocessing have commenced in\npreparation for the application of the causal ML algorithms. Initial data analy-\nsis focuses on understanding the demographic and clinical characteristics of the\nsurvey respondents, as well as identifying key patterns and variances in their\nreported symptoms and medical history. This preliminary analysis serves as a\nfoundational step towards building robust ML models that can accurately di-\nagnose and provide personalised treatment recommendations for endometriosis.\nThus, in this section, we present findings from initial data analysis, including\ndescriptive statistics and visual data representations.\n4.1 Data collection & survey key areas\nWe orchestrated and implemented an extensive online survey to collect diverse\ndata concerning endometriosis and women’s health perceptions. The EndoML\nProject Survey was distributed through online social media support channels\nin two countries (UK and Brazil), whilst in the Austrian fertility clinic a con-\nsolidated version was utilised as an intake form for new patients. Furthermore,\nalthough the survey questions remain consistent across all data collection clus-\nters, translation is necessary to accommodate participants who may be more\ncomfortable responding in their native language, thereby increasing accessibility\nand reducing potential misinterpretations.\nThe survey includes diverse variables such as demographics, menstrual his-\ntory, family background, fertility, sexual health, surgical history, and contracep-\ntive use, alongside gathering individuals’ perceptions on different aspects of the\ncondition. The goal is to integrate a comprehensive dataset with key variables\nof the condition, providing detailed and high-quality information to inform the\ncausal ML model. Following is the summary of these key areas covered by the\nsurvey and the insights they offer:\n–Demographics\n- Coverage: age, country, ethnicity, education, occupation.\n- Insights: identifies demographic patterns and highlight potential biases.\n–Diagnosis & History\n- Coverage: age at diagnosis, diagnostic methods, family history.\n- Insights: highlight current diagnostic delays and genetic predispositions.\n–Symptoms Descriptions\n- Coverage: presence and severity of symptoms like menstrual irregulari-\nties, pelvic pain, and pain during intercourse.\n- Insights: essentialforidentifyingsymptompatternsandvariations,which\naids in making accurate predictions.\n\nEndometriosis Care with Causal Machine Learning\n–Cycle & Bleeding\n- Coverage: menstrual cycle regularity, duration, bleeding intensity.\n- Insights: indicators of menstrual abnormalities, disease/condition pres-\nence, severity and progression.\n–Pain Characteristics\n- Coverage: type, frequency, and intensity of pain during menstruation,\novulation, and intercourse.\n- Insights: differentiates between pain severity levels, manifestations, pain\nsensations, essential for predictive accuracy.\n–Surgeries\n- Coverage: surgical history, types, and outcomes.\n- Insights: details surgical interventions and their effect on symptom relief\nand disease progression.\n–Bowel & Bladder\n- Coverage:symptomslikepainduringurination(ordefecation);frequency,\nseverity.\n- Insights: on the multi-system impact of endometriosis, not only repro-\nductive symptoms; it can assist in identifying advanced stages (III/IV).\n–Pregnancies\n- Coverage: pregnancy history, conception difficulties, outcomes.\n- Insights: highlights impacts on fertility and pregnancy outcomes; it can\nbe used to help guiding treatment plans.\n–Birth Control\n- Coverage: use of birth control, types, effectiveness.\n- Insights: evaluates symptom management effectiveness through birth\ncontrol methods.\nThe diverse responses enabled by the UK Survey (2024) allow us to capture\nthe nuances of the heterogeneous nature of endometriosis, with significant vari-\nance in symptom profiles and women’s experiences. This variability is critical for\ndeveloping a robust model that provides personalised diagnostic and treatment\nrecommendations.\n4.2 Data preprocessing & descriptive statistics\nBuilding on the understanding gained from the diverse questions and responses,\nthis section presents descriptive statistics derived from the collected survey data.\nAnaturalfirststep(pleaserefertoFig.4)istopreparethedata,performingtasks\nsuch as cleaning (e.g., removing empty responses, correcting data entry errors or\n\nA. Hine et al.\ninconsistencies), transforming (i.e., coding responses, standardising data, etc.)\nand handling outliers (i.e., excluding or adjusting them).\nAn example of a common error made by respondents, among others, was\nentering their weight in pounds instead of kilograms as requested. To account\nfor this, we established a threshold within the data to identify instances where\nrespondentslikelymisunderstoodtheinputformat.Then,wemanuallyconverted\nthese presumed pound values to their kilogram equivalents. This issue likely\noccurred because the questionnaire link was shared beyond the UK, leading to\nnumerous responses from participants non-UK residents.\nFollowing, we present some results from the UK Survey data, the character-\nistics of the average respondent, the variance observed in the data distributions\nof some variables and a preliminary data interpretation. Initial data analysis has\nlaid the groundwork for our causal ML algorithms, which aim to leverage the\ngathered insights to enhance endometriosis diagnosis and treatment.\nAverage Respondent. In terms of demographic characteristics, the average\nrespondent age (considering a total of 371 valid responses out of 475) was 33.3\nyears old, with a standard deviation of 8.38, indicating a diverse age range among\nparticipants. Additionally, the distribution of educational backgrounds varied,\nwith the majority of respondents holding a professional degree or equivalent\nqualification. These findings underscore the heterogeneous nature of the survey\nsample and emphasise the importance of considering individual differences in\ndeveloping personalised diagnostic and treatment strategies for endometriosis.\nVariance in Data Distributions.A preliminary analysis revealed significant\nvariance in key variables related to endometriosis, reflecting the diverse experi-\nences of individuals with the condition. For instance, the distribution of diagno-\nsis time (in months) exhibited a wide range, with some individuals experiencing\nmarkedly delayed diagnoses (refer to the histograms in Fig. 6).\nSimilarly, the distribution of the number of days bleeding during a menstrual\nperiod showed considerable variability, with some respondents reporting very\nlong durations whilst others reporting none at all (Fig. 7a); also distribution of\nage at menarche or first period (Fig. 7b).\nThese data distributions highlight the complexity of endometriosis and un-\nderscore the need for tailored approaches to diagnosis and management. To visu-\nally represent these variations, we present histograms, bar charts, and box plots,\ndepicting the distribution and variance of some key variables, providing insights\ninto the heterogeneity of experiences among individuals with endometriosis.\nOverall, the descriptive statistics and variance analysis presented here offer\nvaluable insights into the symptomatology and diversity of experiences associ-\nated with endometriosis. These findings serve as a foundation for further research\nand the development of personalised diagnostic and treatment algorithms aimed\nat improving outcomes for individuals affected by this condition.\n\nEndometriosis Care with Causal Machine Learning\nFig.6: Distribution of diagnosis time (in months) experienced by suspected and\ndiagnosed endometriosis sufferers.\n(a) Menstrual cycle length.\n (b) Age at menarche.\nFig.7: Distribution of menstrual cycle length and age of first period (menarche)\nexperienced by endometriosis sufferers.\n\nA. Hine et al.\nPreliminary Data Interpretation.We illustrate in this paper some insights\nthrough visual representations. For instance, histograms depict the distribution\nof diagnosis time in months for both diagnosed and suspected sufferers (refer\nback to Fig. 6). It is evident from these representations the significant delay\nexperienced by both groups, which emphasises a concerning gap in healthcare\nprovision quality. Preliminary analysis reveals significant variance also in other\nkey variables, indicating the diverse experiences of individuals with endometrio-\nsis. For example, the duration of the menstrual cycle and the age at menarche\n(Fig. 7) show wide ranges, suggesting that endometriosis can manifest differ-\nently among individuals. In addition, the prevalence and variation of common\nsymptoms highlight the importance of personalised varied approaches.\nThe insights derived from the UK Survey data serve as valuable input for\ndeveloping the causal ML approach, allowing us, for example, to understand\nwhy certain diagnostic outcomes occur and why certain interventions are more\neffective. Our aim is to develop more tailored and effective diagnostic outcomes\nfor individuals with endometriosis.\nAnother histogram (Fig. 8) illustrates the distribution of a key variable,\nthe menstrual pain intensity, providing insights into the severity experienced\nby respondents. Additional box plots (Fig. 9) present the distribution of men-\nstrual versus non-menstrual pain intensity (severity), providing a comprehensive\noverview of the dataset in relation to the variable ‘pain’.\nFig.8: Intensity of menstrual pain experienced by endometriosis sufferers.\nFig. 10 presents a bar chart displaying the frequency of comorbidities com-\nmonly linked with endometriosis, shedding light on its potential impact on other\nhealth conditions and the diagnostic challenges that may occur due to comor-\n\nEndometriosis Care with Causal Machine Learning\n(a) Pain during menstruation.\n (b) Pain outwith menstruation.\nFig.9: Distribution of pain intensity during and outwith menstruation.\nFig.10: Most frequent comorbidities reported by endometriosis sufferers.\nbidities. Furthermore, Fig. 11 shows a bar chart highlighting the different types\nof pain experienced by individuals with suspected or diagnosed endometriosis\nwho participated in the UK Survey, demonstrating the multifaceted nature of\nthe condition. The data reveals common pain descriptors among sufferers: 281\nrespondents characterised their pain as ‘stabbing’ and 271 as ‘aching’. The term\n\nA. Hine et al.\nFig.11: Types of pain experienced by endometriosis sufferers.\n‘splitting’ was least frequently used, with only 58 responses, suggesting it may\nbe a less common descriptor. This variation in pain descriptions not only reflects\nthe different pain manifestations in endometriosis but also highlights that not\nall individuals experience pain in the same way [5,10].\nThe visual representations provided complement our descriptive statistics,\nenhancing our understanding of the data and reinforcing the need for person-\nalised approaches in diagnosing and treating endometriosis. They also serve as\nvaluable tools for communication, enabling stakeholders’ understanding of the\nmultifaceted nature and diverse manifestations of the condition. The data high-\nlights that personalised approaches are vital for managing the heterogeneity\nof endometriosis. Future research should focus on integrating machine learning\nwith clinical outcomes to develop dynamic models that can predict individual\ntreatment responses.\nThe next steps in our analysis process will involve the implementation of\nthe Noisy-OR twin diagnostic network and counterfactual algorithm in order\nto make predictions on new users’ probability of endometriosis, based on the\nknowledge gained from the diverse data sources and the expertise of teams of\nclinicians. There is a challenge ahead to ensure the EndoML predictor is accu-\nrate considering the complexities of real-world data. We understand that data\nquality is critical to the solution, as any errors or biases can affect the model’s\nperformance. We may refine our data integration approach to gather more var-\nied and targeted information, enhancing the completeness of our dataset and\nstrengthening our model’s predictive power.\n\nEndometriosis Care with Causal Machine Learning\n5 Conclusion\nCausal machine learning algorithms for diagnostics show promise in providing\naccurate predictions that could in the future be used to aid clinicians in under-\nstanding the root cause of patient symptoms in complex cases. Earlier identifi-\ncation of symptoms may reduce diagnostic delay and indirectly also promote a\nmore sustainable use of resources. Understanding the most likely cause of symp-\ntoms facilitates the development of tailored healthcare frameworks with world-\nwide benefits, particularly in the case of conditions with varied manifestations,\nas with endometriosis. Ultimately, the goal is to reduce the pain and suffering of\npatients, moving beyond traditional management approaches and the recurrent\nuse of medications and surgeries, delaying diagnosis and effective treatment. By\nrefining diagnostic accuracy, we can offer a more efficient, less invasive diagnostic\npathway that seeks to identify the root causes of symptoms caused by conditions\nlike endometriosis, leading to better patient outcomes and quality of life.\nInnovations in health informatics, like causal ML, offer potential for optimal\ndiagnosis and treatment pathways, however, challenges such as data quality,\nbiases from self-reported patient data, and clinical adoption of AI-based tools\nneed addressing. Despite these obstacles, future research should prioritise the\ndevelopment of user-friendly diagnostic tools based on causal decision-making\nalgorithms for both clinicians and patients. In this context, ‘user-friendly’ means\nproviding interfaces that simplify the complex outputs of causal decision-making\nalgorithms into actionable insights. This ensures accessibility without requiring\ndeep technical knowledge. This approach can thus facilitate further potential\nexploitation of underlying AI/ML algorithms in critical scenarios.\nImplementing such tools for endometriosis — a condition marked by complex\nsymptoms and diagnostic challenges — establishes a precedent for transforming\nthe diagnostic processes for other multifactorial diseases in similarly causality-\nfocused ways. Such advancements could significantly improve the diagnostic pro-\ncess and patient experience across various conditions, demonstrating the broad\nrelevance and applicability of these algorithmic tools.\nAcknowledgement\nWe thank Thomas Ebner from the Kinderwunsch Zentrum, Kepler Universität-\nsklinikum, Linz, for valuable insights into the processes in the Austrian health-\ncare system. We thank the team from the School of Nursing, University of São\nPaulo, Brazil, led by Lislaine Aparecida Fracolli with Ana Luiza Vilela Borges,\nCarla Marins Silva and Marlise de Oliveira Pimentel Lima, for ongoing discus-\nsions on the Brazilian perspective of endometriosis care.\nReferences\n1. Agarwal, S.K., Chapron, C., Giudice, L.C., Laufer, M.R., Leyland, N., Miss-\nmer, S.A., Singh, S.S., Taylor, H.S.: Clinical diagnosis of endometriosis: a call\n\nA. Hine et al.\nto action. American journal of obstetrics and gynecology220(4), 354–e1 (2019).\nhttps://doi.org/10.1016/j.ajog.2018.12.039\n2. Ballweg, M.L.: Impact of endometriosis on women’s health: comparative histor-\nical data show that the earlier the onset, the more severe the disease. Best\npractice & research Clinical obstetrics & gynaecology 18(2), 201–218 (2004).\nhttps://doi.org/10.1016/j.bpobgyn.2004.01.003\n3. Blass, I., Sahar, T., Shraibman, A., Ofer, D., Rappoport, N., Linial, M.: Revis-\niting the risk factors for endometriosis: A machine learning approach. Journal of\nPersonalized Medicine 12(7), 1114 (2022). https://doi.org/10.3390/jpm12071114\n4. Burton, C., Iversen, L., Bhattacharya, S., Ayansina, D., Saraswat, L., Sleeman,\nD.: Pointers to earlier diagnosis of endometriosis: a nested case-control study using\nprimary care electronic health records. British Journal of General Practice67(665),\ne816–e823 (2017). https://doi.org/10.3399/bjgp17x693497\n5. Conroy, I., Mooney, S.S., Kavanagh, S., Duff, M., Jakab, I., Robertson, K., Fitzger-\nald, A.L., Mccutchan, A., Madden, S., Maxwell, S., et al.: Pelvic pain: what are\nthe symptoms and predictors for surgery, endometriosis and endometriosis sever-\nity. Australian and New Zealand Journal of Obstetrics and Gynaecology61(5),\n765–772 (2021). https://doi.org/10.1111/ajo.13379\n6. Cox, H., Henderson, L., Andersen, N., Cagliarini, G., Ski, C.: Focus group study\nof endometriosis: Struggle, loss and the medical merry-go-round. International\njournal of nursing practice 9(1), 2–9 (2003). https://doi.org/10.1046/j.1440-\n172x.2003.00396.x\n7. Eskenazi, B., Warner, M.L.: Epidemiology of endometriosis. Obstet-\nrics and gynecology clinics of North America 24(2), 235–258 (1997).\nhttps://doi.org/10.1016/s0889-8545(05)70302-8\n8. Fauconnier, A., Drioueche, H., Huchon, C., Du Cheyron, J., Indersie, E.,\nCandau, Y., Panel, P., Fritel, X.: Early identification of women with en-\ndometriosis by means of a simple patient-completed questionnaire screening\ntool: a diagnostic study. Fertility and Sterility 116(6), 1580–1589 (2021).\nhttps://doi.org/10.1016/j.fertnstert.2021.07.1205\n9. Goldstein, A., Cohen, S.: Self-report symptom-based endometriosis pre-\ndiction using machine learning. Scientific Reports 13(1), 5499 (2023).\nhttps://doi.org/10.1038/s41598-023-32761-8\n10. Gordon, H.G., Mooney, S.S., Conroy, I.C., Grover, S.R.: When pain is not the\nwhole story: Presenting symptoms of women with endometriosis. Australian and\nNew Zealand Journal of Obstetrics and Gynaecology 62(3), 434–438 (2022).\nhttps://doi.org/10.1111/ajo.13482\n11. Hine, A.A., Kuster Filipe Bowles, J., Webber, T.: The need for a non-invasive\ntechnology for endometriosis detection and care. Caring is sharing-exploting the\nvalue in data for health and innovation (2023). https://doi.org/10.3233/shti230073\n12. Iroju, O.G., Olaleke, J.O.: A systematic review of natural language processing in\nhealthcare. International Journal of Information Technology and Computer Science\n8, 44–50 (2015). https://doi.org/10.5815/ijitcs.2015.08.07\n13. Kleczyk, E.J., Peri, A., Yadav, T., Komera, R., Peri, M., Guduru, V.,\nAmirtharaj, S., Huang, M.: Predicting endometriosis onset using machine\nlearning algorithms. BMC Women’s Health (preprint under review) (2020).\nhttps://doi.org/10.21203/rs.3.rs-135736/v1\n14. Kruk, M.E., Gage, A.D., Arsenault, C., Jordan, K., Leslie, H.H., Roder-DeWan,\nS., Adeyi, O., Barker, P., Daelmans, B., Doubova, S.V., et al.: High-quality health\nsystems in the sustainable development goals era: time for a revolution. The\n\nEndometriosis Care with Causal Machine Learning\nLancet global health 6(11), e1196–e1252 (2018). https://doi.org/10.1016/s2214-\n109x(18)30386-3\n15. Malvezzi, H., Marengo, E.B., Podgaec, S., Piccinato, C.d.A.: Endometriosis: cur-\nrent challenges in modeling a multifactorial disease of unknown etiology. J. Transl.\nMed. 18, 1–21 (2020). https://doi.org/10.1186/s12967-020-02471-0\n16. Markham,S.M.,Carpenter,S.E.,Rock,J.A.:Extrapelvicendometriosis.Obstetrics\nand gynecology clinics of North America16(1), 193–219 (1989)\n17. McKee, M., Wouters, O.J.: The Challenges of Regulating Artificial Intelli-\ngence in Healthcare: Comment on “Clinical Decision Support and New Reg-\nulatory Frameworks for Medical Devices: Are We Ready for It?-A Viewpoint\nPaper”. International journal of health policy and management 12 (2023).\nhttps://doi.org/10.34172/ijhpm.2022.7261\n18. Nnoaham, K.E., Hummelshoj, L., Kennedy, S.H., Jenkinson, C., Zondervan, K.T.:\nWorld endometriosis research foundation women’s health symptom survey consor-\ntium. developing symptom-based predictive models of endometriosis as a clinical\nscreening tool: results from a multicenter study. Fertil Steril98(3), 692–701 (2012).\nhttps://doi.org/10.1016/j.fertnstert.2012.04.022\n19. Pearl, J.: Causality. Cambridge university press (2009).\nhttps://doi.org/10.1017/CBO9780511803161\n20. Pergialiotis, V., Pouliakis, A., Parthenis, C., Damaskou, V., Chrelias, C.,\nPapantoniou, N., Panayiotides, I.: The utility of artificial neural net-\nworks and classification and regression trees for the prediction of endome-\ntrial cancer in postmenopausal women. Public Health 164, 1–6 (2018).\nhttps://doi.org/10.1016/j.puhe.2018.07.012\n21. Richens, J.G., Lee, C.M., Johri, S.: Improving the accuracy of medical diagno-\nsis with causal machine learning. Nature communications 11(1), 3923 (2020).\nhttps://doi.org/10.1038/s41467-020-17419-7\n22. Rogers, P.A., D’Hooghe, T.M., Fazleabas, A., Gargett, C.E., Giudice,\nL.C., Montgomery, G.W., Rombauts, L., Salamonsen, L.A., Zondervan,\nK.T.: Priorities for endometriosis research: recommendations from an in-\nternational consensus workshop. Reproductive sciences 16, 335–346 (2009).\nhttps://doi.org/10.1177/1933719108330568\n23. Rolla, E.: Endometriosis: advances and controversies in classifica-\ntion, pathogenesis, diagnosis, and treatment. F1000Research 8 (2019).\nhttps://doi.org/10.12688/f1000research.14817.1\n24. Sankaravadivel, V., Thalavaipillai, S.: Symptoms based endometriosis prediction\nusing machine learning. Bulletin of Electrical Engineering and Informatics10(6),\n3102–3109 (2021). https://doi.org/10.11591/eei.v10i6.3254\n25. Shafrir, A.L., Farland, L., Shah, D., Harris, H., Kvaskoff, M., Zondervan, K., Miss-\nmer, S.: Risk for and consequences of endometriosis: a critical epidemiologic re-\nview. Best practice & research Clinical obstetrics & gynaecology51, 1–15 (2018).\nhttps://doi.org/10.1016/j.bpobgyn.2018.06.001\n26. Simoens, S., Dunselman, G., Dirksen, C., Hummelshoj, L., Bokor, A., Bran-\ndes, I., Brodszky, V., Canis, M., Colombo, G.L., DeLeire, T., et al.: The bur-\nden of endometriosis: costs and quality of life of women with endometriosis\nand treated in referral centres. Human reproduction 27(5), 1292–1299 (2012).\nhttps://doi.org/10.1093/humrep/des073\n27. Sivajohan, B., Elgendi, M., Menon, C., Allaire, C., Yong, P., Bedaiwy, M.A.: Clin-\nical use of artificial intelligence in endometriosis: A scoping review. Npj Digital\nMedicine 5(1), 109 (2022). https://doi.org/10.1038/s41746-022-00638-1","source_license":"CC0","license_restricted":false}