The quality, suitability, and readability of web-based resources on endometriosis-associated dyspareunia: A systematic review

People commonly and increasingly rely on the internet to search for health information, including those related to endometriosis-associated dyspareunia. Yet the content of such websites may be of variable accuracy and quality. This review aims to evaluate the quality, readability, and suitability of web-based resources on endometriosis-associated dyspareunia for patients. We searched Corresponding author: Abdul-Fatawu Abdulai, School of Nursing, University of British Columbia, T201-2211 Wesbrook Mall, Vancouver, BC V6T 2B5, Canada. Email: [email protected] Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by- nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as speci fied on the SAGE and Open Access pages ( https://us.sagepub. com/en-us/nam/open-access-at-sage). 3 databases – Google, Bing, and Yahoo – to identify websites related to endometriosis-associated dyspareunia. Two independent reviewers screened the search results against inclusion and ex- clusion criteria. Another set of two reviewers evaluated the selected websites using validated measurement instruments. Out of 450 websites, 21 met the inclusion criteria and were evaluated. More than half of the websites had information on content updates, reported on authorship, or disclosed sponsorship information. The mean quality and suitability scores were 47.5 (SD = 13.3) and 65.2 (SD = 13.6) respectively, thus suggesting generally adequate quality and suitability levels. However, the mean readability scores exceeded the recommended level for health-related websites. The poor readability of the websites might limit accessibility for a signi ficant propor- tion of patients with low educational levels. The findings of this review have implications for designing high-quality, readable and up-to-date web interventions for people who rely on web platforms as an alternative or complementary source of health information on dyspareunia.

dyspareunia, quality, readability, suitability, review, web-based resources

Endometriosis is a chronic in flammatory condition in which endometrial-like tissue grows outside the uterus, typically in the pelvic cavity. 1 The condition has been estimated to affect approximately 10% of women globally 2 and an unknown number of gender-diverse people. Symptoms include chronic pelvic pain, dysmenorrhea, abnormal menstrual bleeding, dysphasia, dysuria, and impaired fertility. One of the cardinal symptoms of endometriosis is dyspareunia, which has been estimated to affect more than 50% of people with this disease. 2,3 Dyspareunia (i.e., pain experienced during and/ or after penetrative vaginal intercourse) has been found to have detrimental effects on quality of life including reduced sexual function, low self-esteem, and poor interpersonal relationships. 3–7 Despite the negative effects of dyspareunia on quality of life, the symptom is often normalized by healthcare providers and in some cases neglected. 8 Patients have also reported ignoring pain symptoms because they are unaware of the link between dyspareunia and endometriosis. 7 For those who may be aware of their dyspareunia symptoms and inclined to seek medical care, the sensitive and potentially stigmatizing nature of pain, intimacy, and genitalia, may prevent patients from disclosing their symptoms to providers and inhibit patient-provider discussions on the topic. Due to the sensitive nature of dyspareunia and limited patient-provider discussions on the topic, it is highly likely that people have sought related information online. Indeed, the internet has become a major source of health information, with more than 80% of Internet users accessing the world wide web to find health-related information, 9,10 and an estimated 6.75 million daily health searches occurring in Google alone.11 Moreover, people turn to the internet to an even greater extent when searching for information about health conditions perceived as embarrassing and stigmatizing including dyspareunia. 12 For instance, more than 400,000 endometriosis-related Google searches are performed monthly in the United States alone.13 With the internet being a major source of health- related information, people can explore sensitive and personal questions regarding their health in the privacy and comfort of their homes. 14,15 Despite the high use of web-based information, the information provided on these platforms are often complex, and written at an advanced literacy level, with most websites unregulated and of varying quality. 14,16 The complexity of internet-based information often compounds people’s ability 2 Health Informatics Journal to read, comprehend, or utilize the information in their daily lives. 16,17 Exposure to complex and unregulated web-based information may negatively affect peoples ’ understanding, application, and decision-making ability, especially among populations with low literacy or those who experience marginalization.18,19 The use of unregulated web-based information may be prominent in people with sensitive and stigmatizing conditions like dyspareunia because people with such disorders are more likely to resort to web-based resources as complementary or alternative sources of information. 20 The purpose of this review is to evaluate the readability, suitability, and quality of web-based information on endometriosis-associated dyspareunia for end-user patients. It is expected that the findings from this study will inform the design of future tailored web-based interventions for patients on sensitive and potentially stigmatizing health topics who may have low literacy levels. This is important because having quality, suitable and readable web-based resources tailored to patients’ needs and expertise is essential to non-expert patients who rely so much on web-based resources but may not be able to evaluate and distinguish between credible and inaccurate information.

We conducted a 2-step process in this evaluation. First, we conducted a search for the websites and two independent reviewers screened the search results against inclusion and exclusion criteria. Secondly, another set of two reviewers evaluated the selected websites to assess the quality, suitability, and readability. Website searches and screening We conducted a systematic review of English-language web-based resources on endometriosis- associated dyspareunia across the globe. Web searches were conducted across the three most popular search engines including Google, Bing, and Yahoo. 9 Approximately 77% of website users across the globe will begin their search from these 3 sources with an estimated monthly visit of 1.8 billion and 500 million on Google and Bing respectively. 9,21 The following search terms were used for each of the 3 search engines: “Endometriosis + painful sex/sexual pain, ”“ Endometriosis + dyspareunia,” and “Endometriosis + sex ”. To prevent biased results from past search history and search location, we erased all search histories, cookies, and cached data from the web browser before beginning each search. It has been established that more than 97% of people might not review

beyond the first 3 pages of a search. 17 Therefore, search results in the first 5 pages of each search engine were extracted. This process was repeated for all the search terms across the 3 databases. The result for each search term across all search engines were compiled in excel and duplicate websites were removed. Two independent reviewers screened the remaining search results against the inclusion and exclusion criteria. Websites were included if they contained >300 words about endometriosis-associated dyspareunia, were written in English, and were publicly available. Websites were excluded if they provided information on endometriosis-associated dyspareunia exclusively to healthcare professionals or researchers, had dead/broken links, or were merely links to documents, blogs, social media, or online newspapers/magazines. Because this review was focused on dyspareunia, websites that contained information on endometriosis without corre- sponding information on dyspareunia were also excluded. Websites that met the inclusion criteria were entered into an Excel spreadsheet for further data extraction and evaluation. Discrepancies between the two reviewers on the website selection were resolved by discussion with a third Abdulai et al. 3 reviewer. The search and screening of the websites was conducted between May 2022 – October 2022. A repeated search was conducted in October 2023 but found no additional websites. Websites evaluation Following data extraction, another set of two independent reviewers evaluated the websites ’ quality and suitability using validated measurement instruments. 22–25 The purpose of choosing a different set of reviewers was to reduce biases from the screening process. These two reviewers were given a day’s training on how to use each measurement instrument. The evaluation was done independently by each reviewer and was recorded on an Excel sheet. The country where the website was hosted and maintained was also extracted. Discrepancies between the two evaluators were resolved by an ongoing discussion with a third person until a consensus is reached. The details of each mea- surement instrument are outlined below. The evaluation of the website content was conducted between November 2022 – March 2023. A re-evaluation was also conducted in October 2023 but found no major changes to the content of the websites. Quality. Quality was assessed using DISCERN, a 16 item questionnaire that evaluates the quality of health information on a 5-point Likert scale from 0 (very poor quality) to 5 (very high quality). 23 DISCERN has three main sections. Questions 1-8 assess the reliability of the information, 9 – 15 assess the quality of information on treatment choices, while question 16 evaluates overall quality about treatment choices. A cumulative score of the 16 items was calculated and the results were used to classify the quality of each website as either high quality (53 –80), moderate quality (27–52), or low quality (<27). To understand the performance of the websites on the individual components of DISCERN, we calculated the scores for each sub-scale of DISCERN by adding the average scores for each sub-scale (i.e., reliability, quality, and overall quality). The quality of the websites were also assessed using the Journal of American Medical Association (JAMA) benchmark criteria. 26 JAMA is a Yes/No checklist that assesses the quality of websites across four quality attributes including (1) authorship and af filiations, (2) attribution/references 3) disclosure and (4) currency. Suitability. Suitability was assessed using the Suitability and Comprehensibility Assessment of

(SAM) tool.23,24 SAM is a 22-item scale that assesses the suitability of materials across six components including content, literacy demand, graphics, layout and typography, learning stim- ulation and motivation, and cultural appropriateness. Each component assesses web-based in- formation as either “Superior,” (scored 2), “Adequate,” (scored 1) or not suitable (scored 0). The total scores are then divided by the maximum possible score. Websites with scores between 70%- 100% were interpreted as superior, 40% - 69% were interpreted as adequate, and 0%-39% were interpreted as not suitable. Components that did not apply to each website were excluded from the overall maximum possible score. Readability. Readability was assessed by the third author using the online tool, the Flesch-Kincaid Reading Ease test ( https://goodcalculators.com/flesch-kincaid-calculator/). 27 Readability describes the minimum level of education that is needed to read and understand a written text. The Flesch- Kincaid Reading Ease test gives a score of 0 –100, with test scores of 0 being the most dif ficult to read and 100 being the easiest to read. Score ranges were used to categorize the approximate reading grade level needed to read the websites. Because the Flesch-Kincaid Reading Ease test does not provide an understanding of readability from the patients ’ perspective, we engaged our two 4 Health Informatics Journal reviewers to provide an additional perspective on readability by commenting on how easy they find the website content to read. Data analysis The data were summarized using descriptive statistics. The mean, standard deviation, and ranges for all 21 websites were calculated across the 3 criteria of quality, suitability, and readability. We also calculated the scores for sub-dimensions of DISCERN. For the DISCERN and SAM assessments, an intraclass correlation coef ficient (ICC) was calculated to determine the level of agreement between the two evaluators before the final composite scores were computed. The ICC for this study was based on Koo et al 28 guidelines for selecting and reporting intraclass correlation coef ficients for reliability research. All analyses were conducted using SPSS V ersion 28.

The search strategy yielded a total of 450 websites across the 3 search engines. We removed 341 duplicate websites for the remaining 109 web pages, which 2 reviewers assessed based on the inclusion and exclusion criteria outlined above. Following the screening process, 21 websites were included in the final assessment. Figure 1 shows the flowchart for the website selection process. 13 Figure 1. Flowchart for the website selection process. Abdulai et al. 5 (61.9%) of the websites were hosted and maintained in the United States, 5 (23.8%) were global websites, 2 (9.5%) were from Australia, and 1 (4.7%) was in Canada. Quality (DISCERN & JAMA) The overall average DISCERN score was 47.5 with a standard deviation of 13.3, ranging from 20 to 71 out of a possible score of 80. Based on the DISCERN scores, 9 websites (42.8%) were of high quality, 8 (38.1%) were of moderate quality, and 4 (19.1%) were low quality. Table 1 shows the summary of the website quality, suitability, and readability indicators across the 21 websites (including the DISCERN scores). The summed average scores for the reliability sub-scale of DISCERN was 20.7, quality sub-scale of DISCERN was 23.7 and the average overall quality was 3.1. The overall mean score of 47.5 suggests that the websites were generally of moderate quality. There was an excellent or a good level of agreement between the evaluators on the DISCERN scores for 12 of the websites (ICC >0.7), a moderate level of agreement for 7 websites (ICC = 0.5– 0.7) and a poor level of agreement for 1 website (ICC <0.5). Overall, there was an excellent level of agreement between the two evaluators on the DISCERN scores (ICC = 0.9, [CI, 0.95 –0.99] p=< .001). The interrater reliability scores for the DISCERN and SAM assessments are shown in Table 2. The JAMA quality criteria in Table 1 revealed that 16 (76.2%) of the websites have author or contributors’ information stated, 14 (66.6%) reported on the sources of information and references, 11 (52.4%) disclosed sponsorship, advertising, underwriting, commercial funding arrangements or support, or potential con flicts of interest while 15 (71.4%) indicated when the information was last updated. Regarding information updates, only four websites have had their content updated in 2023, one had content update in 2022, 3 had not date for content update while the rest were updated between 2015 and 2019. Suitability The SAM assessment shows that seven (33.3%) of the websites were found to be highly suitable, 12 (57.1%) were found to be adequately suitable, and 2 (9.5%) were found not to be suitable. Overall, the websites were found to be adequately suitable (Mean SAM = 65.2, SD = 13.6, Range 34.4 – 83.7). The websites ’ suitability indicators (i.e, the SAM scores across the 21 websites) are also depicted in Table 1. There was an excellent level of agreement between the two evaluators on the SAM scores for 9 websites (ICC >0.9), a good level of agreement for 1 website (ICC = 0.7 – 0.9), a moderate level of agreement for 10 websites (ICC = 0.5 – 0.7) and a poor level of agreement for 1 website (ICC <0.5). The intraclass correlation coefficients for the SAM scores on each website are shown in Table 2. Graphics and cultural appropriateness components of SAM was not applicable to at least seven of the websites. Readability Based on the Flesch-Kincaid Reading Ease Test, 2 (9.5%) of the websites were assessed as fairly difficult to read (i.e., 10th – 12th grade reading level), 16 (76.2%) were assessed as dif ficult to read (i.e., College student reading level), and three (14.3%) were assessed as very dif ficult to read (i.e., College graduate reading level). Overall, all 21 websites were assessed as fairly difficult to read (Mean = 54.2, Standard Deviation = 8.7, Range = 21.8 – 59.7). None of the websites contained information on dyspareunia that was found to be at a reading level for people with an educational level below 10th grade. The readability of the websites by the US reading age (i.e., Flesch-Kincaid 6 Health Informatics Journal Table 1. Summary of websites ’ quality, suitability, and readability. # Website URL Country Quality (JAMA & DISCERN) A B C Authors? Attribution? Disclosure? Currency? Discern Suitability (%) FRES 1 https://www.medicalnewstoday.com/articles/321417 USA Yes Yes No Yes 57.5 83.7 37.9 2 https://www.webmd.com/women/endometriosis/ endometriosis USA Yes Yes Yes Yes 62.0 80.9 54.2 3 https://www.endofound.org/painful-sex-dyspareunia USA No Yes No No 54.0 73.3 42.0 4 https://www.everydayhealth.com/womens-health/why-sex- hurts-with-endometriosis.aspx USA Yes No Yes Yes 55.5 74.9 49.2 5 endometriosis.net/symptoms/pain-having-sex Global Yes Yes Yes Yes 56.0 82.2 37.0 6 https://www.endometriosis.org/resources/articles/painful- intercourse/ Global Yes No No No 38.5 79.3 44.6 7 drseckin.com/painful-sex-and-endometriosis/ USA No Yes No Yes 41.0 68.1 44.9 8 https://www.huffpost.com/entry/less-painful-sex- endometriosis USA Yes No No Yes 53.5 58.7 45.9 9 https://www.mayoclinic.org/diseases-conditions/painful- intercourse/symptoms-causes/syc-20375967 USA Yes No No Yes 59.0 62.5 37.7 10 https://www.healthgrades.com/right-care/endometriosis/ pain-during-sex-it-could-be-endometriosis USA Yes Yes No Yes 50.0 68.1 26.8 11 https://www.chiavaye.com/blogs/endometriosis/does-sex- make-endometriosis-worse USA No No Yes Yes 50.5 59.2 59.7 12 https://www.jeanhailes.org.au/health-a-z/endometriosis/ relationships (updated) Australia Yes Yes Yes Yes 53.0 69.8 40.3 13 https://www.intimina.com/blog/endometriosis-sex/(updated) Global Yes No No Yes 43.5 64.8 48.1 14 https://www.issm.info/sexual-health-qa/what-is-genito-pelvic- pain-penetration-disorder-gpppd Global No No Yes No 41.5 54.2 44.0 15 https://www.volusonclub.net/empowered-womens-health/ how-to-get-patients-to-talk-about-dyspareunia USA Yes Yes No Yes 43.5 54.5 21.8 16 https://www.endometriosisnews.com/dyspareunia-painful- intercourse/ USA Yes No Yes No 20.0 63.9 29.6 (continued) Abdulai et al. 7 Table 1. (continued) # Website URL Country Quality (JAMA & DISCERN) A B C Authors? Attribution? Disclosure? Currency? Discern Suitability (%) FRES 17 https://www.endopain.endometriosis.org/painful-sex/ Canada Yes Yes Yes No 71.0 79.4 49.6 18 https://www.mummyotherapy.com/blog/2019/10/31/ endometriosis-and-painful-sex Australia Yes Yes Yes Yes 45.5 63.4 42.6 19 https://www.drelist.com/blog/endometriosis-painful-sexual- intercourse/ USA No Yes No No 54.5 59.1 43.1 20 https://www.menstruationresearch.org/2016/03/17/ endometriosis-painful-sex-and-physical-therapy/ USA Yes Yes Yes No 24.0 35.0 43.3 21 https://www.hormonesmatter.com/love-hurts-sex- endometriosis-dyspareunia Global Yes Yes Yes Yes 21.0 34.1 47.6 8 Health Informatics Journal Table 3. The readability of the websites by US reading age. Ease of reading # of websites Educational level (USA) Very easy (90 –100) 0 5th grade Easy (80 –90) 0 6th grade Fairly easy (70 –80) 0 7th grade Plain English (60 –70) 0 8th –9th grade Fairly dif ficult (50 –60) 2 10th –12th grade Difficult (30 –50) 16 College Very dif ficult (0 –30) 3 College graduate Table 2. Summary of interrater reliability for DISCERN and SAM assessment. Website DISCERN SAM ICC p- value ICC p- value 1 https://medicalnewstoday.com/articles/ 0.97 <.001 0.97 <.001 2 https://www.webmd.com/women/endometriosis/endometriosis-intimacy 0.89 <.001 0.97 <.001 3 https://www.endofound.org/painful-sex-dyspareunia 0.96 <.001 0.96 <.001 4 https://www.everydayhealth.com/womens-health/why-sex-hurts-with- endometriosis.aspx 0.97 <.001 0.98 <.001 5 https://endometriosis.net/symptoms/pain-having-sex 0.95 <.001 1.00 6 https://endometriosis.org/resources/articles/painful-intercourse/ 0.99 <.001 0.97 <.001 7 https://drseckin.com/painful-sex-and-endometriosis/ 0.91 <.001 0.95 <.001 8 https://www.huffpost.com/entry/less-painful-sex-endometriosis 0.73 .008 0.80 .001 9 https://www.mayoclinic.org/diseases-conditions/painful-intercourse/ symptoms-causes/syc-20375967 0.70 .013 0.71 .013 10 https://www.healthgrades.com/right-care/endometriosis/pain-during-sex-it- could-be-endometriosis 0.40 .166 0.67 .008 11 https://chiavaye.com/endometriosis/does-sex-make-endometriosis-worse 0.73 .008 0.52 .054 12 https://www.jeanhailes.org.au/health-a-z/endometriosis/relationships 0.55 .06 0.57 .031 13 https://www.intimina.com/blog/endometriosis-sex/ 0.91 <.001 0.97 <.001 14 https://www.issm.info/sexual-health-qa/what-is-genito-pelvic-pain- penetration-disorder-gpppd 0.72 .010 0.74 .010 15 https://www.volusonclub.net/empowered-womens-health/how-to-get- patients-to-talk-about-dyspareunia-and-endometriosis/ 0.68 .016 0.43 .135 16 https://endometriosisnews.com/dyspareunia-painful-intercourse/ 0.67 .018 0.68 .025 17 https://endopain.endometriosis.org/painful-sex/ 0.94 <.001 0.74 .010 18 https://www.mummyotherapy.com/blog/2019/10/31/endometriosis-and- painful-sex 0.77 .004 0.59 .047 19 https://www.drelist.com/blog/endometriosis-painful-sexual-intercourse/ 0.65 .025 0.67 .022 20 https://www.menstruationresearch.org/2016/03/17/endometriosis-painful- sex-and-physical-therapy/ 0.58 .052 0.59 .058 21 https://hormonesmatter.com/love-hurts-sex-endometriosis-dyspareunia 0.76 .004 0.90 <.001 ICC = Intraclass correlation coef ficient, SAM = Suitability assessment of materials. Abdulai et al. 9 Reading Ease Score) is shown in Table 3. Unlike the readability evaluation from the Flesch-Kincaid Reading Ease test, our reviewers ’ comments on readability were generally positive. However, they acknowledged that some content and terminologies like dyspareunia were unfamiliar and difficult to understand even though they can easily read them.

To the best of our knowledge, this study is the first study to examine the quality, suitability, and readability of websites speci fically focused on endometriosis-associated dyspareunia. It is also important to note that while this review was focused on dyspareunia, most of the websites broadly focussed on endometriosis, with few pages dedicated to dyspareunia. This likely stems from the symptoms of dyspareunia occurring, not in isolation, but as one symptom among the numerous other symptoms of endometriosis, such as painful menses, urination, or defecation. 3–5 Furthermore, the comparatively limited focus on dyspareunia also suggests that even in the context of endo- metriosis, the symptom of dyspareunia is not commonly acknowledged, described, explained, or prioritized. This is perhaps explained, in part, by the normalization of dyspareunia generally and the limited awareness and discomfort of healthcare providers involved in generating disease-related information. 7 Even though we searched for endometriosis-associated dyspareunia websites across the global, majority of the websites that meet our inclusion criteria were largely developed and maintained in high income countries. These findings reflected the neglected nature of dyspareunia as a symptom of endometrioses, particularly across developing countries. 29,30 The fact that these websites are de- veloped in high income countries may not take into account the cultural nuances of patients in other parts of the world who access these websites. Overall, there was a wide variability in the readability, suitability, and quality of the websites. All the websites exceeded the recommended grade level for the readability of online health information. Even though the readability of the websites on dyspareunia were consistent with websites on other disease conditions, 16,17,31 they might nevertheless affect access to information by patients with low levels of education (i.e., below 9th grade). Considering the importance of health literacy and the generally neglected nature of endometriosis-associated dyspareunia, providing more readable and easy to understand information may optimise patient decision making and improves engagement with online resources, particularly for patients with limited English pro ficiencies. 32 The quality of the websites was found to be generally moderate (mean DISCERN scores = 47). Even though overall quality was good, a relatively lower score on the reliability sub-scale of DISCERN shows that the websites may be less reliable for patients even though they might have scored high on quality sub-scale. Furthermore, 3 of the websites were rated as poor quality with outdated in- formation. Such poor quality and outdated websites may likely mislead patients to opt for inap- propriate treatment or cause financial loss as a result of purchases of ineffective/harmful products. 33 Similar to the quality assessment, the suitability score was also considered adequate (i.e., an average score of 65.2%). Generally, graphics are seen as an effective way of patient education. 34 However, 7 the websites were missing essential elements relating to graphics and cultural ap- propriateness components of SAM. Given the limited understanding of dyspareunia and the im- portant role of graphics in patient education, 34 graphics and illustrations are needed as a means of conveying complex medical problems. In the context of endometriosis and dyspareunia, we believe that graphics can help break up the complex relationship between endometriosis, pelvic pain, and dyspareunia in an easy-to-understand format, particularly for patients with low educational levels. In addition to our recommendation for graphics, cultural appropriateness should also be thought of 10 Health Informatics Journal as a way of reaching out to patients from ethnic minorities who experience systemic discrimination and inequities in accessing and using healthcare, such as those who are Indigenous and/or racialized, and with limited English language reading comprehension. Cultural appropriateness is important because websites are ubiquitous and could be accessed by people in low income countries even though they are hosted and maintained in high-income countries. 35,36 Implications for healthcare Even though this review was conducted for patients, the findings have implications for everyone involved in endometriosis and dyspareunia care including patients, healthcare professionals, as well as web-based interventions developers. In fact, the websites rated as poor quality or outdated may not only deny patients of the latest evidence and best practices on dyspareunia but also lead to worse health outcomes. Given the increasing reliance on and referral of patients to web-based resources, 16,37,38 these findings have important implications for providing healthcare in an in- creasingly digitalised world. As providers of care, health professionals should advise patients of the inherent inaccuracies, misinformation, or outdated information associated with online platforms. 16 A careful consideration is also required by healthcare professionals and patients to identify available web-based platforms on dyspareunia that might have been exposed to patients. The ability to identify quality web-based platforms might facilitate appropriate decision-making and self- management of endometriosis-associated dyspareunia for patients who rely on web platforms as their main sources of information. While it may be challenging to achieve 100% accuracy on health- related online platforms, efforts must be made to co-create and deploy web-based platforms by engaging with patients in the development process. In cases when co-creation may be challenging, patients should be empowered to identify accurate and credible web-based resources. Furthermore, the accuracy and quality of online health information should not be the only concern for patients and healthcare professionals because a website can be considered as accurate but out of date. Research on endometriosis-associated dyspareunia and its clinical guidelines is constantly evolving and websites but be regularly updated to re flect the latest evidence in the field. 39,40 Strengths and limitations The study had some strengths. Engaging two different set of reviewers for screening and evaluating the websites helped to reduce the biases of the study – improving our findings and applicability of results. For the two website evaluators, there was a higher level of agreement between on both DISCERN and SAM instruments – improving the reliability of our findings. By disabling location services on the computers used for the searches, we were able to eliminate geographical biases thereby ensuring that our search results re flect all websites across the globe. Despite these strengths, the review had some limitations. Even though the level of agreement between the two evaluators were high, we cannot account for the inherent subjectivity of the DISCERN and SAM instrument, which we believed, might have in fluenced an assessment of the websites. The SAM and DISCERN instruments were also developed over 20 years ago at the time of this review and their relatively older nature could limit their applicability to modern websites. Even though recent psychometric analysis has proved the relevance of the tools, 41–44 they still do not allow for a detailed and comprehensive analysis on the various sub-scales. Further studies should modify and expand on the tools to allowed for detailed and nuanced analysis of the sub-scales as they apply to modern websites. Furthermore, only English-language websites were selected for this review and the findings may not be generalizable to websites written in other languages or for Abdulai et al. 11 patients in non-English speaking countries. Further research is needed to evaluate websites in other languages and perhaps compare their quality to those in English. While the limitations in this study are similar to those observed across other web-based health information evaluation studies, 16,31,45 they nevertheless be taken into consideration when interpreting or applying the findings, particularly for patients in non-English speaking countries or in low and middle income countries.

Overall, the suitability and quality of the information on the websites were generally adequate and the readability level exceeded the recommended levels for health-related websites. The poor readability levels can limit patients ’ ability to use health-related websites or lead to serious health consequences if outdated recommendations are applied. Future research should explore insights into how web-based resources should be developed to meet the needs of patients, particularly those with low literacy skills. As modern clinical encounters for patients with stigmatized conditions/disorders gradually move from face-to-face to web-based information-seeking and web-based interactions, 37 regular revisions are warranted to improve the overall quality and readability of health-related websites for patients. This assessment provides valuable information on existing endometriosis- associated dyspareunia websites to guide healthcare providers in directing their patients to credible, quality, and suitable web-based resources. Declaration of con flicting interests The author(s) declared no potential con flicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or pub- lication of this article: This work was supported by the Canadian Institutes of Health Research. ORCID iD Abdul-Fatawu Abdulai  https://orcid.org/0000-0002-9395-8642

1. Yong PJ. Deep dyspareunia in endometriosis: a Proposed framework based on pain mechanisms and genito-pelvic pain penetration disorder. Sex Med Rev 2017; 5(4): 495–507. DOI: 10.1016/j.sxmr.2017.06. 005. 2. Sinaii N, Plumb K, Cotton L, et al. Differences in characteristics among 1,000 women with endometriosis based on extent of disease. Fertil Steril 2008; 89(3): 538 –545. DOI: 10.1016/j.fertnstert.2007.03.069. 3. Denny E. Women ’s experience of endometriosis. J Adv Nurs 2004; 46: 641 –648. DOI: 10.1111/j.1365- 2648.2004.03055.x. 4. Ferrero S, Esposito F, Abbamonte LH, et al. Quality of sex life in women with endometriosis and deep dyspareunia. Fertil Steril 2005; 83(3): 573 –579. DOI: 10.1016/j.fertnstert.2004.07.973. 5. Shum LK, Bedaiwy MA, Allaire C, et al. Deep dyspareunia and sexual quality of life in women with endometriosis. Sex Med 2018; 6(3): 224 –233. DOI: 10.1016/j.esxm.2018.04.006. 6. Evangelista A, Dantas T, Zendron C, et al. Sexual function in patients with deep in filtrating endometriosis. J Sex Med 2014; 11(1): 140 –145. DOI: 10.1111/jsm.12349. 12 Health Informatics Journal 7. Fritzer N, Haas D, Oppelt P , et al. More than just bad sex: sexual dysfunction and distress in patients with endometriosis. Eur J Obstet Gynecol Reprod Biol2013; 169(2): 392–396. DOI: 10.1016/j.ejogrb.2013.04. 001. 8. V ercellini P , Meana M, Hummelshoj L, et al. Priorities for endometriosis research: a proposed focus on deep dyspareunia. Reprod Sci 2011; 18(2): 114 –118. DOI: 10.1177/1933719110382921. 9. NW 1615 L. St, Washington S 800, Inquiries D 20036 U 419 4300 | M 857 8562 | F 419 4372 | M. Health Topics. Washington, DC: Pew Research Center: Internet, Science & Tech, 2011. https://www. pewresearch.org/internet/2011/02/01/health-topics-4/ 10. Government of Canada SC. Canadian internet use survey (CIUS). Canada: Government of Canada, 2023, Published September 8, 2020. https://www23.statcan.gc.ca/imdb/p2SV .pl?Function=getSurvey&SDDS= 4432 (Accessed 8 February 2023). 11. Eysenbach G and Kohler C. What is the prevalence of health-related searches on the World Wide Web? Qualitative and quantitative analysis of search engine queries on the Internet. AMIA Annu Symp Proc 2003; 2003: 225 –229. 12. Redston S, de Botte S and Smith C. Resolving embarrassing medical conditions with online health information. Int J Med Inf 2018; 114: 101 –105, DOI: 10.1016/j.ijmedinf.2018.03.010. 13. Holowka EM. Mediating pain: navigating endometriosis on social media. Front Pain Res 2022; 3: 889990. DOI: 10.3389/fpain.2022.889990. 14. Tonsaker T, Bartlett G and Trpkov C. Health information on the Internet: gold mine or mine field? Can Fam Physician 2014; 60: 407 –408. 15. Tan SSL and Goonawardene N. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res 2017; 19(1): e9. DOI: 10.2196/jmir.5729. 16. Nasser S, Mullan J and Bajorek B. Assessing the quality, suitability and readability of internet-based health information about warfarin for patients. Australas Med J 2012; 5(3): 194–203. DOI: 10.4066/AMJ. 2012862. 17. Hirsch M, Aggarwal S, Barker C, et al. Googling endometriosis: a systematic review of information available on the Internet. Am J Obstet Gynecol 2017: 216: 451 –458. DOI: 10.1016/j.ajog.2016.11.1007. 18. Fahy E, Hardikar R, Fox A, et al. Quality of patient health information on the Internet: reviewing a complex and evolving landscape. Australas Med J 2014; 7(1): 24 –28. DOI: 10.4066/AMJ.2014.1900. 19. Lim MSC, Molenaar A, Brennan L, et al. Young adults ’ use of different social media platforms for health information: insights from web-based conversations. J Med Internet Res 2022; 24(1): e23656. DOI: 10. 2196/23656. 20. Abdulai AF, Howard AF, Yong PJ, et al. Developing an educational website for women with endometriosis-associated dyspareunia: usability and stigma analysis. JMIR Hum Factors 2022; 9(1): e31317. DOI: 10.2196/31317. 21. de Oliveira Silva D, Rathleff MS, Holden S, et al. Patients and clinicians managing patellofemoral pain should not rely on general web-based information. Phys Ther Sport 2020; 45: 176 –180. DOI: 10.1016/j. ptsp.2020.07.004. 22. Charnock D, Shepperd S, Needham G, et al. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999; 53(2): 105–111. 23. Doak CC, Doak LG and Root JH. Teaching patients with low literacy skills. AJN Am J Nurs 1996; 96(12): 16M. 24. Helitzer D, Hollis C, Cotner J, et al. Health literacy demands of written health information materials: an assessment of cervical cancer prevention materials. Cancer Control 2009; 16(1): 70 –78. DOI: 10.1177/ 107327480901600111. 25. Silberg MW, Lundberg GD and Musacchio RA. Assessing, Controlling, and Assuring the Quality of Medical Information on the Internet: caveant Lector et Viewor —let the Reader and Viewer Beware . Abdulai et al. 13 Chicago: JAMA | JAMA Network, 1997. https://jamanetwork.com/journals/jama/article-abstract/415407 (Accessed 8 February 2023). 26. Silberg WM, Lundberg GD and Musacchio RA. Assessing, Controlling, and Assuring the Quality of Medical Information on the Internet: caveant Lector et Viewor—let the Reader and Viewer Beware. JAMA 1997; 277(15): 1244 –1245. DOI: 10.1001/jama.1997.03540390074039. 27. Kincaid J, Fishburne R, Rogers R, et al. Derivation of new readability formulas (automated readability index, fog count and Flesch reading Ease formula) for navy enlisted personnel. Inst Simul Train. 1975; 51. https://stars.library.ucf.edu/istlibrary/56. 28. Koo TK and Li MY . A guideline of selecting and reporting intraclass correlation coef ficients for reliability research. J Chiropr Med 2016; 15(2): 155 –163. DOI: 10.1016/j.jcm.2016.02.012. 29. V elarde MC, Bucu MEM and Habana MAE. Endometriosis as a highly relevant yet neglected gynecologic condition in Asian women. Endocr Connect 2023; 12(11): e230169. DOI: 10.1530/EC-23-0169. 30. Wahl KJ, Imtiaz S, Lisonek M, et al. Dyspareunia in their own words: a qualitative description of endometriosis-associated sexual pain. Sex Med 2020; 9(1): 100274. DOI: 10.1016/j.esxm.2020.10.002. 31. Goff AJ, Barton CJ, Merolli M, et al. Comprehensiveness, accuracy, quality, credibility and readability of online information about knee osteoarthritis. Health Inf Manag 2023; 52: 185 –193, DOI: 10.1177/ 18333583221090579. 32. Labinjo T. A review of the prevalence of endometriosis in African women. J Womens Health Issues Care 2020; 9(4): 2325 –9795. https://www.scitechnol.com/peer-review/a-review-of-the-prevalence-of- endometriosis-in-african-women-ldVb.php?article_id=13039 (Accessed 8 February 2023). 33. Crocco AG, Villasis-Keever M and Jadad AR. Analysis of cases of harm associated with use of health information on the internet. JAMA 2002; 287(21): 2869 –2871. DOI: 10.1001/jama.287.21.2869. 34. Balkac M and Ergun E. Role of infographics in healthcare. Chin Med J 2018; 131(20): 2514–2517. DOI: 10.4103/0366-6999.243569. 35. Birru MS and Steinman RA. Online health information and low-literacy african Americans. J Med Internet Res 2004; 6(3): e26. DOI: 10.2196/jmir.6.3.e26. 36. Friedman DB and Kao EK. A comprehensive assessment of the dif ficulty level and cultural sensitivity of online cancer prevention resources for older minority men. Prev Chronic Dis 2008; 5(1): A07. 37. Andreassen HK, Bujnowska-Fedak MM, Chronaki CE, et al. European citizens ’ use of E-health services: a study of seven countries. BMC Publ Health 2007; 7: 53. DOI: 10.1186/1471-2458-7-53. 38. Abdulai AF, Howard AF and Currie LM. Stigmatizing and de-stigmatizing properties of web apps for sexual health-related conditions: a scoping review. Int J Sex Health , 2022; 34: 144 –159. DOI: 10.1080/ 19317611.2021.1949655. 39. Orr N, Wahl K, Joannou A, International Society for the Study of Women ’s Sexual Health ’s ISSWSH Special Interest Group on Sexual Pain, et al.. Deep dyspareunia: review of pathophysiology and proposed future research priorities. Sex Med Rev 2020; 8(1): 3 –17. DOI: 10.1016/j.sxmr.2018.12.007. 40. Sorensen J, Bautista KE, Lamvu G, et al. Evaluation and treatment of female sexual pain: a clinical review. Cureus 2018; 10(3): e2379. DOI: 10.7759/cureus.2379. 41. Chang MC, Chen YC, Gau BS, et al. Translation and validation of an instrument for measuring the suitability of health educational materials in taiwan: suitability assessment of materials. J Nurs Res 2014; 22(1): 61 –68. DOI: 10.1097/jnr.0000000000000018. 42. Clayton LH. TEMPtEd: development and psychometric properties of a tool to evaluate material used in patient education. J Adv Nurs 2009; 65(10): 2229 –2238. DOI: 10.1111/j.1365-2648.2009.05049.x. 43. Logullo P , Torloni MR, de O C Latorraca C, et al. The Brazilian Portuguese version of the DISCERN instrument: translation procedures and psychometric properties. V alue Health Reg Issues 2019; 20: 172–179. DOI: 10.1016/j.vhri.2019.09.001. 14 Health Informatics Journal 44. Khazaal Y , Chatton A, Cochand S, et al. Brief DISCERN, six questions for the evaluation of evidence- based content of health-related websites. Patient Educ Counsel 2009; 77(1): 33 –37. DOI: 10.1016/j.pec. 2009.02.016. 45. Reynolds TL. Designing health interface technologies to support patients ’ and healthcare providers ’ individual and collaborative work. In: 2020 IEEE international conference on healthcare informatics . Hull: ICHI 2020, 2020. Abdulai et al. 15