{"paper_id":"f973ddca-1bec-450c-a16d-84a4b2f99301","body_text":"Total hip replacement (THR) and total knee replacement (TKR) are proven interventions for patients with advanced arthritis, and are among the most common elective surgical procedures carried out in the UK and Ireland\n 1 . National Joint Registry data for both the UK and Ireland reveals that close to 200,000 total hip and knee replacements are performed each year\n 2 ,\n 3 . Demand for both THR and TKR is set to increase dramatically in the coming decades due to changing demographics and an ageing population, with studies suggesting the demand for TKR in the United States will grow by 673% by 2030\n 4 –\n 6 . Anaesthesia can play a significant role in reducing perioperative morbidity, and in an increasingly complex patient population it is important that patients are given accurate and up to date information about the various anaesthetic techniques used\n 7 .\nInternet use is increasing worldwide, with 85% of adults in the United States using the Internet\n 8 . In the UK, the Oxford Internet Survey group stated that in 2013, 78% of people used the Internet and, of these, up to 71% sought health related information\n 9 . Patient education materials (PEM) can be beneficial for assisting patients in the informed consent procedure, by explaining indications, risks, benefits and alternatives\n 10 . A\n recent online poll  revealed that up to 90% of patients who access the Internet for their health information believe it to be accurate, and over 60% reported that it impacted their medical decision making\n 11 . However, the Internet is a completely unregulated source susceptible to provider bias and has the capacity to negatively influence consumer health outcomes\n 12 . Implementing and enforcing standards is very difficult, and health information available has been shown to be of poor quality and largely unreliable\n 8 ,\n 11 ,\n 13 ,\n 14 . It is important for doctors to be aware of the information available to patients on the Internet and to understand confusion surrounding such information. As well as supplying high quality accurate health information, the readability of the website must be suitable for the target audience. Several medical organisations ,\n including the National Institute of Health  (NIH), and the American Medical Association (AMA)\n recommend that all PEM  should be written at or below sixth grade level (reading age 11-12 years) in order to be effectively understood by the general public\n 15 . However many previous studies have shown that a significant proportion of health information websites are written above this recommended level, suggesting that it may be beyond the comprehension of a substantial proportion of the patient population accessing it\n 16 –\n 18 .\nOver the past number of years there has been several studies assessing both the quality and readability of PEM available on the Internet across all medical specialties, including general and regional anaesthesia for labour and pain procedures\n 19 –\n 22 . We also already know that orthopaedic patients research their conditions extensively online\n 11 ,\n 13 ,\n 23 ,\n 24 , but to our knowledge none of these studies have looked at the availability of high quality health information relating to anaesthesia for common surgical procedures. Therefore, our aim was to assess the readability and quality of patient information available on the Internet relating to anaesthesia for both TKR and THR.\n\nAccording to the policy activities that constitute research at the institute in question, this work met criteria exempt from ethics review.\nOn 26/09/2017 the search terms ‘hip replacement anaesthetic’ and ‘knee replacement anaesthetic’ were entered into the\n top three most commonly used search engines  for 2017: Google, Bing and Yahoo. Most Internet users do not go beyond the first three pages of returned searches\n 22 , so we only included those websites in the analysis; 27 sites for each of the above terms on both Google and Bing (9 websites per page) and 30 for each on Yahoo (10 websites per page), giving a total of 168. Websites were then excluded from further analysis if they were not PEM, if they were written in a language other than English, if they were inaccessible, or in a non-written format, i.e. video. Duplicate websites were also excluded. In total 32 unique websites were identified for examination, as shown in\n Table 1 .\nWebsites listed by search engine, authorship group and the presence or absence of the HONcode and/or Information Standard NHS quality marks. NPO, Not for profit organisations.\nWebsite authorship was determined independently by close examination by the first two authors (R.M, E.P.) and each one was placed in one of the following categories: 1) physician, author or authors were individual or group physicians with no university or research group; 2) academic, author or authors were affiliated with a university or research group; 3) commercial site, author or authors were marketing a product related to the subject; 4) commercial/physician, author or authors were individual or group physicians also marketing a product related to the subject; 6) Government/Not for profit organisations (NPO), author or authors were affiliated with a government or registered charity; 7) media-related, author or authors affiliated with the media; and 7) social/discussion, to reflect the growing trend of the use of these modalities to distribute information\n 12 .\nThe readability of a text is determined as the education level a person completed to understand the written material, based on the US reading grade level\n 10 . We assessed the readability of each website using three validated, commonly used readability assessment tools: the Flesch Reading Ease (FRE) score, the Flesch-Kincaid grade level (FKGL), and the Gunning Fog Index (GFI). The FRE score generates a score between 0–100, using the formula: 206.835 − 1.015(total words/total sentences) − 84.6(total syllables/total words). It is based on the total words, syllables and sentences in a written passage and a score <60 considers the document to be difficult to read by the general public\n 8 ,\n 19 . The FKGL corresponds to the US reading grade level and is calculated using the formula: 0.39(total words/total sentences) + 11.8(total syllables/total words) − 15.59. The GFI is calculated using the formula: 0.4[(words/sentences) + 100(complex words/words), and estimates the number of years of formal education required to read a passage of text\n 8 . Readability scores for all PEM websites were generated using an\n online readability calculator .\nThe quality of each website was assessed by the two authors using the DISCERN instrument, and according to the presence or absence of the Health On the Net (HON) Foundation seal and the Information Standard mark. The DISCERN instrument is a validated rating tool of the quality of health information developed by the NHS Executive Research and Development Programme. It consists of 15 key questions plus an overall quality rating and generates a score between 80 (highest) and 16 (lowest), a lower score being reflective of a website that is of poor quality information on treatment choices\n 25 . The\n HON Foundation criteria  was developed in 1995 by a non-profit, non-governmental organisation, accredited to the Economic and Social Council of the United Nations, in an attempt to improve the quality of internet-based health information. The HON Foundation provides a code of conduct seal for websites that meet its quality and reliability standards\n 8 ,\n 14 . The\n Information Standard quality mark  was established by the UK Department of Health to help patients and the public make informed choices about their lifestyle, condition and options for treatment and care. It is a certification scheme for health and social care information, which indicates that an organisation is a reliable source of health and social care information.\nCalculations were performed using SPSS version 23 (SPSS, Inc., Armonk, NY). Mean scores and standard deviation are presented for normally distributed variables. Median values and standard deviation are presented for non-normally distributed data. One-way ANOVA/Independent samples Kruskal-Wallis Test were used as appropriate in intergroup comparisons. In comparisons between certified and noncertified groups, independent samples T test/Mann Whitney U test were applied as appropriate. Significance was set at\n P < 0.05  for all studies.\n\nOut of the 168 initial search results, 32 were analysed further as per the previously described exclusion criteria (\n Figure 1 ). Each website was categorised according to authorship; seven were academic, seven were commercial/physician, five were discussion/social media related, four were physician, four were Government/NPO related, four were commercial and one was media-related (\n Figure 2 ). Only 8/32 sites (25%) were HONcode certified and 5/32 (15.6%) had the Information Standard quality mark.\n27 sites for each of the above terms on both Google and Bing (9 websites per page) and 30 for each on Yahoo (10 websites per page), giving a total of 168. Websites were then excluded from further analysis if they were not PEM, if they were written in a language other than English, if they were inaccessible, or in a non-written format i.e. video. Duplicate websites were also excluded. In total 32 unique websites were analysed further.\nSeven were academic, seven were commercial/physician, five were discussion/social media related, four were physician, four were Government/NPO related, four were commercial and one was media-related.\nThe readability of each website was assessed using three validated commonly used readability assessment tools: the Flesch Reading Ease (FRE) score, the Flesch-Kincaid grade level (FKGL), and the Gunning Fog Index (GFI).\n Table 2  summarises the study’s readability and quality scores. The overall mean readability scores indicated that the websites as a group were difficult to read. The mean FRE score was 55.2 ±12.8, with social/discussion networks associated with both the minimum (3.3) and maximum (74.2) FRE scores. The mean FKGL score was 8.6 ±1.9, with only six websites (18.8%) having the recommended readability level of sixth-grade or less (\n Figure 3 ). Overall, a mean of 10.4 years (mean GFI 10.4 ±2.6) of formal education was required to read the websites included in this study. Commercial websites had the highest mean GFI (11.7 ±2.3), while social/discussion networks had the lowest (8.3±3.9).\nOverall results for each scoring system, presented as mean ±standard deviation for normally distributed data and median ±standard deviation for non-normally distributed data (FRE Score). FRE, Flesch Reading ease; FKGL, Flesch-Kincaid Grade Level; GFI, Gunning Fox Index; DISCERN, DISCERN instrument.\nThe line denotes the recommended readability level of the sixth-grade level with a minority of websites falling at or below this level.\nOrganisations that have achieved the Information Standard quality mark were as a group easier to read. This was seen across all three readability assessment tools. Member websites had a significantly lower mean FKGL (6.2 vs. 9,\n P < 0.001 ) and GFI (8.8 vs. 10.7,\n P = 0.04 ) and a significantly higher median FRE score (64.2 vs. 9,\n P = 0.02 ) than non-member websites. There was no difference in FKGL (8.5 vs. 8.6,\n P = 0.78 ), GFI (10.8 vs. 10.3,\n P = 0.92 ) or FRE (51.2 vs. 56.4,\n P = 0.31)  between HONcode certified and noncertified groups.\n\nThe DISCERN instrument was used to assess each website. Overall, the mean DISCERN score was low, 40.3 ± 13 out of a maximum of 80. Three of the top five scoring websites were of academic authorship with one being of physician authorship and one Government/NPO. Eight websites (25%) scored 51 or above, representing good quality, with academic authorship associated with both the single highest DISCERN score (61) and the highest mean DISCERN score across all groups (49.7). Six websites (18.75%) scored less than 26 points, representing very poor quality with extensive shortcomings. Average DISCERN scores by authorship groups are shown in\n Figure 4 . Neither HONcode nor the presence of the Information Standard quality mark was associated with a higher mean DISCERN score (\n P=0.08  and\n P=0.7 , respectively). Academic and physician-related websites achieved significantly higher mean DISCERN scores than social/discussion networks (\n P = 0.005  and\n P = 0.032 , respectively;\n Figure 5 ).\nFrom highest (61) to lowest (20) mean DISCERN score. Error bars denote 95% confidence interval.\nSignificantly higher for academic (\n P = 0.005 ) and physician-related (\n P = 0.032)  websites versus the mean DISCERN score for social/discussion networks.\n\nThere can be no doubt that the Internet has changed the manner in which patients access information. Traditionally, information was passed from doctor to patient in a single direction and decisions were made under the paternalistic guidance of the doctor. In the Internet era, in which information is immediately available, this flow of information is no longer appropriate, nor is it acceptable to patients. More and more, patients are accessing this medical information online and using it to make decisions regarding their own healthcare\n 8 ,\n 18 . However, the reliability and suitability of these online patient education materials is increasingly being called into question\n 8 ,\n 13 ,\n 14 ,\n 16 –\n 18 . The majority of Internet users start their search with a search engine, and most do not trawl beyond the websites from the first three pages returned\n 22 . The aim of this study was to assess both the quality and the readability of Internet information relating to anaesthesia for total hip replacement and total knee replacement, using three validated tools to assess readability (FRE, GFI, FKGL) and the DISCERN instrument to assess quality of information obtained. We also looked for websites that displayed HONcode certification or had received the Information Standard NHS quality mark.\nOur results show that 81.2% of the websites assessed were above the recommended sixth-grade readability level for PEM. This should be a concern for healthcare providers; many patients will have a limited understanding of the health information available to them online, and thus even those patient education materials that may be of good quality may not be understood. This could have an adverse effect on the informed consent and decision-making process. These findings echo multiple studies over the last decade, suggesting that producing information at an appropriate readability level is still a challenge for healthcare providers\n 10 ,\n 18 ,\n 19 ,\n 21 . Encouragingly, our study found that websites that had been awarded the Information Standard NHS quality mark were statistically significantly more likely to achieve the appropriate readability level, suggesting that these are the websites that healthcare providers should be recommending to our patients.\nThe quality of the websites was assessed using the DISCERN instrument. It is important to note that the DISCERN instrument does not take into account the readability of the material, and thus when recommending websites to patients, healthcare providers should seek websites which are of both high quality and appropriate readability. In our study, only a small number of the websites analysed (25%) scored highly using the DISCERN instrument, which indicates that most PEM related to anaesthesia for TKR and THR available on the Internet are of poor quality. Websites were more likely to achieve high DISCERN scores if the authors were physicians, affiliated with academic institutions or government agencies. Again, this highlights the importance of healthcare professionals directing patients towards more reliable and appropriate PEM.\nOne of the most significant and disappointing findings from our study relates to the presence or absence of the HONcode seal on websites. Although the HONcode seal indicates that a website has met certain quality and reliability standards, our study did not find that HONcode certified websites achieved higher readability or quality standards than those without the HONcode seal. The Information Standard quality mark was introduced to help patients make informed choices about their condition and options for treatment. In our study, we found that although there was a statistically significant relationship between the presence of the Information Standard quality mark and appropriate reading levels, there was no correlation between this quality mark and quality of information using the DISCERN instrument.\nA number of limitations to this study are recognised. We performed our online search in one country, and only analysed PEM from websites written in the English language. The search was limited to the first three pages of returned websites, as it has been shown previously that the general public usually don’t pursue beyond this\n 22 . We acknowledge that comprehension of healthcare information is not solely related to readability of text and that other factors, i.e. videos and visual tools, can contribute greatly to a patient’s understanding. The examination of such materials was beyond the scope of this study and previous studies have also acknowledged this limitation\n 8 ,\n 16 . While readability indices have been validated in the literature, there is no general consensus on which index is best and each one uses a different formula to calculate readability. Scores by different indices may vary substantially. It should also be noted that although there is a large volume of material available to guide users when appraising websites using the DISCERN instrument, there is still the potential for variability among raters, which is a limitation not present when assessing readability.\nIn conclusion, we aimed to assess the quality and readability of information available online regarding anaesthesia for total knee replacement and total hip replacement. Overall, we found that most of the websites were of poor quality and many had reading levels which were too high for the target audience. These findings echo many other studies that examine online information relating to healthcare\n 8 ,\n 11 ,\n 13 ,\n 16 ,\n 17 ,\n 19 ,\n 20 . We found that while the Information Standard NHS quality mark was associated with improved readability, neither the quality mark nor the HONcode were found to have a statistically significant correlation with quality of material. Based on this study we would encourage doctors and other healthcare professionals to be judicious in the websites they recommend to patients, and to consider both the readability and the quality of the information provided.\n\nFigshare. WorkbookFinalPEM.xlsx. Data figures relating to an internet-based readability and quality control study using recognised quality scoring systems to assess the patient information available online for anaesthesia for total hip and knee replacement surgery.\n https://doi.org/10.6084/m9.figshare.7940753.v1 \n 26 .\nData are available under the terms of the\n Creative Commons Attribution 4.0 International license  (CC-BY 4.0).","source_license":"CC-BY-4.0","license_restricted":false}