Lasting effects of the COVID-19 pandemic on patient satisfaction of English dental service: Sentiment analysis and topic modelling of online reviews

Lasting effects of the COVID-19 pandemic on patient satisfaction of English dental service: Sentiment analysis and topic modelling of online reviews | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Lasting effects of the COVID-19 pandemic on patient satisfaction of English dental service: Sentiment analysis and topic modelling of online reviews Ali Feizollah, Matthew Byrne This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6503136/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Aims This study aimed to determine how changes in English dental practice during and after the COVID pandemic impacted patient perceptions of dental care delivery use sentiment analysis and topic extraction of online reviews Methods Reviews were collected from the NHS website using webscraping and the NHS digital API. Reviews were grouped into pre-, during-, and post pandemic phases, and subcategorised to align with the dates of national standard operating procedures. The AWS Detect Sentiment API categorized each review by sentiment and Non-Negative Matrix Factorization was used to identify topics in positive and negative reviews. Results 48,862 reviews were analysed. Reviews were consistently positive (81.52% pre-; 82.88% during, 82.48% post-) The proportion of Negative reviews decreased (13.87% pre- ,11.54% during-, 9.72% post-). For negative reviews, staff behaviour and professionalism prevalent topics in all time periods. During the pandemic, topics of treatment costs and poor communication emerged. In positive reviews, patient comfort and anxiety management were consistently identified. Appreciation for COVID-19 precautions emerged as a new theme during the pandemic. Conclusions Unstructured patient feedback is a rich data source to assess patient experiences. This research suggests appointment management, communication, and staff interactions are priorities for quality improvement in NHS dentistry. Health sciences/Health care/Dentistry/Dental public health/Dental epidemiology COVID-19 Dental Care Patient Satisfaction Natural Language Processing Health Services Accessibility Sentiment Analysis Health Policy Quality Improvement Internet-Based Intervention Health Communication Figures Figure 1 1. Introduction Patient feedback serves as a critical measure of service quality. This feedback may be collected through structured means such as predetermined questionnaires, or through Unstructured Patient Feedback (UPF). UPF typically comprises "free text" comments of a patient’s unique care experience. Manual methods of gathering and processing UPF are time and labour intensive, creating delays between feedback and action. 1 Online reviews have increased the accessibility of UPF. For practices, it offers insights into patient’s perceptions that can help guide quality improvement; for patients it may inform decisions about which providers they choose to access. 2 In England, various online platforms host online reviews of dental practices 3 , including NHS.uk. NHS.uk has pages for every dental practice holding an NHS contract in England. Through its "Leave a review" function, patients can rate practices on a 1–5 star scale and provide UPF. Contemporary Natural Language Processing (NLP) offers powerful methods to analyse large volumes of textual data algorithmically. Two particularly valuable techniques for analysing UPF are Sentiment Analysis (SA), which describes whether a text is broadly positive or negative in sentiment 4 , and Topic Modelling (TM), which identifies salient themes discussed within text collections. 5 When applied to UPF over time, these techniques can reveal changing patient sentiments toward a service and the underlying reasons for these changes. Bahja and Lycett have demonstrated the utility of these technologies in predicting patient sentiment and identifying key themes within NHS Choices website feedback. 6 The global COVID-19 pandemic provides a unique backdrop to assess whether the sentiments and content of UPF respond to external influences and whether NLP techniques can effectively capture these changes. Primary dental care in England was particularly affected by the pandemic, making it an ideal context for studying shifts in patient feedback under extraordinary circumstances. Between March 2020 and August 2022, NHS dental practices in England operated under six distinct phases of care provision, ranging from complete closure with only emergency care available through designated centres to gradual reopening with strict protocols, until the eventual return to full activity. 7 , 8 , 9 , 10 , 11 The COVID-19 pandemic has not only affected access to dental care but may have also impacted patients' perceptions and experiences of the care they receive. 12 By comparing online reviews before, during, and after the pandemic, this study aims to uncover changes in patient satisfaction and identify contributing factors. This information can help stakeholders improve dental care services. While previous studies 13 , 14 have explored text mining in healthcare feedback, limited research examines longitudinal trends in patient sentiment specific to NHS dentistry during the COVID-19 pandemic. The objectives of this study are to: (1) assess the overall sentiment of online reviews of NHS dental practices in England before, during, and after the COVID-19 pandemic, and (2) to identify themes and factors contributing to patient satisfaction or dissatisfaction during these periods. We hypothesize that patient satisfaction may have decreased during the pandemic compared to the pre-pandemic period due to access challenges and service delivery changes. 2. Materials and Methods Data Collection Data for this study were sourced from publicly available patient reviews on the NHS.uk website, in two rounds of collection. In the first round of data collection, a Python 3 script equipped with the BeautifulSoup library 15 , was used to collect the review texts, date of visit, and star ratings for each dental practice in England. The data collection was constrained to the first 10 reviews per dental practice due to webscraper limitation. Reviews were collected from March 2019 to June 2022. In the second round of data collection, Python script was developed to query the NHS Developers API 16 , and identify dental practices by their national contract identifier (V-code). All available reviews of dental practices between April 2022 and March 2024 were collected to minimize the potential influence of the COVID-19 pandemic on the data, thereby allowing for a more accurate after-pandemic analysis. Data Arrangement Between March 2020 and August 2022, NHS dental practices in England operated under six distinct phases of care provision, as outlined by Standard Operating Procedures (SOPs) from the Office of the Chief Dental Officer (OCDO). 17 These phases represented meaningful time periods with distinct service delivery conditions, summarized in Table 1 . Table 1 Timeline of phased reopening and operational requirements for dental practices during and after the COVID-19 pandemic Phase Dates Description 1 25/3/2020–07/06/2020 General Practices closed. Emergency care in Urgent Dental Care centres only 2 08/06/2020–31/12/2020 General practices open Strict standard operating procedures commenced Limited AGP with 60 mins fallow time 20% contracted activity 3 01/01/2021–31/03/2021 Requirement of practices to meet 45% of contracted activity 4 01/04/2021–31/12/2021 Requirement of practices to meet 60% of contracted activity. Checkups and emergency care prioritised 5 01/01/2022–30/06/2022 Requirement of practices to meet 85% of contracted activity 6 01/08/2022 - 100% contracted activity, removal of all restrictions In Phase 1 (March-June 2020) dental practices were closed, with emergency care available only through designated urgent dental care centres; Phase 2 (June-December 2020) marked the gradual reopening with strict SOPs that including limitation of aerosol-generating procedures, increased used of personal protective equipment, and extended fallow periods after treating patients; subsequent phases saw incremental increases in activity targets from 45% (Phase 3: January-March 2021) to 60% (Phase 4: April-December 2021) to 85% (Phase 5: January-June 2022), until full activity resumed with the removal of all restrictions in August 2022 (Phase 6). 7 , 8 , 9 , 10 , 11 These SOPs, summarised in Table 1 , limited the amount and types of treatment that practices could offer. 7 Table 2 presents how our collected data corresponds to these pandemic phases with their corresponding dates, data collection methods, and corpus characteristics. Table 2 Data collection periods and sources corresponding to COVID-19 pandemic phases Period Phase Dates Source Corpus content Pre-pandemic 0 01/03/2019–28/02/2020 Webscraper Partial Corpus During Pandemic 1 25/03/2020–07/06/2020 Webscraper Partial Corpus 2 08/06/2020–31/12/2020 Webscraper Partial Corpus 3 01/01/2021–31/03/2021 Webscraper Partial Corpus 4 01/04/2021–31/12/2021 Webscraper Partial Corpus 5 01/01/2022–30/06/2022 Webscraper Partial Corpus Post-pandemic 6 01/03/2023–31/03/2024 NHS API All reviews Whilst variations on the dental contract continued until August 2022, review data from this source was not available. Sentiment Analysis Prior to analysis, simple cleaning of the review text performed to remove any aberrant punctuation resulting from the scraping process. Each review was presented as a separate text file to the Amazon Web Services (AWS) Detect Sentiment API 18 , which provided a categorial output of one of four sentiments: Positive, negative, neutral and mixed. This commercially available tool was selected for its ease of integration and scalability. 19 AWS has been demonstrated to have a good associated with human reviewers in identifying the sentiment of dental reviews. 20 Sentiment outputs were then collated per time period. Topic Identification Topic extraction from the text corpora were performed using Non-Negative Matrix Factorization (NMF) 21 . NMF was chosen for its effectiveness in identifying interpretable topics from large text corpora. 22 Topic extraction was performed following sentiment analysis. Reviews assigned a positive or negative sentiment were included for topic extraction in order to identify what factors caused positive or negative responses. Prior to identifying the topics, we performed text pre-processing steps using Python and relevant NLP libraries comprising: Expanding contractions to their full forms using a Python script (e.g. don’t to do not). Converting text to lowercase using Python’s built-in ‘lower’ method. Splitting the text into individual words (tokenisation) using ‘word_tokenize’ function from the ‘NLTK’ library. 23 Removal of non-alphabetic characters using ‘regex’ in Python. Reduction of suffixed/prefixed words to their root form (lemmatisation) using the WordNet Lemmatizer 24 from the ‘NLTK’ library (e.g. eating and ate to eat). Topics were extracted separately for pre, during, and after the pandemic periods, and the top keywords for each topic were identified. In the context of the NMF model, each review is represented as a combination of different topics; each topic has a certain weight indicating its presence within that review. We decided on the number of topics in the NMF by checking coherence scores and manually evaluating how distinct the topics were. Then, using Nonnegative Double Singular Value Decomposition (nndsvd) initialization 25 , the model parameters were iteratively refined until they converged, with a maximum of 500 iterations to ensure convergence. The keywords within the computed topics were analysed by the authors to name and validate these topics. A prevalence score was then calculated to serve as a quantitative measure of how dominant or frequently discussed a topic was within the dataset. Thus, topics with higher prevalence scores appeared more consistently and were more commonly discussed by users. To account for the variation in the number of reviews received in each time period, topic prevalence was standardised within each time period using min-max normalisation. 26 , whereby a topic’s prevalence score is divided against the maximum prevalence score to rescale the scores within each period to a range of 0 to 1. This preserved the relative differences between topics within each period, while also allowing for direct comparisons across different periods. For example, if during a certain period the highest (raw) prevalence score is 50 for Topic A, and Topic B and Topic C have scores of 40 and 25 respectively, the normalised scores would be 1.00 (50/50), 0.80 (40/50), and 0.50 (25/50). Similar normalised scores (e.g., 1.00 and 0.95) indicate comparable levels of discussion, whereas a larger gap (e.g., 1.00 versus 0.50) indicates a more dominant topic relative to the others. 3. Results Webscraping of reviews during the period March 2019 - June 2022 identified 24,304 reviews. 150 could not be attributed to a dental practice site and were excluded from analysis. As the webscraper used was limited to collecting only the 10 most recent reviews for each practice, any practice with > 10 reviews will not have all of their information represented. Of the 24,154 reviews, 16,308 were from the pre-pandemic period of March 2019 - February 2020, and 7,846 were from during the pandemic (March 2020 – June 2022). A call to the NHS.uk API made in April 2024 provided 33,242 reviews from April 2022 to April 2024. For the purposes of analysis, a 13-month period from March 2023 to March 2024 was selected as a representative sample containing 24,595 reviews. These reviews represented all feedback received by all practices during this period. A total of 37 reviews were excluded due to duplication or being too short (less than 3 words), leaving 24,558 reviews for analysis. Sentiment analysis The results, which is tabulated in Table 3 , show a predominance of positive sentiments. Table 3 Proportions of sentiments of reviews by time periods (in percentage). Target refers to activity levels expected in OCDO SOPs Pre-pandemic (100% target) Phase 1 (Closedown) Phase 2 (20% target) Phase 3 (45% target) Phase 4 (60% target) Phase 5 (85% target) All during pandemic Post-pandemic (100% target) 3/19 − 2/20 25/3/20 − 7/6/20 8/6/20–31/12/20 1/1/21–31/3/21 1/4/21–30/6/21 1/1/22–30/6/22 25/3/20–30/6/22 3/23 − 3/24 Positive 81.52(n = 13295) 87.12(n = 176) 80.27(n = 932) 83.46(n = 1600) 84.89(n = 1664) 81.77(n = 2131) 82.88(n = 6503) 82.42(n = 20242) Negative 13.87(n = 2262) 7.42(n = 15) 14.55(n = 169) 12.25(n = 235) 10(n = 196) 11.16(n = 291) 11.54(n = 906) 9.72(n = 2388) Neutral 3.89(n = 636) 3.96(n = 8) 3.53(n = 41) 3.39(n = 65) 4.03(n = 79) 5.52(n = 144) 4.29(n = 337) 1.06(n = 262) Mixed 0.7(n = 115) 1.48(n = 3) 1.63(n = 19) 0.88(n = 17) 1.07(n = 21) 1.53(n = 40) 1.27(n = 100) 6.80(n = 1666) 100(n = 16308) 100(n = 202) 100(n = 1161) 100(n = 1917) 100(n = 1960) 100(n = 2606) 100(n = 7846) 100(n = 24558) 81.52% of the pre-pandemic reviews were positive (n = 13,295), while during the pandemic, the proportion of reviews with positive sentiments increased to 82.88% (n = 6,503). During the initial closedown (Phase 1) when only emergency services were likely available, the proportion of positive reviews peaked at 87.12% (n = 176/202). This proportion of positive reviews dropped to 80.27% in phase 2, then rose to 83.46% (n = 1,600) in Phase 3 and at 84.89% (n = 1,664) in Phase 4, before slightly reducing to 81.77% (n = 2,131) in Stage 4. The proportion of positive reviews post-pandemic was 82.42% which is higher than pre-pandemic, but lower than ‘all during pandemic’. Conversely, negative reviews decreased from 13.87% pre-pandemic to 11.54% during the pandemic, with the lowest negativity occurring during the closedown at 7.42% (n = 15). They decreased again to the lowest of 9.72% post-pandemic. Neutral reviews saw a minor increase from 3.89% pre-pandemic to 4.29% during the pandemic and then dropped to 1.06% post-pandemic. Mixed reviews, which encompass both positive and negative sentiments, increased from 0.7% pre-pandemic to 1.27% during the pandemic, and further to 6.8% post-pandemic. Figure 1 demonstrates the month-by-month variation of the proportion of reviews by each sentiment scores. 4. Qualitative analysis results 4.1. Negative reviews The identified topics, the keywords in each topic, and normalised prevalence for reviews assigned with negative sentiment, during each time period are shown in Table 4 . Table 4 Topics for negative reviews of pre, during, and post pandemic Topic Keywords Normalised Prevalence Pre-pandemic Staff behaviour and professionalism staff, rude, reception, unprofessional, unhelpful, way, bad, feel, attended, need 1.00 Waiting times minutes, late, time, waiting, seen, mins, wait, day, 30, arrived 0.946 Cancellations and appointment management cancelled, appointments, notice, hygienist, times, years, work, time, wait, months 0.792 Pain and emergency situations pain, emergency, days, help, called, got, later, day, antibiotics, extraction 0.746 Dental treatments and experiences filling, pain, treatment, teeth, went, felt, given, needed, left, weeks 0.699 Phone communication issues phone, answer, called, tried, day, make, rang, hold, rude 0.698 NHS vs. private treatment nhs, patients, private, treatment, taking, register, new, pay, dental, website 0.697 During pandemic Staff behaviour and professionalism rude, staff, reception, unprofessional, recommend, unhelpful, dentists, awful, poor, manager 1.00 Treatment costs and procedures treatment, pay, check, dental, money, filling, cost, teeth, ppe, needed 0.907 Cancellations and appointment management cancelled, appointments, booked, time, day, work, times, rebooked, months, covid 0.766 Phone communication issues phone, answer, message, tried, trying, service, times, make, contact, book 0.751 NHS vs. private treatment nhs, patients, private, taking, practise, new, covid, dental, website, poor 0.711 Waiting times and need to wait outside of clinic minutes, outside, late, 10, waiting, time, wait, arrived, 15, cold 0.688 Pain and emergency situations pain, left, got, called, emergency, filling, teeth, help, problem 0.674 Post-pandemic Staff behaviour and patient experience staff, nervous, dental, feel, reception, experience, going, bad, ease 1.00 Cancellations and appointment management cancelled, appointments, phone, months, times, book, day, booked, answer, called 0.993 Dental hygiene and cleanings teeth, check, polish, clean, hygienist, went, scale, mouth, gums, felt 0.858 NHS vs. private treatment nhs, private, patients, pay, taking, years, new, afford, website, longer 0.838 Pain and emergency situations filling, pain, day, got, days, later, weeks, replaced, root, broken 0.739 Treatment and problem resolution treatment, problem, needed, given, required, seen, sorted, pay, received, time 0.689 Waiting times time, minutes, late, seen, wait, 10, waiting, 30, arrived, mins 0.667 As mentioned before, in our analysis, topic prevalence scores were normalized using min–max normalization such that the most prevalent topic in a given period is assigned a score of 1.00, with all other topics expressed as a ratio of this maximum value. Thus, when multiple topics exhibit similar normalized scores, it implies that they are discussed with comparable frequency. In contrast, a large difference in scores (high variance) between topics indicates that one theme is significantly more dominant than the others. For example, if Topic A has a score of 1.00, Topic B has a score of 0.95, and Topic C scores 0.50, then Topics A and B are nearly equal in their prevalence, while Topic C is less frequently mentioned. The topic modelling results reveal shifting patterns in patient concerns across three periods: pre, during, and post pandemic. Before the pandemic, the primary issues centred around staff behaviour and professionalism, with a normalised prevalence of 1.00. Patients frequently mentioned rudeness, unprofessional conduct, and unhelpful attitudes, particularly regarding reception staff. Waiting times and punctuality followed closely at 0.946, with patients expressing frustration about late appointments and extended waiting periods. Appointment scheduling and cancellations ranked third at 0.792, often involving issues with hygienist appointments and long wait times for bookings. During the pandemic period, staff behaviour and professionalism remained the top concern. However, treatment costs and procedures emerged as a significant issue, ranking second (0.907). This topic encompassed concerns about payments, dental check-ups, and notably, PPE requirements. Appointment-related problems persisted, with cancellations and re-bookings becoming more prominent (0.766). Communication difficulties, especially regarding phone services, also featured prominently (0.751). The post-pandemic period saw a similar pattern with staff behaviour and patient experience prevalence score of 1.00. Appointment scheduling and communication issues closely followed at 0.993. Dental hygiene services emerged as a distinct concern (0.858). The long-standing issue of NHS versus private practice gained prominence, reaching 0.838 prevalence. 4.2. Positive reviews The positive reviews of all three periods were selected for topic modelling analysis. Table 5 shows the topics, keywords, and the normalised prevalence score. Table 5 Topics for positive reviews of pre, during, and post pandemic Topic Keywords Normalised Prevalence Pre-pandemic Patient recommendations and dental anxiety management recommend, highly, ease, fear, surgery, years, dental, dentists, understanding, going 1.00 Routine dental check-ups and cleaning teeth, going, check, polish, scale, quick, mouth, given, thorough, cleaning 0.905 Pain management and dental procedures pain, abscess, infected, bad, extracted, went, experience, taken, antibiotics, say 0.824 Patient comfort and anxiety management nervous, feel, coming, time, bad, started, experience, visiting, staff, experiences 0.720 Emergency dental care and problem resolution problem, emergency, crown, sorted, day, time, went, checkup, getting, broken 0.622 Positive staff interactions and patient reassurance thank, scared, went, today, team, smile, saw, worried, attentive, step 0.622 NHS vs. private treatment nhs, treatment, dental, private, patients, needed, hesitation, work, ago, received 0.585 During pandemic Long-term dental care and fear management teeth, years, dentists, going, extremely, removed, forward, team, fear, look 1.00 Patient comfort and anxiety management nervous, really, feel, ease, bad, thank, understanding, highly, extraction, relaxed 0.937 Dental procedures and pain management filling, day, got, pain, surgery, days, called, went, service, later 0.842 COVID-19 precautions and professional care problem, seen, covid, quickly, explained, professionally, gum, precautions, given, caring 0.774 Staff professionalism and patient comfort staff, praise, way, reception, anxiety, problems, professional, ease, hygienist, practise 0.752 Appreciation for service during difficult times times, difficult, thank, time, service, especially, covid, excellent, amazing, complaints 0.698 Comprehensive dental treatment and patient care treatment, dental, check, time, received, nhs, took, surgeon, feel, extraction 0.651 Post-pandemic Service and professional care service, excellent, great, care, years, received, professional, recommend, highly, thank 1.00 Patient comfort and anxiety management feel, ease, nervous, comfortable, really, make, relaxed, lovely, felt, makes 0.994 Efficient and thorough appointments time, seen, teeth, took, check, pleasant, long, explain, waiting, wait 0.929 Staff friendliness and professionalism staff, friendly, helpful, reception, professional, polite, efficient, welcoming, pleasant, clean 0.911 Clear communication of treatment plans treatment, explained, options, clearly, going, pleased, fully, plan, excellent, step 0.888 Caring dental team dental, nurse, lovely, kind, team, care, experience, receptionists, thank, recommend 0.844 Quality dental advice and hygiene good, advice, experience, really, nice, given, clean, gave, hygienist, job 0.708 The topic modelling analysis of positive patient reviews in dental practices reveals evolving patterns across three distinct periods: pre-pandemic, during pandemic, and post-pandemic. In the pre-pandemic period, the most prevalent topic was "Patient Recommendations and Dental Anxiety Management" (1.00), followed closely by "Routine Dental Check-ups and Cleaning" (0.905). "Pain Management and Dental Procedures" (0.824) and "Patient Comfort and Anxiety Management" (0.720) were also significant themes. Less prevalent but still notable were "Emergency Dental Care and Problem Resolution" (0.622), "Positive Staff Interactions and Patient Reassurance" (0.622), and "NHS and Private Dental Treatment Options" (0.585). During the pandemic, the focus shifted slightly. "Long-term Dental Care and Fear Management" became the most prevalent topic (1.00), followed by "Patient Comfort and Anxiety Management" (0.937). "Dental Procedures and Pain Management" (0.842) remained important, while a new topic emerged: "COVID-19 Precautions and Professional Care" (0.774). "Staff Professionalism and Patient Comfort" (0.752), "Appreciation for Service During Difficult Times" (0.698), and "Comprehensive Dental Treatment and Patient Care" (0.651) rounded out the topics. In the post-pandemic period, there was a significant shift in topic prevalence and content. "Excellent Service and Professional Care" became the most prevalent topic (1.00), closely followed by "Patient Comfort and Anxiety Management" (0.994). "Efficient and Thorough Appointments" (0.929) and "Staff Friendliness and Professionalism" (0.911) also gained prominence. "Clear Communication of Treatment Plans" (0.888), "Caring Dental Team and Positive Experience" (0.844), and "Quality Dental Advice and Hygiene" (0.708) completed the post-pandemic topics. 5. Discussion This study analysed a 5-year corpus of NHS.uk dental reviews to assess how patient concerns evolved across pre-pandemic, during-pandemic, and post-pandemic periods. By examining reviews in relation to changing COVID-19 contractual arrangements, we can link patient experiences to policy decisions and evaluate the pandemic's lasting impact on dental service expectations. Sentiment scores remained consistently positive through all periods. An interesting finding was that the lowest proportion of negative reviews were seen around the first phase of the COVID response when access was poorest. This may suggest patient appreciation for receiving care despite restrictions, and could reflect dentists having more time to communicate with fewer patients they saw. Topic modelling revealed that interpersonal aspects of care dominated across both positive and negative reviews. Staff friendliness and professionalism became more prominent in positive reviews post-pandemic. This could reflect increased patient appreciation for dental staff working under challenging conditions, or improved staff-patient interactions as a result of adapted practices. Staff behaviour and professionalism dominated negative reviews during the pandemic potentially due to the increased stress and anxiety among both patients and staff during the pandemic. Staff interactions remained the most prominent theme of negative reviews post-pandemic suggesting an increased use of online review platforms to air dissatisfaction, or potentially an increase in conflict between patients and dental staff. The persistence of communication challenges, particularly phone-related issues, throughout all time periods resonates with previous studies. 27 This persistence suggests that dental practices may need to reevaluate and improve their communication strategies, especially considering the increased prevalence of appointment-related concerns post-pandemic. The pandemic likely exacerbated existing communication difficulties, with practices struggling to manage high volumes of cancellations, re-bookings, and patient inquiries. While communication issues persisted throughout all periods, the pandemic introduced unique themes absent pre- and post-pandemic. COVID-19 precautions were mentioned in both positive reviews—where patients appreciated the tangible commitments to safety—and in negative reviews, where social distancing led to the need to wait outside of the surgery. These findings align with previous research on mandatory enhanced PPE in dental practices during COVID-19. The study documented both positive feedback, such as an increased perception of safety, and negative impacts, including communication difficulties and discomfort for both patients and staff. 28 Furthermore, waiting times and punctuality were major concerns in negative reviews pre-pandemic, but decreased in importance during subsequent periods. This shift might reflect new appointment systems, reduced patient volumes, or these issues being overshadowed by more pressing safety and access concerns. Beyond service delivery changes, the pandemic also affected practice business models. Discussion of NHS versus private treatment options decreased in prominence from pre-pandemic (0.697) to post-pandemic (0.585). This could indicate shifts in practice models, changes in patient perceptions, or alterations in NHS services availability due to pandemic pressure. The lasting reduction of access to NHS dental services has been widely reported in the media. 27 Many dental services have handed back their dental contracts and are only offering care on a private basis or removing their NHS provision to patients. The changing dynamics between NHS and private practice, as reflected in the increasing prevalence of this topic suggest that patients sensed a systemic change in healthcare delivery. 29 The emergence of treatment cost themes in negative reviews during pandemic may reflect this conflict of NHS and private healthcare delivery. Patients likely faced new financial pressures and safety concerns, which were directly reflected in their feedback. Post-pandemic, the focus shifted toward dental hygiene services and anxiety management, suggesting both a routine care backlog and greater emphasis on overall patient experience. This finding aligns with broader observations about changing patient behaviours and expectations in healthcare delivery post-pandemic, where patients have increasingly prioritized safety, accessibility, and mental well-being in their healthcare experiences. 29 Several limitations of this study should be acknowledged. First, the data was sourced from online reviews on the NHS.uk website, which may not fully represent the experiences of all patients, especially those who do not use or have access to online review platforms. A previous study demonstrating the demographics of patients that leave online reviews suggests that this population skews to middle aged female patients, suggesting that this demographic may be overrepresented. 30 Second, despite over 7,000 NHS-contracted practices in England, many lacked online reviews, creating potential selection bias. Leaving an online review has several barriers to entry that must be overcome. This may suggest that those who are inclined to go to the effort have had a particularly positive or negative experience and thus may not reflect average experiences. Third, the AWS Detect Sentiment API, while robust, has limitations in capturing nuanced human language. 31 The large dataset used may mitigate the risk of ‘mis-categorisation’ of reviews into incorrect sentiment scores, however this error may be more prevalent in the time periods with fewer reviews. The use of an ‘off the shelf’ tool offers ease of use, but limits control over consistency of algorithms. 32 The striking reduction in negative sentiments and increase in mixed sentiments post-pandemic might reflect either changing algorithms 33 , or evolving patient review behaviours toward more nuanced expressions that don't fit neatly into positive/negative categories. 34 Conclusions This study has shown that the proportion of online reviews of NHS dental services with positive sentiment is generally high and remained high throughout the changing landscape of the COVID-19 pandemic. Whilst the proportion of negative reviews remained relatively constant pre- and during the pandemic, topic extraction has demonstrated changes in the factors that stimulated with key themes of dissatisfaction including pain and emergency situations, appointment cancellations, staff behaviour, NHS versus private treatment accessibility, waiting times, and phone communication issues. During the pandemic, new challenges emerged such as increased appointment cancellations due to COVID-19 restrictions, difficulties with waiting outside practices due to social distancing, and exacerbated phone communication issues. Despite these challenges, the high level of positive sentiment during the initial lockdown phase indicates robust patient appreciation for emergency dental services. Topic modelling of reviews demonstrated evolving patient priorities and appreciations, with a notable increase in the recognition of service quality, staff professionalism, and clear communication. Assessment of negative reviews suggest that dental practices should prioritise quality improvement efforts in appointment management, patient communication, stress management. Future research should expand data sources, incorporate qualitative methods, and analyse the long-term impacts of the pandemic on patient satisfaction. Declarations Data availability All data supporting the findings of this study are available upon reasonable request by contacting the corresponding author. Ethical Considerations As all data used in this study are open source and readily available, no ethical approval was required, as confirmed by the University of Manchester Ethics Committee. Conflict of interest declaration The authors declare that they have no conflicts of interest related to this research. This study received no specific funding from agencies in the public, commercial, or not-for-profit sectors that could have influenced the design, execution, interpretation, or reporting of this work. Author contributions statement First author: Conceptualization, Methodology, Data collection, Software development, Data analysis, Topic modelling, Interpretation of results, Writing - original draft, Writing - review & editing, Visualization. Second author: Conceptualization, Methodology, Supervision, Validation, Interpretation of results, Writing - review & editing, Project administration, Resources, Funding acquisition. Both authors have read and approved the final manuscript for submission. References Kumah E, Osei-Kesse F, Anaba C. Understanding and using patient experience feedback to improve health care quality: Systematic review and framework development. Journal of patient-centered research and reviews. 2017;4:24-31. Vakati HS, Jebakumar R. Predicting ratings for user reviews and opinion mining analyze for physicians and hospitals. Asian Journal of Pharmaceutical and Clinical Research. 2017;10:47-9. Lin Y, Hong YA, Henson BS, Stevenson RD, Hong S, Lyu T, et al. Assessing patient experience and healthcare quality of dental care using patient online reviews in the united states: Mixed methods study. Journal of Medical Internet Research. 2020;22:e18652. Bo P, Lillian L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval. 2008;2:1-35. Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. Journal of Machine Learning Research. 2003;3:993-1022. Bahja M. A text-mining based approach to capturing the nhs patient experience: Brunel University London; 2017. Owen C, Seddon C, Clarke K, Bysouth T, Johnson D. The impact of the covid-19 pandemic on the mental health of dentists in wales. British Dental Journal. 2022;232:44-54. Keat R. Covid-19 lockdown and recovery: A dental public health perspective from lancashire and south cumbria. Primary Dental Journal. 2021;10:31-40. Hardwick C, Day E, Carlton E, Dudding T. The consequence of the closure of primary care dental services on secondary care during the covid-19 pandemic – a national outlook. Advances in Oral and Maxillofacial Surgery. 2024;13:100475. NHS. Letters, updates and additional guidance for dental teams. 2022. Available at https://www.england.nhs.uk/coronavirus/publication/preparedness-letters-for-dental-care/ (accessed 1st July 2024). Institute for government. Timeline of uk government coronavirus lockdowns and measures, march 2020 to december 2021. 2021. Available at https://www.instituteforgovernment.org.uk/sites/default/files/timeline-coronavirus-lockdown-december-2021.pdf (accessed 25th June 2024). Luo JYN, Liu PP, Wong MCM. Patients' satisfaction with dental care: A qualitative study to develop a satisfaction instrument. BMC Oral Health. 2018;18:15. Beaton L, Knights J, Barnsley L, Araujo M, Clarkson J, Freeman R, et al. Longitudinal online diaries with dental practitioners and dental care professionals during the covid-19 pandemic: A trajectory analysis. Front Oral Health. 2022;3:1074655. R OC, Landes D, Harris R. Trends and inequalities in realised access to nhs primary care dental services in england before, during and throughout recovery from the covid-19 pandemic. British Dental Journal. 2023. Richardson L. Beautiful soup documentation. 2007. Available at https://www.crummy.com/software/BeautifulSoup/bs4/doc/ (accessed 1st June 2024). NHS. Api and integration catalogue. 2025. Available at https://digital.nhs.uk/developer/api-catalogue (accessed 7th April 2025). Balbir Kumar RG, Prashant Sirohyia , Brajesh Kumar Ratre , Ram Singh. Impact of covid-19 pandemic on medical practices. Asian Pacific Journal of Cancer Care. 2022;7:1-11. Amazon. Detectsentiment. 2025. Available at https://docs.aws.amazon.com/comprehend/latest/APIReference/API_DetectSentiment.html (accessed 1st April 2025). Sha M. A cloud based sentiment analysis through logistic regression in aws platform. Computer Systems Science & Engineering. 2023;45:1-15. Byrne M, O’Malley L, Glenny A-M, Pretty I, Tickle M. Assessing the reliability of automatic sentiment analysis tools on rating the sentiment of reviews of nhs dental practices in england. PLOS ONE. 2021;16:e0259797. Lee D, Seung HS. Algorithms for non-negative matrix factorization. 14th International Conference on Neural Information Processing Systems (NIPS'00). Denver, USA2000. p. 535 - 41. Goyal A, Kashyap I. Comprehensive analysis of topic models for short and long text data. International Journal of Advanced Computer Science & Applications. 2023;14:249-59. Bird S. Nltk: The natural language toolkit. Proceedings of the COLING/ACL 2006 interactive presentation sessions2006. p. 69-72. NLTK. Source code for nltk.Stem.Wordnet. 2023. Available at https://www.nltk.org/_modules/nltk/stem/wordnet.html (accessed 1st June 2024). Sutrisman RT, Murfi H. Analysis of non-negative double singular value decomposition initialization method on eigenspace-based fuzzy c-means algorithm for indonesian online news topic detection. 2018 6th International Conference on Information and Communication Technology (ICoICT): IEEE; 2018. p. 55-60. Henderi H, Wahyuningsih T, Rahwanto E. Comparison of min-max normalization and z-score normalization in the k-nearest neighbor (knn) algorithm to test the accuracy of types of breast cancer. International Journal of Informatics and Information Systems. 2021;4:13-20. Aminu AQ, McMahon AD, Clark C, Sherriff A, Buchanan C, Watling C, et al. Inequalities in access to nhs primary care dental services in scotland during the covid-19 pandemic. British Dental Journal. 2023:1-6. Dover H, Harris M. The implications of covid-19 enhanced personal protective equipment for aerosol generating procedures on uk dental practices. A qualitative study. Annual Clinical Journal of Dental Health. 2023;12:13-7. British Dental Association. Dentists: Covid inquiry ignoring biggest hit and weakest recovery in healthcare. 2024. Available at https://www.bda.org/media-centre/dentists-covid-inquiry-ignoring-biggest-hit-and-weakest-recovery-in-healthcare/ (accessed 18th February 2025). Im EO, Shin HJ, Chee W. Characteristics of midlife women recruited through internet communities/groups. Computers Informatics Nursing. 2008;26:39-48. Liu B. Sentiment analysis: Mining opinions, sentiments, and emotions: Cambridge university press; 2020. Ribeiro FN, Araújo M, Gonçalves P, André Gonçalves M, Benevenuto F. Sentibench-a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science. 2016;5:1-29. Zhang L, Wang S, Liu B. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2018;8:e1253. Greaves F, Pape UJ, King D, Darzi A, Majeed A, Wachter RM, et al. Associations between internet-based patient ratings and conventional surveys of patient experience in the english nhs: An observational study. BMJ quality & safety. 2012;21:600-5. Additional Declarations There is no duality of interest Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: revise 16 Jun, 2025 Review # 1 received at journal 07 Jun, 2025 Review # 2 received at journal 03 Jun, 2025 Reviewer # 2 agreed at journal 22 May, 2025 Reviewer # 1 agreed at journal 14 May, 2025 Reviewers invited by journal 29 Apr, 2025 Editor assigned by journal 24 Apr, 2025 Submission checks completed at journal 24 Apr, 2025 First submitted to journal 22 Apr, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6503136","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research","associatedPublications":[],"authors":[{"id":449641831,"identity":"b66e4eb7-eb6a-42f9-ba04-11e8e43ca1a3","order_by":0,"name":"Ali Feizollah","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABNElEQVRIie2QP0vDQBiH33CQLFeyngTsV7hQsIrVfJVIIFNR3DoUiQTi4hcQip8hU8DtQiBd4l64JVmcOli6pAjqm9JBuPpndLgHDn7w8vC+vwPQaP4jlhHBBBgGEhFMCP06F6pCUKk6BQPBtFP8nxR81c4lRvIHxY5JBGJy7NnEuF1fP2b9ofWcN5t2dAVWURNaKQor8DBRMXIQG7HzkEn36f4yGFA/PIloyAldqIehQlYJMzkGp5dJI13QIwf8ggOMgdBXxeh3W/J3RlGJ33oz6aEy3LT+Bwd7uVfhWyViDEPi9CJ50W0B6gsOrNuiHuaiIkTJOHZJTmkpg7QaDxwaBtxkLzyfqfUP53dNLaY3nm3FhaRTeZbOK3fVjs65bQdNvSzV+nu/fov5/Uij0Wg0v/AJ43dplbO9Me8AAAAASUVORK5CYII=","orcid":"","institution":"University of Manchester","correspondingAuthor":true,"prefix":"","firstName":"Ali","middleName":"","lastName":"Feizollah","suffix":""},{"id":449641832,"identity":"f45f4c1d-a012-48cc-a60a-6f23601c9de3","order_by":1,"name":"Matthew Byrne","email":"","orcid":"https://orcid.org/0000-0002-0283-782X","institution":"University of Manchester","correspondingAuthor":false,"prefix":"","firstName":"Matthew","middleName":"","lastName":"Byrne","suffix":""}],"badges":[],"createdAt":"2025-04-22 10:21:05","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6503136/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6503136/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":82175838,"identity":"2307e5ca-eccd-4843-9ffd-3f46106d0baa","added_by":"auto","created_at":"2025-05-07 11:10:33","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":454081,"visible":true,"origin":"","legend":"\u003cp\u003eMonth-by-month sentiment analysis score for pre-pandemic, during pandemic, and post-pandemic.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-6503136/v1/5a9edfd3b83413c9db783e39.png"},{"id":82177108,"identity":"6aa77c7d-aa2e-4219-81e4-5cd8eb6edd37","added_by":"auto","created_at":"2025-05-07 11:18:34","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1232989,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6503136/v1/713fe690-dd22-4058-a46a-58647e645d18.pdf"}],"financialInterests":"There is no duality of interest","formattedTitle":"Lasting effects of the COVID-19 pandemic on patient satisfaction of English dental service: Sentiment analysis and topic modelling of online reviews","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003ePatient feedback serves as a critical measure of service quality. This feedback may be collected through structured means such as predetermined questionnaires, or through Unstructured Patient Feedback (UPF). UPF typically comprises \"free text\" comments of a patient\u0026rsquo;s unique care experience. Manual methods of gathering and processing UPF are time and labour intensive, creating delays between feedback and action.\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eOnline reviews have increased the accessibility of UPF. For practices, it offers insights into patient\u0026rsquo;s perceptions that can help guide quality improvement; for patients it may inform decisions about which providers they choose to access.\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e In England, various online platforms host online reviews of dental practices\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e, including NHS.uk. NHS.uk has pages for every dental practice holding an NHS contract in England. Through its \"Leave a review\" function, patients can rate practices on a 1\u0026ndash;5 star scale and provide UPF.\u003c/p\u003e \u003cp\u003eContemporary Natural Language Processing (NLP) offers powerful methods to analyse large volumes of textual data algorithmically. Two particularly valuable techniques for analysing UPF are Sentiment Analysis (SA), which describes whether a text is broadly positive or negative in sentiment\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e, and Topic Modelling (TM), which identifies salient themes discussed within text collections.\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e When applied to UPF over time, these techniques can reveal changing patient sentiments toward a service and the underlying reasons for these changes. Bahja and Lycett have demonstrated the utility of these technologies in predicting patient sentiment and identifying key themes within NHS Choices website feedback.\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eThe global COVID-19 pandemic provides a unique backdrop to assess whether the sentiments and content of UPF respond to external influences and whether NLP techniques can effectively capture these changes. Primary dental care in England was particularly affected by the pandemic, making it an ideal context for studying shifts in patient feedback under extraordinary circumstances. Between March 2020 and August 2022, NHS dental practices in England operated under six distinct phases of care provision, ranging from complete closure with only emergency care available through designated centres to gradual reopening with strict protocols, until the eventual return to full activity.\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e,\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e,\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e,\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eThe COVID-19 pandemic has not only affected access to dental care but may have also impacted patients' perceptions and experiences of the care they receive.\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e By comparing online reviews before, during, and after the pandemic, this study aims to uncover changes in patient satisfaction and identify contributing factors. This information can help stakeholders improve dental care services. While previous studies\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e have explored text mining in healthcare feedback, limited research examines longitudinal trends in patient sentiment specific to NHS dentistry during the COVID-19 pandemic.\u003c/p\u003e \u003cp\u003eThe objectives of this study are to: (1) assess the overall sentiment of online reviews of NHS dental practices in England before, during, and after the COVID-19 pandemic, and (2) to identify themes and factors contributing to patient satisfaction or dissatisfaction during these periods. We hypothesize that patient satisfaction may have decreased during the pandemic compared to the pre-pandemic period due to access challenges and service delivery changes.\u003c/p\u003e"},{"header":"2. Materials and Methods","content":"\u003cp\u003e \u003cb\u003eData Collection\u003c/b\u003e \u003c/p\u003e \u003cp\u003eData for this study were sourced from publicly available patient reviews on the NHS.uk website, in two rounds of collection. In the first round of data collection, a Python 3 script equipped with the BeautifulSoup library\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e, was used to collect the review texts, date of visit, and star ratings for each dental practice in England. The data collection was constrained to the first 10 reviews per dental practice due to webscraper limitation. Reviews were collected from March 2019 to June 2022.\u003c/p\u003e \u003cp\u003eIn the second round of data collection, Python script was developed to query the NHS Developers API\u003csup\u003e\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e, and identify dental practices by their national contract identifier (V-code). All available reviews of dental practices between April 2022 and March 2024 were collected to minimize the potential influence of the COVID-19 pandemic on the data, thereby allowing for a more accurate after-pandemic analysis.\u003c/p\u003e \u003cp\u003e \u003cb\u003eData Arrangement\u003c/b\u003e \u003c/p\u003e \u003cp\u003eBetween March 2020 and August 2022, NHS dental practices in England operated under six distinct phases of care provision, as outlined by Standard Operating Procedures (SOPs) from the Office of the Chief Dental Officer (OCDO).\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e These phases represented meaningful time periods with distinct service delivery conditions, summarized in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTimeline of phased reopening and operational requirements for dental practices during and after the COVID-19 pandemic\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePhase\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDates\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDescription\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e25/3/2020\u0026ndash;07/06/2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eGeneral Practices closed.\u003c/p\u003e \u003cp\u003eEmergency care in Urgent Dental Care centres only\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e08/06/2020\u0026ndash;31/12/2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eGeneral practices open\u003c/p\u003e \u003cp\u003eStrict standard operating procedures commenced\u003c/p\u003e \u003cp\u003eLimited AGP with 60 mins fallow time 20% contracted activity\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e01/01/2021\u0026ndash;31/03/2021\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRequirement of practices to meet 45% of contracted activity\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e01/04/2021\u0026ndash;31/12/2021\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRequirement of practices to meet 60% of contracted activity. Checkups and emergency care prioritised\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e01/01/2022\u0026ndash;30/06/2022\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRequirement of practices to meet 85% of contracted activity\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e01/08/2022 -\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100% contracted activity, removal of all restrictions\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn Phase 1 (March-June 2020) dental practices were closed, with emergency care available only through designated urgent dental care centres; Phase 2 (June-December 2020) marked the gradual reopening with strict SOPs that including limitation of aerosol-generating procedures, increased used of personal protective equipment, and extended fallow periods after treating patients; subsequent phases saw incremental increases in activity targets from 45% (Phase 3: January-March 2021) to 60% (Phase 4: April-December 2021) to 85% (Phase 5: January-June 2022), until full activity resumed with the removal of all restrictions in August 2022 (Phase 6).\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e,\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e,\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e,\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e These SOPs, summarised in Table \u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, limited the amount and types of treatment that practices could offer.\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e presents how our collected data corresponds to these pandemic phases with their corresponding dates, data collection methods, and corpus characteristics.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eData collection periods and sources corresponding to COVID-19 pandemic phases\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePeriod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePhase\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDates\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eSource\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCorpus content\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePre-pandemic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e01/03/2019\u0026ndash;28/02/2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWebscraper\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePartial Corpus\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003eDuring Pandemic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e25/03/2020\u0026ndash;07/06/2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWebscraper\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePartial Corpus\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e08/06/2020\u0026ndash;31/12/2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWebscraper\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePartial Corpus\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e01/01/2021\u0026ndash;31/03/2021\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWebscraper\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePartial Corpus\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e01/04/2021\u0026ndash;31/12/2021\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWebscraper\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePartial Corpus\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e01/01/2022\u0026ndash;30/06/2022\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWebscraper\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePartial Corpus\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePost-pandemic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e01/03/2023\u0026ndash;31/03/2024\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNHS API\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eAll reviews\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eWhilst variations on the dental contract continued until August 2022, review data from this source was not available.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSentiment Analysis\u003c/b\u003e \u003c/p\u003e \u003cp\u003ePrior to analysis, simple cleaning of the review text performed to remove any aberrant punctuation resulting from the scraping process. Each review was presented as a separate text file to the Amazon Web Services (AWS) Detect Sentiment API\u003csup\u003e\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e, which provided a categorial output of one of four sentiments: Positive, negative, neutral and mixed. This commercially available tool was selected for its ease of integration and scalability.\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e AWS has been demonstrated to have a good associated with human reviewers in identifying the sentiment of dental reviews.\u003csup\u003e\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e Sentiment outputs were then collated per time period.\u003c/p\u003e \u003cp\u003e \u003cb\u003eTopic Identification\u003c/b\u003e \u003c/p\u003e \u003cp\u003eTopic extraction from the text corpora were performed using Non-Negative Matrix Factorization (NMF)\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e. NMF was chosen for its effectiveness in identifying interpretable topics from large text corpora.\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e Topic extraction was performed following sentiment analysis. Reviews assigned a positive or negative sentiment were included for topic extraction in order to identify what factors caused positive or negative responses.\u003c/p\u003e \u003cp\u003ePrior to identifying the topics, we performed text pre-processing steps using Python and relevant NLP libraries comprising:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eExpanding contractions to their full forms using a Python script (e.g. don\u0026rsquo;t to do not).\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eConverting text to lowercase using Python\u0026rsquo;s built-in \u0026lsquo;lower\u0026rsquo; method.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eSplitting the text into individual words (tokenisation) using \u0026lsquo;word_tokenize\u0026rsquo; function from the \u0026lsquo;NLTK\u0026rsquo; library.\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eRemoval of non-alphabetic characters using \u0026lsquo;regex\u0026rsquo; in Python.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eReduction of suffixed/prefixed words to their root form (lemmatisation) using the WordNet Lemmatizer\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e from the \u0026lsquo;NLTK\u0026rsquo; library (e.g. eating and ate to eat).\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003eTopics were extracted separately for pre, during, and after the pandemic periods, and the top keywords for each topic were identified. In the context of the NMF model, each review is represented as a combination of different topics; each topic has a certain weight indicating its presence within that review. We decided on the number of topics in the NMF by checking coherence scores and manually evaluating how distinct the topics were. Then, using Nonnegative Double Singular Value Decomposition (nndsvd) initialization\u003csup\u003e\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e, the model parameters were iteratively refined until they converged, with a maximum of 500 iterations to ensure convergence. The keywords within the computed topics were analysed by the authors to name and validate these topics.\u003c/p\u003e \u003cp\u003eA prevalence score was then calculated to serve as a quantitative measure of how dominant or frequently discussed a topic was within the dataset. Thus, topics with higher prevalence scores appeared more consistently and were more commonly discussed by users. To account for the variation in the number of reviews received in each time period, topic prevalence was standardised within each time period using min-max normalisation.\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e, whereby a topic\u0026rsquo;s prevalence score is divided against the maximum prevalence score to rescale the scores within each period to a range of 0 to 1. This preserved the relative differences between topics within each period, while also allowing for direct comparisons across different periods. For example, if during a certain period the highest (raw) prevalence score is 50 for Topic A, and Topic B and Topic C have scores of 40 and 25 respectively, the normalised scores would be 1.00 (50/50), 0.80 (40/50), and 0.50 (25/50). Similar normalised scores (e.g., 1.00 and 0.95) indicate comparable levels of discussion, whereas a larger gap (e.g., 1.00 versus 0.50) indicates a more dominant topic relative to the others.\u003c/p\u003e"},{"header":"3. Results","content":"\u003cp\u003eWebscraping of reviews during the period March 2019 - June 2022 identified 24,304 reviews. 150 could not be attributed to a dental practice site and were excluded from analysis. As the webscraper used was limited to collecting only the 10 most recent reviews for each practice, any practice with \u0026gt;\u0026thinsp;10 reviews will not have all of their information represented. Of the 24,154 reviews, 16,308 were from the pre-pandemic period of March 2019 - February 2020, and 7,846 were from during the pandemic (March 2020 \u0026ndash; June 2022).\u003c/p\u003e \u003cp\u003eA call to the NHS.uk API made in April 2024 provided 33,242 reviews from April 2022 to April 2024. For the purposes of analysis, a 13-month period from March 2023 to March 2024 was selected as a representative sample containing 24,595 reviews. These reviews represented all feedback received by all practices during this period. A total of 37 reviews were excluded due to duplication or being too short (less than 3 words), leaving 24,558 reviews for analysis.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSentiment analysis\u003c/b\u003e \u003c/p\u003e \u003cp\u003eThe results, which is tabulated in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e, show a predominance of positive sentiments.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eProportions of sentiments of reviews by time periods (in percentage). Target refers to activity levels expected in OCDO SOPs\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"9\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePre-pandemic\u003c/p\u003e \u003cp\u003e(100% target)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePhase 1 (Closedown)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePhase 2\u003c/p\u003e \u003cp\u003e(20% target)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePhase 3\u003c/p\u003e \u003cp\u003e(45% target)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003ePhase 4\u003c/p\u003e \u003cp\u003e(60% target)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003ePhase 5\u003c/p\u003e \u003cp\u003e(85% target)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eAll during pandemic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003ePost-pandemic\u003c/p\u003e \u003cp\u003e(100% target)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3/19\u0026thinsp;\u0026minus;\u0026thinsp;2/20\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e25/3/20\u0026thinsp;\u0026minus;\u0026thinsp;7/6/20\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003e8/6/20\u0026ndash;31/12/20\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1/1/21\u0026ndash;31/3/21\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1/4/21\u0026ndash;30/6/21\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1/1/22\u0026ndash;30/6/22\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003e25/3/20\u0026ndash;30/6/22\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003e3/23\u0026thinsp;\u0026minus;\u0026thinsp;3/24\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePositive\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e81.52(n\u0026thinsp;=\u0026thinsp;13295)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e87.12(n\u0026thinsp;=\u0026thinsp;176)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e80.27(n\u0026thinsp;=\u0026thinsp;932)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e83.46(n\u0026thinsp;=\u0026thinsp;1600)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e84.89(n\u0026thinsp;=\u0026thinsp;1664)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e81.77(n\u0026thinsp;=\u0026thinsp;2131)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e82.88(n\u0026thinsp;=\u0026thinsp;6503)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e82.42(n\u0026thinsp;=\u0026thinsp;20242)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eNegative\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e13.87(n\u0026thinsp;=\u0026thinsp;2262)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7.42(n\u0026thinsp;=\u0026thinsp;15)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e14.55(n\u0026thinsp;=\u0026thinsp;169)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e12.25(n\u0026thinsp;=\u0026thinsp;235)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e10(n\u0026thinsp;=\u0026thinsp;196)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e11.16(n\u0026thinsp;=\u0026thinsp;291)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e11.54(n\u0026thinsp;=\u0026thinsp;906)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e9.72(n\u0026thinsp;=\u0026thinsp;2388)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eNeutral\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.89(n\u0026thinsp;=\u0026thinsp;636)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.96(n\u0026thinsp;=\u0026thinsp;8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3.53(n\u0026thinsp;=\u0026thinsp;41)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3.39(n\u0026thinsp;=\u0026thinsp;65)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e4.03(n\u0026thinsp;=\u0026thinsp;79)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e5.52(n\u0026thinsp;=\u0026thinsp;144)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e4.29(n\u0026thinsp;=\u0026thinsp;337)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e1.06(n\u0026thinsp;=\u0026thinsp;262)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMixed\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.7(n\u0026thinsp;=\u0026thinsp;115)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.48(n\u0026thinsp;=\u0026thinsp;3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.63(n\u0026thinsp;=\u0026thinsp;19)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.88(n\u0026thinsp;=\u0026thinsp;17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1.07(n\u0026thinsp;=\u0026thinsp;21)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.53(n\u0026thinsp;=\u0026thinsp;40)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1.27(n\u0026thinsp;=\u0026thinsp;100)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e6.80(n\u0026thinsp;=\u0026thinsp;1666)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;16308)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;202)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;1161)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;1917)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;1960)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;2606)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;7846)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e\u003cb\u003e100(n\u0026thinsp;=\u0026thinsp;24558)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e81.52% of the pre-pandemic reviews were positive (n\u0026thinsp;=\u0026thinsp;13,295), while during the pandemic, the proportion of reviews with positive sentiments increased to 82.88% (n\u0026thinsp;=\u0026thinsp;6,503). During the initial closedown (Phase 1) when only emergency services were likely available, the proportion of positive reviews peaked at 87.12% (n\u0026thinsp;=\u0026thinsp;176/202). This proportion of positive reviews dropped to 80.27% in phase 2, then rose to 83.46% (n\u0026thinsp;=\u0026thinsp;1,600) in Phase 3 and at 84.89% (n\u0026thinsp;=\u0026thinsp;1,664) in Phase 4, before slightly reducing to 81.77% (n\u0026thinsp;=\u0026thinsp;2,131) in Stage 4. The proportion of positive reviews post-pandemic was 82.42% which is higher than pre-pandemic, but lower than \u0026lsquo;all during pandemic\u0026rsquo;.\u003c/p\u003e \u003cp\u003eConversely, negative reviews decreased from 13.87% pre-pandemic to 11.54% during the pandemic, with the lowest negativity occurring during the closedown at 7.42% (n\u0026thinsp;=\u0026thinsp;15). They decreased again to the lowest of 9.72% post-pandemic. Neutral reviews saw a minor increase from 3.89% pre-pandemic to 4.29% during the pandemic and then dropped to 1.06% post-pandemic. Mixed reviews, which encompass both positive and negative sentiments, increased from 0.7% pre-pandemic to 1.27% during the pandemic, and further to 6.8% post-pandemic. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e demonstrates the month-by-month variation of the proportion of reviews by each sentiment scores.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e"},{"header":"4. Qualitative analysis results","content":"\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e4.1. Negative reviews\u003c/h2\u003e \u003cp\u003eThe identified topics, the keywords in each topic, and normalised prevalence for reviews assigned with negative sentiment, during each time period are shown in Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTopics for negative reviews of pre, during, and post pandemic\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTopic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eKeywords\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNormalised Prevalence\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"6\" rowspan=\"7\"\u003e \u003cp\u003e\u003cb\u003ePre-pandemic\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStaff behaviour and professionalism\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003estaff, rude, reception, unprofessional, unhelpful, way, bad, feel, attended, need\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eWaiting times\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eminutes, late, time, waiting, seen, mins, wait, day, 30, arrived\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.946\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCancellations and appointment management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ecancelled, appointments, notice, hygienist, times, years, work, time, wait, months\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.792\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePain and emergency situations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003epain, emergency, days, help, called, got, later, day, antibiotics, extraction\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.746\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDental treatments and experiences\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003efilling, pain, treatment, teeth, went, felt, given, needed, left, weeks\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.699\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePhone communication issues\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ephone, answer, called, tried, day, make, rang, hold, rude\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.698\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNHS vs. private treatment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003enhs, patients, private, treatment, taking, register, new, pay, dental, website\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.697\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"6\" rowspan=\"7\"\u003e \u003cp\u003e\u003cb\u003eDuring pandemic\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStaff behaviour and professionalism\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003erude, staff, reception, unprofessional, recommend, unhelpful, dentists, awful, poor, manager\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTreatment costs and procedures\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etreatment, pay, check, dental, money, filling, cost, teeth, ppe, needed\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.907\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCancellations and appointment management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ecancelled, appointments, booked, time, day, work, times, rebooked, months, covid\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.766\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePhone communication issues\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ephone, answer, message, tried, trying, service, times, make, contact, book\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.751\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNHS vs. private treatment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003enhs, patients, private, taking, practise, new, covid, dental, website, poor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.711\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eWaiting times and need to wait outside of clinic\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eminutes, outside, late, 10, waiting, time, wait, arrived, 15, cold\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.688\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePain and emergency situations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003epain, left, got, called, emergency, filling, teeth, help, problem\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.674\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"6\" rowspan=\"7\"\u003e \u003cp\u003e\u003cb\u003ePost-pandemic\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStaff behaviour and patient experience\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003estaff, nervous, dental, feel, reception, experience, going, bad, ease\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCancellations and appointment management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ecancelled, appointments, phone, months, times, book, day, booked, answer, called\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.993\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDental hygiene and cleanings\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eteeth, check, polish, clean, hygienist, went, scale, mouth, gums, felt\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.858\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNHS vs. private treatment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003enhs, private, patients, pay, taking, years, new, afford, website, longer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.838\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePain and emergency situations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003efilling, pain, day, got, days, later, weeks, replaced, root, broken\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.739\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTreatment and problem resolution\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etreatment, problem, needed, given, required, seen, sorted, pay, received, time\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.689\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eWaiting times\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etime, minutes, late, seen, wait, 10, waiting, 30, arrived, mins\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.667\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAs mentioned before, in our analysis, topic prevalence scores were normalized using min\u0026ndash;max normalization such that the most prevalent topic in a given period is assigned a score of 1.00, with all other topics expressed as a ratio of this maximum value. Thus, when multiple topics exhibit similar normalized scores, it implies that they are discussed with comparable frequency. In contrast, a large difference in scores (high variance) between topics indicates that one theme is significantly more dominant than the others. For example, if Topic A has a score of 1.00, Topic B has a score of 0.95, and Topic C scores 0.50, then Topics A and B are nearly equal in their prevalence, while Topic C is less frequently mentioned.\u003c/p\u003e \u003cp\u003eThe topic modelling results reveal shifting patterns in patient concerns across three periods: pre, during, and post pandemic. Before the pandemic, the primary issues centred around staff behaviour and professionalism, with a normalised prevalence of 1.00. Patients frequently mentioned rudeness, unprofessional conduct, and unhelpful attitudes, particularly regarding reception staff. Waiting times and punctuality followed closely at 0.946, with patients expressing frustration about late appointments and extended waiting periods. Appointment scheduling and cancellations ranked third at 0.792, often involving issues with hygienist appointments and long wait times for bookings.\u003c/p\u003e \u003cp\u003eDuring the pandemic period, staff behaviour and professionalism remained the top concern. However, treatment costs and procedures emerged as a significant issue, ranking second (0.907). This topic encompassed concerns about payments, dental check-ups, and notably, PPE requirements. Appointment-related problems persisted, with cancellations and re-bookings becoming more prominent (0.766). Communication difficulties, especially regarding phone services, also featured prominently (0.751).\u003c/p\u003e \u003cp\u003eThe post-pandemic period saw a similar pattern with staff behaviour and patient experience prevalence score of 1.00. Appointment scheduling and communication issues closely followed at 0.993. Dental hygiene services emerged as a distinct concern (0.858). The long-standing issue of NHS versus private practice gained prominence, reaching 0.838 prevalence.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e4.2. Positive reviews\u003c/h2\u003e \u003cp\u003eThe positive reviews of all three periods were selected for topic modelling analysis. Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e shows the topics, keywords, and the normalised prevalence score.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTopics for positive reviews of pre, during, and post pandemic\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTopic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eKeywords\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNormalised Prevalence\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"6\" rowspan=\"7\"\u003e \u003cp\u003e\u003cb\u003ePre-pandemic\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePatient recommendations and dental anxiety management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003erecommend, highly, ease, fear, surgery, years, dental, dentists, understanding, going\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRoutine dental check-ups and cleaning\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eteeth, going, check, polish, scale, quick, mouth, given, thorough, cleaning\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.905\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePain management and dental procedures\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003epain, abscess, infected, bad, extracted, went, experience, taken, antibiotics, say\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.824\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePatient comfort and anxiety management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003enervous, feel, coming, time, bad, started, experience, visiting, staff, experiences\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.720\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEmergency dental care and problem resolution\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eproblem, emergency, crown, sorted, day, time, went, checkup, getting, broken\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.622\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePositive staff interactions and patient reassurance\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ethank, scared, went, today, team, smile, saw, worried, attentive, step\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.622\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNHS vs. private treatment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003enhs, treatment, dental, private, patients, needed, hesitation, work, ago, received\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.585\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"6\" rowspan=\"7\"\u003e \u003cp\u003e\u003cb\u003eDuring pandemic\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLong-term dental care and fear management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eteeth, years, dentists, going, extremely, removed, forward, team, fear, look\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePatient comfort and anxiety management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003enervous, really, feel, ease, bad, thank, understanding, highly, extraction, relaxed\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.937\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDental procedures and pain management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003efilling, day, got, pain, surgery, days, called, went, service, later\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.842\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCOVID-19 precautions and professional care\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eproblem, seen, covid, quickly, explained, professionally, gum, precautions, given, caring\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.774\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStaff professionalism and patient comfort\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003estaff, praise, way, reception, anxiety, problems, professional, ease, hygienist, practise\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.752\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppreciation for service during difficult times\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etimes, difficult, thank, time, service, especially, covid, excellent, amazing, complaints\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.698\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eComprehensive dental treatment and patient care\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etreatment, dental, check, time, received, nhs, took, surgeon, feel, extraction\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.651\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"6\" rowspan=\"7\"\u003e \u003cp\u003e\u003cb\u003ePost-pandemic\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eService and professional care\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eservice, excellent, great, care, years, received, professional, recommend, highly, thank\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePatient comfort and anxiety management\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003efeel, ease, nervous, comfortable, really, make, relaxed, lovely, felt, makes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.994\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEfficient and thorough appointments\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etime, seen, teeth, took, check, pleasant, long, explain, waiting, wait\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.929\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStaff friendliness and professionalism\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003estaff, friendly, helpful, reception, professional, polite, efficient, welcoming, pleasant, clean\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.911\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eClear communication of treatment plans\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003etreatment, explained, options, clearly, going, pleased, fully, plan, excellent, step\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.888\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCaring dental team\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003edental, nurse, lovely, kind, team, care, experience, receptionists, thank, recommend\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.844\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eQuality dental advice and hygiene\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003egood, advice, experience, really, nice, given, clean, gave, hygienist, job\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.708\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe topic modelling analysis of positive patient reviews in dental practices reveals evolving patterns across three distinct periods: pre-pandemic, during pandemic, and post-pandemic.\u003c/p\u003e \u003cp\u003eIn the pre-pandemic period, the most prevalent topic was \"Patient Recommendations and Dental Anxiety Management\" (1.00), followed closely by \"Routine Dental Check-ups and Cleaning\" (0.905). \"Pain Management and Dental Procedures\" (0.824) and \"Patient Comfort and Anxiety Management\" (0.720) were also significant themes. Less prevalent but still notable were \"Emergency Dental Care and Problem Resolution\" (0.622), \"Positive Staff Interactions and Patient Reassurance\" (0.622), and \"NHS and Private Dental Treatment Options\" (0.585).\u003c/p\u003e \u003cp\u003eDuring the pandemic, the focus shifted slightly. \"Long-term Dental Care and Fear Management\" became the most prevalent topic (1.00), followed by \"Patient Comfort and Anxiety Management\" (0.937). \"Dental Procedures and Pain Management\" (0.842) remained important, while a new topic emerged: \"COVID-19 Precautions and Professional Care\" (0.774). \"Staff Professionalism and Patient Comfort\" (0.752), \"Appreciation for Service During Difficult Times\" (0.698), and \"Comprehensive Dental Treatment and Patient Care\" (0.651) rounded out the topics.\u003c/p\u003e \u003cp\u003eIn the post-pandemic period, there was a significant shift in topic prevalence and content. \"Excellent Service and Professional Care\" became the most prevalent topic (1.00), closely followed by \"Patient Comfort and Anxiety Management\" (0.994). \"Efficient and Thorough Appointments\" (0.929) and \"Staff Friendliness and Professionalism\" (0.911) also gained prominence. \"Clear Communication of Treatment Plans\" (0.888), \"Caring Dental Team and Positive Experience\" (0.844), and \"Quality Dental Advice and Hygiene\" (0.708) completed the post-pandemic topics.\u003c/p\u003e \u003c/div\u003e"},{"header":"5. Discussion","content":"\u003cp\u003eThis study analysed a 5-year corpus of NHS.uk dental reviews to assess how patient concerns evolved across pre-pandemic, during-pandemic, and post-pandemic periods. By examining reviews in relation to changing COVID-19 contractual arrangements, we can link patient experiences to policy decisions and evaluate the pandemic's lasting impact on dental service expectations.\u003c/p\u003e \u003cp\u003eSentiment scores remained consistently positive through all periods. An interesting finding was that the lowest proportion of negative reviews were seen around the first phase of the COVID response when access was poorest. This may suggest patient appreciation for receiving care despite restrictions, and could reflect dentists having more time to communicate with fewer patients they saw.\u003c/p\u003e \u003cp\u003eTopic modelling revealed that interpersonal aspects of care dominated across both positive and negative reviews. Staff friendliness and professionalism became more prominent in positive reviews post-pandemic. This could reflect increased patient appreciation for dental staff working under challenging conditions, or improved staff-patient interactions as a result of adapted practices.\u003c/p\u003e \u003cp\u003eStaff behaviour and professionalism dominated negative reviews during the pandemic potentially due to the increased stress and anxiety among both patients and staff during the pandemic. Staff interactions remained the most prominent theme of negative reviews post-pandemic suggesting an increased use of online review platforms to air dissatisfaction, or potentially an increase in conflict between patients and dental staff.\u003c/p\u003e \u003cp\u003eThe persistence of communication challenges, particularly phone-related issues, throughout all time periods resonates with previous studies.\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e This persistence suggests that dental practices may need to reevaluate and improve their communication strategies, especially considering the increased prevalence of appointment-related concerns post-pandemic. The pandemic likely exacerbated existing communication difficulties, with practices struggling to manage high volumes of cancellations, re-bookings, and patient inquiries.\u003c/p\u003e \u003cp\u003eWhile communication issues persisted throughout all periods, the pandemic introduced unique themes absent pre- and post-pandemic. COVID-19 precautions were mentioned in both positive reviews—where patients appreciated the tangible commitments to safety—and in negative reviews, where social distancing led to the need to wait outside of the surgery. These findings align with previous research on mandatory enhanced PPE in dental practices during COVID-19. The study documented both positive feedback, such as an increased perception of safety, and negative impacts, including communication difficulties and discomfort for both patients and staff.\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e Furthermore, waiting times and punctuality were major concerns in negative reviews pre-pandemic, but decreased in importance during subsequent periods. This shift might reflect new appointment systems, reduced patient volumes, or these issues being overshadowed by more pressing safety and access concerns.\u003c/p\u003e \u003cp\u003eBeyond service delivery changes, the pandemic also affected practice business models. Discussion of NHS versus private treatment options decreased in prominence from pre-pandemic (0.697) to post-pandemic (0.585). This could indicate shifts in practice models, changes in patient perceptions, or alterations in NHS services availability due to pandemic pressure. The lasting reduction of access to NHS dental services has been widely reported in the media.\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e Many dental services have handed back their dental contracts and are only offering care on a private basis or removing their NHS provision to patients. The changing dynamics between NHS and private practice, as reflected in the increasing prevalence of this topic suggest that patients sensed a systemic change in healthcare delivery.\u003csup\u003e\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eThe emergence of treatment cost themes in negative reviews during pandemic may reflect this conflict of NHS and private healthcare delivery. Patients likely faced new financial pressures and safety concerns, which were directly reflected in their feedback. Post-pandemic, the focus shifted toward dental hygiene services and anxiety management, suggesting both a routine care backlog and greater emphasis on overall patient experience. This finding aligns with broader observations about changing patient behaviours and expectations in healthcare delivery post-pandemic, where patients have increasingly prioritized safety, accessibility, and mental well-being in their healthcare experiences.\u003csup\u003e\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eSeveral limitations of this study should be acknowledged. First, the data was sourced from online reviews on the NHS.uk website, which may not fully represent the experiences of all patients, especially those who do not use or have access to online review platforms. A previous study demonstrating the demographics of patients that leave online reviews suggests that this population skews to middle aged female patients, suggesting that this demographic may be overrepresented.\u003csup\u003e\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e Second, despite over 7,000 NHS-contracted practices in England, many lacked online reviews, creating potential selection bias. Leaving an online review has several barriers to entry that must be overcome. This may suggest that those who are inclined to go to the effort have had a particularly positive or negative experience and thus may not reflect average experiences. Third, the AWS Detect Sentiment API, while robust, has limitations in capturing nuanced human language.\u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e The large dataset used may mitigate the risk of ‘mis-categorisation’ of reviews into incorrect sentiment scores, however this error may be more prevalent in the time periods with fewer reviews. The use of an ‘off the shelf’ tool offers ease of use, but limits control over consistency of algorithms.\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e The striking reduction in negative sentiments and increase in mixed sentiments post-pandemic might reflect either changing algorithms\u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, or evolving patient review behaviours toward more nuanced expressions that don't fit neatly into positive/negative categories.\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e "},{"header":"Conclusions","content":"\u003cp\u003eThis study has shown that the proportion of online reviews of NHS dental services with positive sentiment is generally high and remained high throughout the changing landscape of the COVID-19 pandemic. Whilst the proportion of negative reviews remained relatively constant pre- and during the pandemic, topic extraction has demonstrated changes in the factors that stimulated with key themes of dissatisfaction including pain and emergency situations, appointment cancellations, staff behaviour, NHS versus private treatment accessibility, waiting times, and phone communication issues. During the pandemic, new challenges emerged such as increased appointment cancellations due to COVID-19 restrictions, difficulties with waiting outside practices due to social distancing, and exacerbated phone communication issues. Despite these challenges, the high level of positive sentiment during the initial lockdown phase indicates robust patient appreciation for emergency dental services. Topic modelling of reviews demonstrated evolving patient priorities and appreciations, with a notable increase in the recognition of service quality, staff professionalism, and clear communication. Assessment of negative reviews suggest that dental practices should prioritise quality improvement efforts in appointment management, patient communication, stress management. Future research should expand data sources, incorporate qualitative methods, and analyse the long-term impacts of the pandemic on patient satisfaction.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eData availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll data supporting the findings of this study are available upon reasonable request by contacting the corresponding author.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical Considerations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAs all data used in this study are open source and readily available, no ethical approval was required, as confirmed by the University of Manchester Ethics Committee.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of interest declaration\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no conflicts of interest related to this research. This study received no specific funding from agencies in the public, commercial, or not-for-profit sectors that could have influenced the design, execution, interpretation, or reporting of this work.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFirst author: Conceptualization, Methodology, Data collection, Software development, Data analysis, Topic modelling, Interpretation of results, Writing - original draft, Writing - review \u0026amp; editing, Visualization.\u003c/p\u003e\n\u003cp\u003eSecond author: Conceptualization, Methodology, Supervision, Validation, Interpretation of results, Writing - review \u0026amp; editing, Project administration, Resources, Funding acquisition.\u003c/p\u003e\n\u003cp\u003eBoth authors have read and approved the final manuscript for submission.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eKumah E, Osei-Kesse F, Anaba C. Understanding and using patient experience feedback to improve health care quality: Systematic review and framework development. Journal of patient-centered research and reviews. 2017;4:24-31.\u003c/li\u003e\n\u003cli\u003eVakati HS, Jebakumar R. Predicting ratings for user reviews and opinion mining analyze for physicians and hospitals. Asian Journal of Pharmaceutical and Clinical Research. 2017;10:47-9.\u003c/li\u003e\n\u003cli\u003eLin Y, Hong YA, Henson BS, Stevenson RD, Hong S, Lyu T, et al. Assessing patient experience and healthcare quality of dental care using patient online reviews in the united states: Mixed methods study. Journal of Medical Internet Research. 2020;22:e18652.\u003c/li\u003e\n\u003cli\u003eBo P, Lillian L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval. 2008;2:1-35.\u003c/li\u003e\n\u003cli\u003eBlei DM, Ng AY, Jordan MI. Latent dirichlet allocation. Journal of Machine Learning Research. 2003;3:993-1022.\u003c/li\u003e\n\u003cli\u003eBahja M. A text-mining based approach to capturing the nhs patient experience: Brunel University London; 2017.\u003c/li\u003e\n\u003cli\u003eOwen C, Seddon C, Clarke K, Bysouth T, Johnson D. The impact of the covid-19 pandemic on the mental health of dentists in wales. British Dental Journal. 2022;232:44-54.\u003c/li\u003e\n\u003cli\u003eKeat R. Covid-19 lockdown and recovery: A dental public health perspective from lancashire and south cumbria. Primary Dental Journal. 2021;10:31-40.\u003c/li\u003e\n\u003cli\u003eHardwick C, Day E, Carlton E, Dudding T. The consequence of the closure of primary care dental services on secondary care during the covid-19 pandemic \u0026ndash; a national outlook. Advances in Oral and Maxillofacial Surgery. 2024;13:100475.\u003c/li\u003e\n\u003cli\u003eNHS. Letters, updates and additional guidance for dental teams. 2022. Available at https://www.england.nhs.uk/coronavirus/publication/preparedness-letters-for-dental-care/ (accessed 1st July 2024).\u003c/li\u003e\n\u003cli\u003eInstitute for government. Timeline of uk government coronavirus lockdowns and measures, march 2020 to december 2021. 2021. Available at https://www.instituteforgovernment.org.uk/sites/default/files/timeline-coronavirus-lockdown-december-2021.pdf (accessed 25th June 2024).\u003c/li\u003e\n\u003cli\u003eLuo JYN, Liu PP, Wong MCM. Patients\u0026apos; satisfaction with dental care: A qualitative study to develop a satisfaction instrument. BMC Oral Health. 2018;18:15.\u003c/li\u003e\n\u003cli\u003eBeaton L, Knights J, Barnsley L, Araujo M, Clarkson J, Freeman R, et al. Longitudinal online diaries with dental practitioners and dental care professionals during the covid-19 pandemic: A trajectory analysis. Front Oral Health. 2022;3:1074655.\u003c/li\u003e\n\u003cli\u003eR OC, Landes D, Harris R. Trends and inequalities in realised access to nhs primary care dental services in england before, during and throughout recovery from the covid-19 pandemic. British Dental Journal. 2023.\u003c/li\u003e\n\u003cli\u003eRichardson L. Beautiful soup documentation. 2007. Available at https://www.crummy.com/software/BeautifulSoup/bs4/doc/ (accessed 1st June 2024).\u003c/li\u003e\n\u003cli\u003eNHS. Api and integration catalogue. 2025. Available at https://digital.nhs.uk/developer/api-catalogue (accessed 7th April 2025).\u003c/li\u003e\n\u003cli\u003eBalbir Kumar RG, Prashant Sirohyia , Brajesh Kumar Ratre , Ram Singh. Impact of covid-19 pandemic on medical practices. Asian Pacific Journal of Cancer Care. 2022;7:1-11.\u003c/li\u003e\n\u003cli\u003eAmazon. Detectsentiment. 2025. Available at https://docs.aws.amazon.com/comprehend/latest/APIReference/API_DetectSentiment.html (accessed 1st April 2025).\u003c/li\u003e\n\u003cli\u003eSha M. A cloud based sentiment analysis through logistic regression in aws platform. Computer Systems Science \u0026amp; Engineering. 2023;45:1-15.\u003c/li\u003e\n\u003cli\u003eByrne M, O\u0026rsquo;Malley L, Glenny A-M, Pretty I, Tickle M. Assessing the reliability of automatic sentiment analysis tools on rating the sentiment of reviews of nhs dental practices in england. PLOS ONE. 2021;16:e0259797.\u003c/li\u003e\n\u003cli\u003eLee D, Seung HS. Algorithms for non-negative matrix factorization. 14th International Conference on Neural Information Processing Systems (NIPS\u0026apos;00). Denver, USA2000. p. 535 - 41.\u003c/li\u003e\n\u003cli\u003eGoyal A, Kashyap I. Comprehensive analysis of topic models for short and long text data. International Journal of Advanced Computer Science \u0026amp; Applications. 2023;14:249-59.\u003c/li\u003e\n\u003cli\u003eBird S. Nltk: The natural language toolkit. Proceedings of the COLING/ACL 2006 interactive presentation sessions2006. p. 69-72.\u003c/li\u003e\n\u003cli\u003eNLTK. Source code for nltk.Stem.Wordnet. 2023. Available at https://www.nltk.org/_modules/nltk/stem/wordnet.html (accessed 1st June 2024).\u003c/li\u003e\n\u003cli\u003eSutrisman RT, Murfi H. Analysis of non-negative double singular value decomposition initialization method on eigenspace-based fuzzy c-means algorithm for indonesian online news topic detection. 2018 6th International Conference on Information and Communication Technology (ICoICT): IEEE; 2018. p. 55-60.\u003c/li\u003e\n\u003cli\u003eHenderi H, Wahyuningsih T, Rahwanto E. Comparison of min-max normalization and z-score normalization in the k-nearest neighbor (knn) algorithm to test the accuracy of types of breast cancer. International Journal of Informatics and Information Systems. 2021;4:13-20.\u003c/li\u003e\n\u003cli\u003eAminu AQ, McMahon AD, Clark C, Sherriff A, Buchanan C, Watling C, et al. Inequalities in access to nhs primary care dental services in scotland during the covid-19 pandemic. British Dental Journal. 2023:1-6.\u003c/li\u003e\n\u003cli\u003eDover H, Harris M. The implications of covid-19 enhanced personal protective equipment for aerosol generating procedures on uk dental practices. A qualitative study. Annual Clinical Journal of Dental Health. 2023;12:13-7.\u003c/li\u003e\n\u003cli\u003eBritish Dental Association. Dentists: Covid inquiry ignoring biggest hit and weakest recovery in healthcare. 2024. Available at https://www.bda.org/media-centre/dentists-covid-inquiry-ignoring-biggest-hit-and-weakest-recovery-in-healthcare/ (accessed 18th February 2025).\u003c/li\u003e\n\u003cli\u003eIm EO, Shin HJ, Chee W. Characteristics of midlife women recruited through internet communities/groups. Computers Informatics Nursing. 2008;26:39-48.\u003c/li\u003e\n\u003cli\u003eLiu B. Sentiment analysis: Mining opinions, sentiments, and emotions: Cambridge university press; 2020.\u003c/li\u003e\n\u003cli\u003eRibeiro FN, Ara\u0026uacute;jo M, Gon\u0026ccedil;alves P, Andr\u0026eacute; Gon\u0026ccedil;alves M, Benevenuto F. Sentibench-a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science. 2016;5:1-29.\u003c/li\u003e\n\u003cli\u003eZhang L, Wang S, Liu B. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2018;8:e1253.\u003c/li\u003e\n\u003cli\u003eGreaves F, Pape UJ, King D, Darzi A, Majeed A, Wachter RM, et al. Associations between internet-based patient ratings and conventional surveys of patient experience in the english nhs: An observational study. BMJ quality \u0026amp; safety. 2012;21:600-5.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"british-dental-journal","isNatureJournal":false,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"bdj","sideBox":"Learn more about [British Dental Journal](http://www.nature.com/bdj/)","snPcode":"41415","submissionUrl":"https://mts-bdj.nature.com/cgi-bin/main.plex","title":"British Dental Journal","twitterHandle":"@the_bdj","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"COVID-19, Dental Care, Patient Satisfaction, Natural Language Processing, Health Services Accessibility, Sentiment Analysis, Health Policy, Quality Improvement, Internet-Based Intervention, Health Communication","lastPublishedDoi":"10.21203/rs.3.rs-6503136/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6503136/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eAims\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThis study aimed to determine how changes in English dental practice during and after the COVID pandemic impacted patient perceptions of dental care delivery use sentiment analysis and topic extraction of online reviews\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods\u003c/b\u003e\u003c/p\u003e \u003cp\u003eReviews were collected from the NHS website using webscraping and the NHS digital API. Reviews were grouped into pre-, during-, and post pandemic phases, and subcategorised to align with the dates of national standard operating procedures. The AWS Detect Sentiment API categorized each review by sentiment and Non-Negative Matrix Factorization was used to identify topics in positive and negative reviews.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults\u003c/b\u003e\u003c/p\u003e \u003cp\u003e48,862 reviews were analysed. Reviews were consistently positive (81.52% pre-; 82.88% during, 82.48% post-) The proportion of Negative reviews decreased (13.87% pre- ,11.54% during-, 9.72% post-). For negative reviews, staff behaviour and professionalism prevalent topics in all time periods. During the pandemic, topics of treatment costs and poor communication emerged. In positive reviews, patient comfort and anxiety management were consistently identified. Appreciation for COVID-19 precautions emerged as a new theme during the pandemic.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusions\u003c/b\u003e\u003c/p\u003e \u003cp\u003eUnstructured patient feedback is a rich data source to assess patient experiences. This research suggests appointment management, communication, and staff interactions are priorities for quality improvement in NHS dentistry.\u003c/p\u003e","manuscriptTitle":"Lasting effects of the COVID-19 pandemic on patient satisfaction of English dental service: Sentiment analysis and topic modelling of online reviews","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-05-07 11:10:28","doi":"10.21203/rs.3.rs-6503136/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"revise","date":"2025-06-16T15:23:27+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"This content is not available.","date":"2025-06-07T13:20:08+00:00","index":1,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2025-06-03T14:47:11+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2025-05-22T08:24:33+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2025-05-14T13:47:45+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewersInvited","content":"","date":"2025-04-29T10:56:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-04-24T17:08:06+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-04-24T17:08:00+00:00","index":"","fulltext":""},{"type":"submitted","content":"British Dental Journal","date":"2025-04-22T10:17:40+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"british-dental-journal","isNatureJournal":false,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"bdj","sideBox":"Learn more about [British Dental Journal](http://www.nature.com/bdj/)","snPcode":"41415","submissionUrl":"https://mts-bdj.nature.com/cgi-bin/main.plex","title":"British Dental Journal","twitterHandle":"@the_bdj","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"8a935790-5063-413f-b5cf-a3a6bb8a36e9","owner":[],"postedDate":"May 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":47846344,"name":"Health sciences/Health care/Dentistry/Dental public health/Dental epidemiology"}],"tags":[],"updatedAt":"2025-07-23T09:55:12+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-07 11:10:28","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6503136","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6503136","identity":"rs-6503136","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}