Understanding Public Attitudes Towards Human Papillomavirus Vaccination in Japan: Insights from Social Media Stance Analysis Using Large Language Models

Background: Despite the reinstatement of proactive human papillomavirus (HPV) vaccine recommendations in 2022, Japan continues to face persistently low HPV vaccination rates, posing significant public health challenges. Misinformation, complacency, and accessibility issues have been identified as key factors undermining vaccine uptake.

This study aims to analyze the evolution of public attitudes towards HPV vaccination in Japan by examining social media content. The goal is to understand how factors such as misinformation, public health events, and attitudes toward other vaccines, like COVID-19, influence vaccine hesitancy.

We collected tweets related to HPV vaccine from 2011 to 2021. Traditional natural language processing (NLP) methods and large language models (LLMs) was utilized to perform stance analysis on collected data. The analysis included stance identification, time series analysis, topic modeling, and logic analysis. We framed our findings within the context of the WHO's 3Cs model—Confidence, Complacency, and Convenience.

Public confidence in the HPV vaccine fluctuated in response to government policies and media events, with misinformation playing a critical role in eroding trust. Complacency increased following the suspension of recommendations in 2013 but decreased as advocacy resumed in 2020. Accessibility (Convenience) was also found to be a key determinant of vaccination uptake. Increased confidence in HPV vaccines appeared to have a positive influence on confidence in other vaccines, such as COVID-19. Our findings suggest the need for targeted interventions focusing on addressing misinformation, improving accessibility, and maintaining consistent communication to enhance vaccine confidence.

Our findings underscore the importance of targeted public health interventions to restore and maintain vaccine confidence in Japan. While vaccine confidence has shown a slow increase, sustained efforts are necessary to secure long-term improvements. Confidence in one vaccine may positively influence perceptions of other vaccines. Addressing misinformation, reducing complacency, and enhancing vaccine accessibility are key strategies to improve uptake. This study also demonstrates the utility of LLMs in offering a deeper understanding of public health attitudes. To effectively combat vaccine hesitancy and improve coverage, interventions must prioritize consistent communication, localized strategies, and an integrated approach to vaccine narratives.

HPV vaccine; vaccine confidence; large language model; stance analysis; topic modeling . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.

In 2020, the World Health Organization (WHO) launched a strategy to eliminate cervical cancer, aiming for all countries to achieve a 90% coverage rate for the human papillomavirus (HPV) vaccination by 2030[1]. In 2023, WHO released estimates of the first- dose vaccination coverage for females in 133 countries, with a global average of 62%, and 15 countries have already met WHO's target [2]. Amid the context of HPV vaccination in high-income countries, Japan is in a critical situation. In Japan, the bivalent HPV vaccine was licensed in October 2009, and the quadrivalent vaccine in July 2011. Public subsidies for girls in grades 7 to 11 began in 2010 [3]. Starting in April 2013, both bivalent and quadrivalent HPV vaccines were included in Japan's national immunization program for girls aged 12-16 [4]. However, after widespread reports of adverse events related to HPV vaccination, the Japanese Ministry of Health, Labour and Welfare (MHLW) announced the suspension of its proactive recommendation for the HPV vaccine in June of the same year [5,6], leading to an immediate and sharp decline in public trust and acceptance of the HPV vaccine [7–11] and significantly increased cervical cancer incidence [12,13]. Larson et al. reported that the longer the recommendation suspension in Japan lasted, the greater the public's concern grew [11]. The WHO Global Advisory Committee on Vaccine Safety commented on the situation in Japan in 2015, stating, "policy decisions based on weak evidence, leading to lack of use of safe and effective vaccines, can result in real harm.” [14] Recent studies reveal that vaccine hesitancy in Japan has seen only marginal decrease since the proactive HPV vaccine recommendations were reinstated in 2022 [15–19]. Despite government efforts, the vaccination rate remains significantly below pre-2013 levels [19]. Several factors have contributed to this slow recovery. Tarada et al. and Lelliott et al. suggest that misinformation and persistent public fears regarding vaccine safety remain key obstacles [16,20], and Ueda et al. reported negative media coverage from the 2013 suspension continuing to affect perceptions [21]. However, localized initiatives, such as those implemented in Shiki City [18], have demonstrated that regional government support, public funding, and targeted campaigns can lead to substantial increases in vaccine uptake. Miyagi et al. emphasize the role of transparent communication and consistent public health messaging, which are crucial in mitigating the effects of past misinformation [17]. These factors all relate to the influence of multimedia, particularly social media, in recent years. Social media plays a significant role in shaping public perceptions of the HPV vaccine, and recent studies have highlighted both its positive and negative impacts. For instance, Dunn et al. found that exposure to negative information about the HPV vaccine on social media was associated with lower vaccine coverage, underscoring the influence of media controversies on public acceptance [22]. Similarly, Teoh et al. emphasized that social media platforms can both effectively communicate HPV vaccination recommendations and serve as conduits for misinformation, affecting public attitudes and behaviors [23]. In contrast, a study by Pedersen et al. demonstrated that strategic social media campaigns focusing on positive messaging significantly improved engagement rates and public support for HPV vaccination in Denmark [24]. These studies collectively indicate that social has the potential . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint to either promote vaccine uptake through well-targeted campaigns or amplify vaccine hesitancy through the spread of misinformation. This dual nature of social media's influence is particularly relevant in Japan, where negative media coverage has had a profound impact on public attitudes towards HPV vaccination [21,25,26]. Recently, NLP methods are applied to the social media analysis, making social media analysis more accurate and can catch nuanced information from large amount of data [27]. Beside the traditional NLP methods like n-gram [28] and topic modeling [29], Deep learning (DL) models are used for classification tasks, especial sentiment analysis in social media [30–33]. Tomaszewski et al. built a DL model to surveillant misinformation in social media about HPV [34]. As the LLMs become main stream of DL, there are also works starting to analyze social media with LLMs [35]. In this study, we aim to understand the persistently low HPV vaccination rate by analyzing the evolution of public attitudes towards HPV vaccination in Japan between 2011 and 2021. We utilize NLP models to evaluate social media content before the proactive recommendation was reinstated. Our analysis examines the dynamics of vaccine confidence over time, the relationship between public health events and shifts in public opinion, and the interplay between attitudes towards HPV and COVID-19 vaccines. By combining stance analysis, time series analysis, topic modeling, and logic analysis, we seek to provide a deeper understanding of vaccine confidence and the factors that drive vaccine uptake.

Data Collection and Preprocessing This study utilized the X (formerly Twitter) API to collect Japanese tweets related to the HPV vaccine between 2011 and 2021. Tweets were retrieved using various Japanese

for the HPV vaccine. The number of tweets generally increased each year, resulting in a total of 228,376 tweets. After excluding tweets with unrecognizable dates, 228,300 tweets were retained for subsequent analysis. Following data collection, the tweets underwent cleaning and preprocessing. Retweets were removed using the Python package tweepy [36]. Web links, special characters, emojis, and ampersands were also eliminated, and full-width English characters were converted to half-width, lowercase characters. Annotation To facilitate model training and fine-tuning, 2.5% of the total tweets per year were randomly selected for annotation. This study focused on public stances towards the HPV vaccine, employing a three-category annotation system: advocate, oppose, and unknown. Detailed category definitions are provided in Table 1. The annotation team consisted of four medical professionals, three of whom independently annotated each tweet. Tweets with unanimous annotations were classified as Tier 1, those with two matching annotations as Tier 2, and those without any matching annotations as Tier 3. Tier 1 data, deemed to have the highest consistency, utilized the initial annotations as the final labels. Tier 2 and Tier 3 data underwent further consensus meetings, with final labels determined through arbitration. A total of 5,758 tweets were annotated, with 5,050 in Tier 1, 701 in Tier 2, and 7 in Tier 3. The distribution across the advocate, oppose, and unknown categories in the annotated data was 1,646, 697, and 3,415, respectively. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Table 1 Definitions of stances to the HPV Vaccine Category Definition Mock tweet example advocate Affirmatively advocacy of the HPV vaccine, highlighting its efficacy in preventing HPV- related diseases, the critical role of vaccination in public health, and positive vaccination outcomes. These messages distinctly advocate for the HPV vaccine's utilization, reinforcing its importance in health prevention measures. Just got my HPV vaccine today! Feeling grateful for the science that protects us from cervical cancer. Highly recommend it to everyone eligible! opposite Explicitly challenge or critique the Human Papillomavirus (HPV) vaccine, focusing on concerns related to its efficacy, safety, or potential adverse effects. The critical perspective must be distinctly directed at the HPV vaccine in isolation, excluding broader criticisms of vaccines at large, political deliberations concerning vaccination policies, or financial considerations associated with vaccine procurement. I'm skeptical about the HPV vaccine due to the side effects I've read about. More research is needed before making it a widespread recommendation. Let's be cautious. unknown Neither explicitly endorse nor directly oppose the HPV vaccine. This category encapsulates tweets that voice opposition stemming from broad anti-vaxxers, policies, or concerns about the cost, without specifically expressing the advocate or opposite of the HPV vaccine itself. It also includes tweets that are too ambiguous, neutral, or irrelevant to ascertain a clear position concerning the HPV vaccine. Why isn't the HPV vaccine free for everyone? If it's so important, shouldn't the government cover the cost? Still trying to understand the policy behind this. Stance analysis with deep learning models Model Selection To achieve optimal results, we first needed to select the best-performing model. We compared several traditional NLP classification models and LLMs. Among traditional models, we considered long short-term memory (LSTM) [37], Bidirectional Encoder Representations from Transformers (BERT) [38], and DistilBERT [39]. For LLMs, we compared the open-source models General-purpose Multilingual Encoder-Decoder model . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint with Gemma-2-2b [40], Gemma-2-9b, and Llama-3.1-8b [41]. We also evaluated Gemini Pro 1.0 [42] from Google AI Studio because of free-trial. For the LSTM model, we designed a two-layer bidirectional architecture. Tweets were tokenized using the Python package Ginza [43]. Due to the manageable parameter size of this model, we trained it from scratch. For the BERT models, we selected Tohoku BERT [44] and Line DistilBERT [45] as baselines. A classification head was added to the first token of the final hidden layer of these models, and fine-tuning was performed using our tweet data, while keeping the BERT parameters frozen. For the LLMs, computational resource constraints prevented us from fine-tuning models with more than 10 billion parameters, and budgetary constraints precluded fine-tuning using APIs such as ChatGPT [46] or Claude [47]. Therefore, we opted for open-source models and tested both baseline and Quantization and Low-Rank Adaptation (QLoRA)-fine- tuned versions on the tweet data [48]. For Gemini Pro 1.0, we directly finetuned the model within the Google AI studio. Given the significant class imbalance in the dataset, the 5,758 annotated tweets were split into training, validation, and test sets, with the proportion of each category maintained across the splits. To ensure a robust evaluation, 100 tweets from each category were randomly selected to form the test set. The remaining data within each category was then partitioned into training and validation sets at a ratio of 4:1. Due to the training sample size

(<500) imposed by Gemini 1.0 Pro, a subset of 498 labeled tweets (distributed equally across the three categories at a ratio of 166:166:166) was utilized for fine-tuning the model for the HPV vaccine stance classification task. Model performance was assessed using standard evaluation metrics, namely precision, recall, and F1-score. Details about the labeled dataset are shown in Table 2. Table 2 number of labeled tweets of different stances in the dataset category train set validation set test set advocate 2652 663 100 oppose 477 120 100 unknown 1236 310 100 To further investigate the performance boundaries of the optimal model, we explored the impact of varying category distributions within the training data. We experimented with different ratios of advocate, oppose, and unknown categories, specifically 166:166:166, 150:200:150, 125:250:125, 100:300:100, 75:350:75, and 50:400:50. For each ratio, data . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint was randomly sampled three times, and the model's performance was evaluated, with the

averaged to ensure robustness. Upon identifying the optimal category ratio, we fine-tuned the best-performing model five times using this ratio, each time with a new random data sample. The remaining labeled data was utilized for performance evaluation, and the three models exhibiting the highest performance were selected for final inference. For each unlabeled tweet, these three models generated independent predictions. If two or more models agreed on the predicted label, that label was assigned to the tweet. In cases where all three models produced different predictions, the tweet was classified as "unknown." Time series analysis of stances Following the stance classification of all tweets, a comprehensive time series analysis was performed on the extracted data to examine the evolution of public sentiment towards the HPV vaccine in Japan between 2011 and 2021. To mitigate the influence of outliers, the monthly counts of tweets advocating for and opposing the HPV vaccine were log- transformed. The difference between these counts was then calculated to capture the dynamic shifts and relative strength of each stance over time. To pinpoint significant structural breaks within the time series data, the Pruned Exact Linear Time (PELT) algorithm [49], a computationally efficient method for change point detection, was employed. These identified change points corresponded to major shifts in public opinion regarding the HPV vaccine, potentially linked to specific events or information dissemination campaigns. By juxtaposing the time series analysis results with a timeline of key news events and public health policies, a comprehensive interpretation of the observed trends was achieved. This approach facilitated a deeper understanding of the factors influencing public acceptance of the HPV vaccine and provided empirical evidence to inform future public health interventions and policy decisions. LDA topic modeling analysis In this study, we employed Latent Dirichlet Allocation (LDA) [49] topic modeling, a widely used technique for extracting latent thematic information from large volumes of text data, to analyze the Japanese tweets related to the HPV vaccine between 2011 and 2021. We aimed to gain a deeper understanding of the specific perspectives held by Japanese Twitter users across different stances towards the HPV vaccine by LDA modeling. To determine the optimal number of topics that best represented the thematic structure of the dataset, we experimented with various topic numbers ranging from 1 to 50. For each configuration, we evaluated the model's performance using perplexity and coherence scores [50]. Details about optimal topic number selection is in appendix 4. After selecting the optimal number of topics based on a balance of lower perplexity and higher coherence, we conducted further qualitative analysis to refine and interpret the final thematic content. Similar to Niu et al. [51], we calculated the monthly average expectation of tweets belonging to different topics for fine-grain time series analysis. In line with prior research indicating that misinformation is an important factor for vaccine hesitation [20,23,52], we aimed to investigate the impact of misinformation on the HPV vaccine. First, we identified topics that were likely to contain misinformation through group discussion, focusing on tweets with an estimated probability of over 80% of belonging to . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint those topics. We then utilized the Claude LLM model [47] to assess whether each tweet was attempting to disseminate misinformation regarding the HPV vaccine. To validate the accuracy of the model's classification, we randomly selected 25 tweets categorized as misinformation and 25 categorized as non-misinformation by the model. Three independent volunteers evaluated the credibility of the information in these tweets by cross-referencing reputable news sources and data from the MHLW [53]. The verification process was repeated three times to estimate the model's classification accuracy. Explore relationship between HPV vaccination and COVID-19 Considering the chronological overlap between the duration of data collection and the COVID-19 pandemic, a significant historical health event, this research further investigated the potential relationship between HPV vaccination and COVID-19. Specifically, the

was to determine whether the COVID-19 pandemic had an impact on public attitudes towards HPV vaccination. From our dataset, all tweets comprising the keyword "COVID-19" in both Japanese and English were extracted. In addition to the time series analysis and LDA topic modeling previously described the correlation between all HPV vaccine-related tweets and the tweets incorporating the "COVID-19" keywords was examined. Moreover, specific time points where the total number of tweets and the tweets containing “COVID-19” keywords exhibited simultaneous increases were investigated. To estimate the percentage of tweets pertaining to the pivotal events, 100 tweets were randomly selected thrice from the time points. The number of tweets related to the paramount event was verified by three different volunteers, and the percentage of related tweets was calculated by averaging the three results and computing the confidence interval (CI). To gain a deeper understanding of the spikes in the data, the influence of COVID-19 vaccine-related pivotal events on the peak time points was also assessed. Furthermore, we investigated the potential causality between the stance towards the HPV vaccine and the stance towards the COVID-19. Specifically, we analyzed tweets containing "COVID-19" keywords and expressing either an advocate or opposed stance. We employed a logic analysis by LLM to determine whether the tweet utilized the HPV vaccine as an illustrative example to convey the author's perspective on the COVID-19, vice versa, or neither. The claude-3-opus model was directly employed for inference, with the prompt instructing it to read the tweet and provide an output of "HPV to COVID" if the tweet used the HPV vaccine as an example to express an idea about the COVID-19 vaccine, "COVID to HPV" if the reverse was true, or "Not related" if neither was applicable. The classified tweets were tallied weekly and visually represented to demonstrate the results.

Model Selection The comparison of all models we tried are shown in Figure 1. F1 scores of larger models get better performance in most cases, except for Llama 3.1. This may be related to the fact that it was not finetuned on Japanese corpora. Finetuned Gemini 1.0 pro got the best results in our experiment, and we decided to do the following steps based on Gemini 1.0 pro. Detailed

are in Appendix 2 and 3. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 1 . Comparison of weighted F1 scores among all the models on 300 test data. Bars show the average and standard deviation on three test datasets selected by different random seeds. The Numbers above the bars show the average of weighted F1 scores. As the allowed number of tweets for finetuning Gemini 1.0 pro is fixed (<500), we experimented with different data ratios to find the most effective category ratio to finetune the model. The results indicated that the model performed best when the ratio of advocate, opposite, and unknown categories was 150:200:150. This ratio consistently appeared in validation experiments, achieving the highest average F1-score of 0.968, as shown in Figure 2. Detailed model performance for each ratio can be found in Appendix 1. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 2 Comparison of weighted F1 scores among models trained with different ratios of categories in the training data. Bars show the average and standard deviation on three test datasets selected by different random seeds. The Numbers above the bars show the average of weighted F1 scores. After conducting five rounds of random data selection and fine-tuning using the optimal data ratio, the performance of the fine-tuned Gemini 1.0 Pro model was evaluated on the remaining labeled data. The evaluation results showed a significant improvement in precision, recall, and F1-score for the fine-tuned model. The average F1-score of the three best-performing models was 0.924, with a precision of 0.928 and recall of 0.924. In comparison, the baseline performance of the untuned Gemini 1.0 Pro model on the same test dataset showed an F1-score of 0.781, precision of 0.829, and recall of 0.796. This comparison clearly demonstrates the importance of fine-tuning in enhancing the model's performance for specific tasks. When classifying the unlabeled Japanese tweets, the results from the three best models were mostly consistent. Among all tweets, 86.85% received three identical result labels, and 13.06% received two identical result labels. In cases where all three labels were completely inconsistent, these tweets were classified as unknown, accounting for 0.09% of all tweets. Time series analysis of stance We first analyzed the monthly trend of public stances towards HPV vaccine from 2011 to 2021, as shown in Figure 3. Tweets for advocate and opposed stances both increased rapidly within a decade, while advocation gradually dominated the stances. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 3 Log-scaled monthly number of tweets of advocate and oppose stances. The time series analysis of difference between advocate and opposed stances is visualized in Figure 4, unveiled several pivotal junctures marked by substantial shifts in the disparity between pro- and anti-HPV vaccination stances, suggesting notable fluctuations in public opinion. Overall, we can see there are three periods of fluctuations and three periods being relatively stable in stances. Employing the PELT algorithm, we successfully discerned structural change points within the data, aligning closely with significant shifts in public attitudes towards HPV vaccines. Notably, these change points were observed in 2013, 2016, and 2020, coinciding with key public health events and policy developments. In 2013, the Japanese government's decision to suspend its recommendation for the HPV vaccine [54] precipitated a marked decline in public confidence. Subsequently, in late 2016, widespread discourse surrounding vaccine safety was triggered by legal action taken by individuals alleging adverse side effects [55]. Finally, 2020 witnessed a resurgence in public advocacy for HPV vaccination, with numerous petitions urging the government to reinstate active recommendations, prompting a policy review [56]. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 4 Monthly differences between number of tweets of advocate and oppose stances. The significant low and high periods are marked in gray. LDA topic modeling analysis We applied LDA modeling on the tweets of advocate, oppose, unknown stances separately. The best topic number for advocate and unknown are three, and that for oppose is four. The details of top words and weights are shown in Appendix 4. We calculated the ratio of the monthly expectations of tweets belonging to different topics to get fine-grained insight to the weight of topics in each stance. The distribution of various themes within the "advocate" stance is illustrated in Figure 5. The proportion of "Scientific and media discourse on HPV Vaccine Safety" (Topic 1) underwent a significant increase until 2015, subsequently experiencing a gradual decline. Conversely, the proportion of "HPV vaccine effectiveness and broader public health measures" (Topic 2) diminished, reaching its lowest point around 2015, after which it exhibited a consistent increase. The proportion of "Policy and advocacy for HPV vaccination promotion" (Topic 3) remained relatively stable, although a slight upward trend has been observed since 2019. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 5 Monthly change of ratio of different topics in tweets of advocate stance from 2011 to 2021. In the analysis of tweets expressing the "opposing" stance in Figure 6, "Skepticism and opposition to vaccination" (Topic 2) demonstrated notable peaks in weight during the years 2013 and 2015, followed by a gradual decline and eventual stabilization of topic weight. The proportion of "Scientific warnings and public health risks" (Topic 4) experienced a surge in late 2012, followed by a sustained increase throughout the decade, ultimately becoming the dominant theme within the opposing stance. Figure 6 Monthly change of ratio of different topics in tweets of opposed stance from 2011 to 2021. In the tweets classified as "unknown" by stance shown in Figure 7, "HPV Vaccine Efficacy and Public Health Initiatives" (Topic 2) was the dominant theme prior to the year 2013, however, it experienced a sharp decline thereafter. Conversely, the proportion of tweets expressing "Opposition, Misinformation, and Activism Surrounding HPV Vaccination" (Topic 3) displayed a gradual increase and has become the predominant theme since 2018. Notably, the theme of "HPV Vaccine Safety and Governmental Oversight" (Topic 1) exhibited a gradual increase throughout the entire period under consideration. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 7 Monthly change of ratio of different topics in tweets where the stances are unknown from 2011 to 2021. In tweets of opposed stance, we identified "Scientific warnings and public health risks" (Topic 4) as the most likely to contain misinformation about the HPV vaccine. We extracted tweets with over 80% probability of belonging to this topic, yielding a total of 3,001 tweets. The misinformation classification results, obtained using the Claude LLM, are presented in Figure 8. The classification accuracy, determined through random sampling and verification conducted thrice, ranged from 89.74% to 100% (CI=95%). Our findings reveal that the prevalence of misinformation increased in most years, with exceptions in 2014 and 2015. The ratio of misinformation in HPV vaccine-related tweets exhibited a distinctive pattern from 2011 to 2021. The trend initiated with a sharp spike in 2012, followed by a rapid decline to its nadir in 2015. Subsequently, the ratio gradually increased, reaching a secondary, albeit lower, peak in 2018, after which it began a gradual decline. This temporal pattern suggests fluctuating levels of misinformation dissemination throughout the study period, with notable inflection points that warrant further investigation. Figure 8 Count and ratio of tweets classified as misinformation. The bars shows the total number of tweets found as misinformation each year, and the line shows the ratio of the tweets containing misinformation within the total number of tweets each year. Relationship between HPV vaccination and COVID-19 We conducted a weekly time series analysis on the stances of tweets containing the COVID- 19 keywords, as illustrated in Figure 9. There are no tweets with “COVID-19” keyword in . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint the first two weeks of 2020, so all the calculations began from the 3rd week. The Pearson correlation coefficient (r) was 0.793 (p = 1.59 ∗ 10−23), indicating a relatively high correlation between all tweets and those containing “COVID-19” keywords. The time lag for the highest cross correlation was zero. Several peaks in both all tweets and tweets containing “COVID-19” keywords were observed in 2021 (week 3, week 34~35, and week 45), with the 34~35th week exhibiting the highest peak during the entire study period, which coincided with a campaign to call for resumption of active HPV vaccination recommendations [57] (37.0%~39.8% HPV vaccine-related tweets; 1.5% ~ 9.1% tweets with “COVID-19” keywords; CI=95%). Figure 9 Number of all tweets related to HPV vaccine (gray), and the tweets containing the “COVID-19” keyword (blue), with key events related to COVID-19 vaccine marked. The points beginning with “N” are negative key events that may lead to opposition to COVID-19 vaccines, and the points beginning with “P” are positive key events that may lead to advocacy to COVID-19 vaccines. A comparative analysis was performed to examine the differences between advocate and opposed stances in both the overall tweet corpus and the subset containing the "COVID-19" keyword, as illustrated in Figure 10. The correlation coefficient (r) between advocate and oppose sentiment differences is substantial (r = 0.723, p = 6.09 ∗ 10−18), with no temporal lag observed in the highest cross-correlation. In the broader context of all HPV vaccine- related tweets, advocate sentiment generally prevailed over oppose sentiment. However, for tweets containing "COVID-19" keywords, the dominant stance fluctuated over time. Both datasets exhibited a pronounced peak in advocacy during weeks 34-35 of 2021, while oppose sentiment dominated in week 22 of 2020. This pattern was consistent across the entire tweet corpus and the "COVID-19" keyword subset. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 10 The difference between tweets of advocate stance and oppose stance for all HPV vaccine tweets (gray) and tweets containing the “COVID-19” keyword (blue), with key events related to COVID-19 vaccine marked. The points beginning with “N” are negative key events that may lead to opposition to COVID-19 vaccines, and the points beginning with “P” are positive key events that may lead to advocacy to COVID-19 vaccines. We additionally assessed the impact of COVID-19 vaccine-related events on the volume of all HPV vaccine-related tweets and tweets containing the keywords "COVID-19." The details of the key events were gathered from [58] and are provided in Table 1 of Appendix 4. As illustrated in Figure 9, it is evident that the key events do not consistently result in peaks in tweet volume for both all HPV vaccine-related tweets and tweets containing the keywords "COVID-19." Furthermore, as depicted in Figure 10, the positive or negative events do not always lead to an increase in the number of supportive or opposing stances. LDA was conducted on tweets containing COVID-19 keywords of advocating and opposing stances separately. The detailed results are presented in Appendix 4. To investigate nuanced changes in topic weights over time, the expectation of tweets belonging to different topics for both advocate and oppose stances was calculated and visualized in Appendix 4. Analysis of this data revealed no significant changes in the proportion of different topics during 2020 and 2021. Finally, we investigated the potential causal relationship between attitudes towards HPV and COVID-19 vaccines by logic analysis. To evaluate the accuracy of Claude doing the classification task without finetuning, we randomly sampled 100 tweets three times and let three volunteers judge the correctness. The accuracy is relatively high (72.92%~91.74%, CI=95%). The weekly classification results are illustrated in Figure 11. The "HPV to COVID" category predominated throughout the entire study period for both supportive and opposing stances. Notably, tweets advocating for vaccines consistently outnumbered those opposing them across all categories. . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint Figure 11 Weekly number of tweets with “COVID-19” keywords, using HPV vaccine as example to share stance to COVID-19 vaccine (“HPV to COVID”), using COVID-19 vaccine as example to share stance to HPV vaccine (“COVID to HPV”), or neither (“Not related”). Left: result on tweets with advocate stance; Right: result on tweets with opposed stance.

Principal Results In this work, we employed traditional NLP models and LLMs to perform stance analysis on social media content, focusing on the public discourse around the HPV vaccine. Our results demonstrate that stance analysis using LLMs provides a more nuanced understanding compared to traditional sentiment analysis. Unlike traditional sentiment analysis, which captures general emotional tones, LLMs can identify specific stances—such as supportive, opposing, or neutral positions—allowing for a deeper comprehension of public attitudes and decision-making processes. Moreover, the successful application of LLMs highlights their potential as powerful tools for social media analysis, providing deep insights into public health discourse by capturing more complex logic like causality and context-specific nuances. These results not only validate the effectiveness of our methodology but also provide strong empirical support for using large language models in similar tasks in the future. In the context of our findings about HPV vaccine, we observed increase in public advocacy and vaccine confidence towards HPV vaccine, although fair propagation and communication is still a key factor to further drive the HPV vaccination. The WHO's 3Cs model (Confidence, Complacency, Convenience) offers a comprehensive framework for understanding the factors influencing vaccine hesitancy in Japan. Confidence pertains to trust in the vaccine's safety, efficacy, and the healthcare system responsible for its administration, and plays the key role in the stance towards HPV vaccine in Japan. In our analysis, we observed significant fluctuations in public stances towards the HPV vaccine, which were closely linked to government policy decisions and media coverage. The 2013 suspension of HPV vaccine recommendations by the Japanese government was a key moment that led to a sharp decline in public trust. This decline can be attributed to increased public uncertainty and fears regarding vaccine safety, as the time series analysis shows. The LDA topic modeling results also support this, showing a decline . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint in the proportion of tweets related to "HPV Vaccine Efficacy and Public Health Initiatives" (Topic 2) where the stance is unknown after 2013, while "Opposition, Misinformation, and Activism Surrounding HPV Vaccination" (Topic 3) gradually became the predominant theme since 2018, highlighting growing public distrust and misinformation. Conversely, the resurgence in advocacy in 2020 reflected an improvement in public confidence, likely influenced efforts to reinstate vaccine recommendations. This is reflected in the increased proportion of tweets related to "Scientific and Media Discourse on HPV Vaccine Safety" within the 'advocate' stance, which showed a significant increase until 2015, followed by a gradual decline, indicating ongoing advocacy efforts. Our findings illustrate that confidence is highly sensitive to policy decisions and that restoring public trust after a decline requires consistent and clear communication efforts. Furthermore, we explored the role of misinformation in undermining public confidence in the HPV vaccine. The spike in the ratio of tweets containing misinformation in 2012 likely contributed to the significant drop in vaccine confidence observed in the following year, which aligns with findings from other studies [22,31,59]. From 2014 onwards, the ratio of misinformation exhibited slight changes but remained consistently present, and the overall number of misinformation tweets increased in most years, indicating the persistent and long-lasting impact of misinformation. Addressing misinformation is therefore essential for restoring and maintaining public trust in vaccines, as unchecked misinformation can severely erode public confidence and contribute to vaccine hesitancy. Complacency refers to the perceived necessity of vaccination, which diminishes when individuals consider the risk of vaccine-preventable diseases to be low. Eric et al. found that "not necessary" was consistently among the top reasons cited by parents for not vaccinating their children against HPV between 2010 and 2020 [60]. The topic "HPV Vaccine Efficacy and Public Health Initiatives" under the 'unknown' stance experienced a sharp decline post- 2013, indicating diminished public engagement and perceived necessity. The perceived low risk of HPV, combined with concerns about side effects, may contributed to a reduced sense of urgency regarding vaccination. However, our time series analysis showed that complacency began to decrease in 2020 as advocacy efforts intensified, and the public became more aware of the risks of HPV and the benefits of vaccination. The upward trend in the topic "HPV Vaccine Effectiveness and Broader Public Health Measures" within the 'advocate' stance since 2015 further reflects this shift towards increased awareness and decreasing complacency. This shift underscores the importance of maintaining proactive communication about the risks of vaccine-preventable diseases to counteract complacency. Convenience encompasses factors such as the availability, affordability, and accessibility of vaccination. In our study, the importance of convenience became evident when analyzing periods of increased advocacy for HPV vaccination. For example, the resurgence of advocacy in 2020 coincided with governmental initiatives aimed at improving access to the HPV vaccine, including the reintroduction of public recommendations and the enhancement of vaccination services. This alignment is reflected in the relatively stable proportion of tweets related to 'Policy and Advocacy for HPV Vaccination Promotion' within the 'advocate' stance, which exhibited a slight upward trend since 2019. These changes may . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint encourage efforts to facilitate access to vaccination services, and play a critical role in influencing public willingness to get vaccinated over time. Beside the findings about HPV vaccine, we also noticed that the stances towards the HPV vaccine may influence stances towards the COVID-19 vaccine during the COVID-19 pandemic. Specifically, our logic analysis revealed that increased confidence in the HPV vaccine may lead to higher confidence in the COVID-19 vaccine. Tweets categorized under 'HPV to COVID'—which used the HPV vaccine as a reference point for the COVID-19 vaccine stance—predominated throughout the study period for both supportive and opposing stances. Notably, advocacy for vaccines consistently outnumbered opposition across all categories, suggesting that rising confidence in one vaccine may positively impact confidence in other vaccines. For future work, we plan to explore why LLMs outperform traditional models, focusing on factors like model size and external information. We are also considering building a LLM to identify misinformation about HPV vaccine with higher accuracy. These efforts will refine our understanding of LLMs and inform better public health strategies.

We admit that our work has many limitations, including the fact that social media users may not represent the entire population, which limits the generalizability of our findings. Additionally, the models are not 100% accurate and may introduce classification errors, particularly when dealing with nuanced or ambiguous stances and logics. Furthermore, the

used for collecting data may have led to the omission of relevant tweets or added irrelevant tweets, potentially impacting the comprehensiveness of our analysis. Comparison with Prior Work Our work stands out from prior studies due to the scale and depth of the dataset, which includes ten years of tweets in Japanese, providing a comprehensive view of public attitudes over time. We applied both traditional NLP methods and cutting-edge LLMs, allowing us to analyze nuanced shifts in public sentiment and stance with greater precision. Unlike traditional surveys, which provide snapshots of attitudes, our approach captures real-time changes in public discourse. Additionally, compared to studies relying on Google search trends, our analysis offers more fine-grained insights into specific stances and topics, highlighting not just information-seeking behavior, but the underlying attitudes that drive vaccine hesitancy or advocacy.

This study highlights the effectiveness of LLMs for stance analysis in understanding public attitudes towards HPV vaccination. By applying the WHO's 3Cs model, we contextualized the complex factors influencing stances towards HPV vaccine. Public confidence fluctuated significantly in response to government actions and media coverage, showing the sensitivity of trust to policy decisions, while complacency was impacted by perceived risks and proactive advocacy. Convenience was crucial for improving vaccine accessibility, shaping public willingness to get vaccinated. Moreover, our findings suggest that confidence in one vaccine, such as HPV, may influence confidence in others, like COVID-19, highlighting interconnected public health narratives. Addressing misinformation, enhancing . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint communication, and improving accessibility remain key strategies for building trust and reducing hesitancy. Future public health strategies can benefit from these insights to design effective interventions aimed at boosting vaccine confidence and uptake.

This work was supported by the JST SPRING (grant number JPMJSP2110) and Google PhD fellowship. Conflicts of Interest none declared. Abbreviations BERT: Bidirectional Encoder Representations from Transformers CI: confidence interval DL: deep learning Gemma: General-purpose Multilingual Encoder-Decoder model HPV: human papillomavirus LDA: latent Dirichlet allocation LLM: large language model LSTM: long short-term memory MHLW: Japanese Ministry of Health, Labour and Welfare NLP: natural language processing PELT: pruned exact linear time QLoRA: Quantization and Low-Rank Adaptation

1. Cervical cancer. Available from: https://www.who.int/health-topics/cervical-cancer [accessed Jul 17, 2024] 2. WHO Immunization Data portal - Global. Immunization Data. Available from: https://immunizationdata.who.int/ [accessed Jul 17, 2024] 3. Documents for the national conference of prefectural officials (temporary special grant for emergency promotion of vaccination against cervical cancer, etc.)(in Japanese). Available from: https://www.mhlw.go.jp/bunya/kenkou/other/101209.html [accessed Jul 17, 2024] 4. Immunization Act (in Japanese). Available from: https://elaws.e- gov.go.jp/document?lawid=323AC0000000068 [accessed Jul 17, 2024] 5. Response to Routine Immunization for Human Papillomavirus Infection (Recommendation)(in Japanese). Available from: https://www.mhlw.go.jp/stf/shingi2/0000091963.html [accessed Jul 17, 2024] 6. Sipp D, Frazer IH, Rasko JEJ. No Vacillation on HPV Vaccination. Cell United States; 2018 Mar 8;172(6):1163–1167. PMID:29522737 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint 7. Hanley SJB, Yoshioka E, Ito Y, Kishi R. HPV vaccination crisis in Japan. Lancet 2015 Jun 27;385(9987):2571. PMID:26122153 8. Ueda Y, Enomoto T, Sekine M, Egawa-Takata T, Morimoto A, Kimura T. Japan’s failure to vaccinate girls against human papillomavirus. Am J Obstet Gynecol Elsevier; 2015 Mar 1;212(3):405–406. PMID:25434842 9. Tanaka Y, Ueda Y, Yoshino K, Kimura T. History repeats itself in Japan: Failure to learn from rubella epidemic leads to failure to provide the HPV vaccine. Hum Vaccin Immunother 2017 Aug 3;13(8):1859–1860. PMID:28604161 10. Sekine M, Kudo R, Yamaguchi M, Hanley SJB, Hara M, Adachi S, Ueda Y, Miyagi E, Ikeda S, Yagi A, Enomoto T. Japan’s Ongoing Crisis on HPV Vaccination. Vaccines (Basel) 2020 Jul 6;8(3). PMID:32640691 11. Larson HJ, Wilson R, Hanley S, Parys A, Paterson P. Tracking the global spread of vaccine sentiments: the global response to Japan’s suspension of its HPV vaccine recommendation. Hum Vaccin Immunother 2014 Nov 13;10(9):2543–2550. PMID:25483472 12. Yagi A, Ueda Y, Kakuda M, Nakagawa S, Hiramatsu K, Miyoshi A, Kobayashi E, Kimura T, Kurosawa M, Yamaguchi M, Adachi S, Kudo R, Sekine M, Suzuki Y, Sukegawa A, Ikeda S, Miyagi E, Enomoto T, Kimura T. Cervical Cancer Protection in Japan: Where Are We? Vaccines (Basel) Switzerland; 2021 Nov 1;9(11). PMID:34835194 13. Sekine M. Japanese Crisis of HPV Vaccination. Int J Pathol Clin Res ClinMed International Library; 2016 Jun 30;2(2). 14. Ministry of Health, Labour, and Welfare. Global Advisory Committee statement on safety of HPV vaccines. Available from: https://www.mhlw.go.jp/file/05-Shingikai- 10601000-Daijinkanboukouseikagakuka-Kouseikagakuka/0000125190.pdf [accessed Jul 18, 2024] 15. Namba M, Kaneda Y, Kawasaki C, Shrestha R, Tanimoto T. Underlying background of the current trend of increasing HPV vaccination coverage in Japan. Glob Health Med Japan; 2023 Aug 31;5(4):255–256. PMID:37655180 16. Lelliott M, Sahker E, Poudyal H. A review of parental vaccine hesitancy for human Papillomavirus in Japan. J Clin Med 2023 Mar 2;12(5). PMID:36902790 17. Miyagi E. Human papillomavirus (HPV) vaccination in Japan. J Obstet Gynaecol Res Wiley; 2024 Jul 9; PMID:38979785 18. Hayashi Y, Shimizu Y, Netsu S, Hanley S, Konno R. High HPV vaccination uptake rates for adolescent girls after regional governmental funding in Shiki City, Japan. Vaccine Netherlands; 2012 Jun 27;30(37):5547–5550. PMID:22749837 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint 19. Yagi A, Ueda Y, Kimura T. HPV Vaccine Issues in Japan: A review of our attempts to promote the HPV vaccine and to provide effective evaluation of the problem through social-medical and behavioral-economic perspectives. Vaccine Netherlands; 2024 Apr 13; PMID:38616440 20. Terada M, Okuhara T, Nagasawa T, Okada H, Goto E, Kiuchi T. Public perception of the resumption of HPV vaccine recommendation in Japan: Twitter content analysis. Health Promot Int England; 2023 Dec 1;38(6). PMID:37966160 21. Ueda N, Yokouchi R, Onoda T, Ogihara A. Characteristics of Articles About Human Papillomavirus Vaccination in Japanese Newspapers: Time-Series Analysis Study. JMIR Public Health Surveill Canada; 2017 Dec 19;3(4):e97. PMID:29258972 22. Dunn AG, Leask J, Zhou X, Mandl KD, Coiera E. Associations Between Exposure to and Expression of Negative Opinions About Human Papillomavirus Vaccines on Social Media: An Observational Study. J Med Internet Res 2015 Jun 10;17(6):e144. PMID:26063290 23. Teoh D. The Power of Social Media for HPV Vaccination--Not Fake News! Am Soc Clin Oncol Educ Book American Society of Clinical Oncology (ASCO); 5 2019;(39):75–78. 24. Pedersen EA, Loft LH, Jacobsen SU, Søborg B, Bigaard J. Strategic health communication on social media: Insights from a Danish social media campaign to address HPV vaccination hesitancy. Vaccine Elsevier BV; 6 2020;38(31):4909–4915. 25. Jwa S, Yuyama Y, Yoshida H, Hamazaki T. A favorable impression of vaccination leads to a better vaccination rate for the human papillomavirus vaccine: A Japanese questionnaire survey investigation. Vaccine X England; 2022 Dec 23;13:100254. PMID:36686401 26. Tsuda K, Yamamoto K, Leppold C, Tanimoto T, Kusumi E, Komatsu T, Kami M. Trends of Media Coverage on Human Papillomavirus Vaccination in Japanese Newspapers. Clin Infect Dis United States; 2016 Dec 15;63(12):1634–1638. PMID:27660235 27. Louis A. Natural language processing for social media. Comput Linguist Assoc Comput Linguist MIT Press; 2016 Dec 1;42(4):833–836. 28. DucharmeRéjean. A neural probabilistic language model. J Mach Learn Res JMLR.orgPUB6573; 2003 Mar 1; doi: 10.5555/944919.944966 29. Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. the Journal of machine Learning research JMLR. org; 2003;3:993–1022. 30. Du J, Luo C, Shegog R, Bian J, Cunningham RM, Boom JA, Poland GA, Chen Y, Tao C. Use of Deep Learning to Analyze Social Media Discussions About the Human Papillomavirus Vaccine. JAMA Netw Open 2020 Nov 2;3(11):e2022025. PMID:33185676 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint 31. Melton CA, White BM, Davis RL, Bednarczyk RA, Shaban-Nejad A. Fine-tuned Sentiment Analysis of COVID-19 Vaccine-Related Social Media Data: Comparative Study. J Med Internet Res 2022 Oct 17;24(10):e40408. PMID:36174192 32. Qorib M, Oladunni T, Denis M, Ososanya E, Cotae P. Covid-19 vaccine hesitancy: Text mining, sentiment analysis and machine learning on COVID-19 vaccination Twitter dataset. Expert Syst Appl 2023 Feb;212:118715. PMID:36092862 33. Niu Q, Liu J, Kato M, Shinohara Y, Matsumura N, Aoyama T, Nagai-Tanima M. Public Opinion and Sentiment Before and at the Beginning of COVID-19 Vaccinations in Japan: Twitter Analysis. JMIR Infodemiology JMIR Infodemiology; 2022 May 9;2(1):e32335. 34. Tomaszewski T, Morales A, Lourentzou I, Caskey R, Liu B, Schwartz A, Chin J. Identifying False Human Papillomavirus (HPV) Vaccine Information and Corresponding Risk Perceptions From Twitter: Advanced Predictive Models. J Med Internet Res 2021 Sep 9;23(9):e30451. PMID:34499043 35. Radwan A, Amarneh M, Alawneh H, Ashqar HI, AlSobeh A, Magableh AAAR. Predictive analytics in mental health leveraging LLM embeddings and machine learning models for social media analysis. Int J Web Serv Res IGI Global; 2024 Feb 14;21(1):1–22. 36. Tweepy. Available from: https://www.tweepy.org/ [accessed Oct 6, 2024] 37. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput MIT Press - Journals; 1997 Nov 15;9(8):1735–1780. PMID:9377276 38. Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional Transformers for language understanding. arXiv [csCL]. 2018. Available from: http://arxiv.org/abs/1810.04805 [accessed Sep 22, 2024] 39. Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv [csCL]. 2019. Available from: http://arxiv.org/abs/1910.01108 [accessed Oct 6, 2024] 40. Gemma Team. Gemma 2: Improving open language models at a practical size. arXiv [csCL]. 2024. Available from: http://arxiv.org/abs/2408.00118 [accessed Oct 6, 2024] 41. Introducing Llama 3.1: Our most capable models to date. Meta AI. Available from: https://ai.meta.com/blog/meta-llama-3-1/ [accessed Oct 6, 2024] 42. Gemini Team. Gemini: A family of highly capable multimodal models. arXiv [csCL]. 2023. Available from: http://arxiv.org/abs/2312.11805 [accessed Oct 6, 2024] 43. GiNZA - Japanese NLP Library. GiNZA - Japanese NLP Library. Available from: https://megagonlabs.github.io/ginza/ [accessed Oct 6, 2024] 44. tohoku-nlp/bert-base-japanese-v3 · Hugging Face. Available from: https://huggingface.co/tohoku-nlp/bert-base-japanese-v3 [accessed Oct 6, 2024] . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint 45. line-corporation/line-distilbert-base-japanese · Hugging Face. Available from: https://huggingface.co/line-corporation/line-distilbert-base-japanese [accessed Oct 6, 2024] 46. Liu Y, Han T, Ma S, Zhang J, Yang Y, Tian J, He H, Li A, He M, Liu Z, Wu Z, Zhao L, Zhu D, Li X, Qiang N, Shen D, Liu T, Ge B. Summary of ChatGPT-related research and perspective towards the future of large language models. arXiv [csCL]. 2023. doi: 10.1016/j.metrad.2023.100017 47. Introducing the next generation of Claude. Available from: https://www.anthropic.com/news/claude-3-family [accessed Oct 6, 2024] 48. Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient Finetuning of Quantized LLMs. arXiv [csLG]. 2023. Available from: http://arxiv.org/abs/2305.14314 [accessed Oct 6, 2024] 49. Killick R, Fearnhead P, Eckley IA. Optimal detection of changepoints with a linear computational cost. arXiv [statME]. 2011. doi: 10.1080/01621459.2012.737745 50. Gan J, Qi Y. Selection of the optimal number of topics for LDA topic model-taking patent policy analysis as an example. Entropy (Basel) MDPI AG; 2021 Oct 3;23(10):1301. PMID:34682025 51. Niu Q, Liu J, Kato M, Nagai-Tanima M, Aoyama T. The Effect of Fear of Infection and Sufficient Vaccine Reservation Information on Rapid COVID-19 Vaccination in Japan: Evidence From a Retrospective Twitter Analysis. J Med Internet Res 2022 Jun 9;24(6):e37466. PMID:35649182 52. Zimet GD, Rosberger Z, Fisher WA, Perez S, Stupiansky NW. Beliefs, behaviors and HPV vaccine: Correcting the myths and the misinformation. Prev Med Elsevier BV; 11 2013;57(5):414–418. 53. Human papillomavirus infection - cervical cancer (uterine cervix cancer) and HPV vaccine (in Japanese). Available from: https://www.mhlw.go.jp/bunya/kenkou/kekkaku-kansenshou28/index.html [accessed Oct 6, 2024] 54. Ishizaki Y, Gomi H. Human papillomavirus vaccination and postural tachycardia syndrome, deconditioning and exercise-induced hyperalgesia: An alternate interpretation of the reported adverse reactions. J Obstet Gynaecol Res Australia; 2020 Mar 9;46(5):678–683. PMID:32153078 55. Bodily JM, Tsunoda I, Alexander JS. Scientific Evaluation of the Court Evidence Submitted to the 2019 Human Papillomavirus Vaccine Libel Case and Its Decision in Japan. Front Med Switzerland; 2020 Jul 29;7:377. PMID:32850893 56. Rigney J. HPV Vaccination in Japan: The Journey to Resuming a National Immunisation Programme. Available from: . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint https://cancerprevention.qmul.ac.uk/index.php/2022/05/27/hpv-vaccination-in- japan-the-journey-to-resuming-a-national-immunisation-programme/ [accessed Oct 6, 2024] 57. Takahiro K. We call for the resumption of active HPV vaccination recommendations. Change.org. 2021. Available from: https://www.change.org/p/%E7%94%B0%E6%9D%91%E5%8E%9A%E7%94%9F %E5%8A%B4%E5%83%8D%E5%A4%A7%E8%87%A3- hpv%E3%83%AF%E3%82%AF%E3%83%81%E3%83%B3%E3%81%AE%E7%A9% 8D%E6%A5%B5%E7%9A%84%E6%8E%A5%E7%A8%AE%E5%8B%A7%E5%A5% A8%E3%81%AE%E5%86%8D%E9%96%8B%E3%82%92%E6%B1%82%E3%82%8 1%E3%81%BE%E3%81%99 [accessed Oct 6, 2024] 58. NHK. NHK news record about COVID-19 in time series. NHK NEWS WEB. Available from: https://www3.nhk.or.jp/news/special/coronavirus/chronology/ [accessed Oct 6, 2024] 59. Zhang H, Wheldon C, Tao C, Dunn AG, Guo Y, Huo J, Bian J. How to improve public health via mining social media platforms: A case study of human papillomaviruses (HPV). Social Web and Health Research Cham: Springer International Publishing; 2019. p. 207–231. ISBN:9783030147136 60. Adjei Boakye E, Nair M, Abouelella DK, Joseph CLM, Gerend MA, Subramaniam DS, Osazuwa-Peters N. Trends in reasons for human Papillomavirus vaccine hesitancy: 2010-2020. Pediatrics American Academy of Pediatrics (AAP); 2023 Jun 1;151(6):e2022060410. PMID:37218460 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted October 8, 2024. ; https://doi.org/10.1101/2024.10.07.24315018doi: medRxiv preprint