Detection of Medical Conspiracy Theories with Limited Resources: Using Data from Prior Epidemics and LLMs

Detection of Medical Conspiracy Theories with Limited Resources: Using Data from Prior Epidemics and LLMs | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 21 April 2025 V1 Latest version Share on Detection of Medical Conspiracy Theories with Limited Resources: Using Data from Prior Epidemics and LLMs Authors : Ipek Baris Schlicht 0000-0002-5037-2203 [email protected] , Damir Korenčić , Berta Chulvi , Lucie Flek , and Paolo Rosso Authors Info & Affiliations https://doi.org/10.22541/au.174522531.12427873/v1 592 views 350 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Online dissemination of conspiracy theories (CTs) during epidemics poses significant risks to public health. This paper addresses the problem of detecting CTs in social media posts with an emphasis on the resource-constrained scenarios characterized by the absence of labeled datasets and the high cost of expert annotation. To address these challenges, we investigate resource-efficient methods for CT detection across multiple epidemics. We construct a novel dataset of CT-labeled social media posts covering four major epidemics from the past decade: Ebola, Zika, COVID-19, and Monkeypox. We conduct extensive experiments addressing four research questions: (1) the performance of BERT-like models on individual epidemics, (2) the ability to transfer knowledge from past epidemics to new ones, (3) the efficacy of zero-shot classification using Large Language Models (LLMs), and (4) the feasibility of training BERT-like models on LLM-labeled datasets. Our findings indicate that BERT-like models exhibit highly variable performance across epidemics. Transfer learning from prior epidemics can be effective and their performance can be improved with the number of prior datasets. Zero-shot LLM classifiers, including ensemble methods, achieve performance that matches or surpasses that of fine-tuned BERT-like models. Finally, we demonstrate that BERT-like models trained on LLM-labeled datasets achieve results close to the models trained on expert-annotated data, offering a practical alternative when expert labeling is infeasible. While automated methods can be useful for data analysis, we caution against automatization of content filtering due to the inherent difficulty of CT detection and the potential biases of language models. Supplementary Material File (yrjtwkvpjhcptksdhmwtdvwpwszzynzt.pdf) Download 728.32 KB Information & Authors Information Version history V1 Version 1 21 April 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords bert-like models conspiracy theory detection large language models medical domain transfer learning transformers Authors Affiliations Ipek Baris Schlicht 0000-0002-5037-2203 [email protected] Universitat Politecnica de Valencia View all articles by this author Damir Korenčić Institut Ruder Boskovic View all articles by this author Berta Chulvi Symanto Research View all articles by this author Lucie Flek Bonn-Aachen International Center for Information Technology View all articles by this author Paolo Rosso Universitat Politecnica de Valencia View all articles by this author Metrics & Citations Metrics Article Usage 592 views 350 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Ipek Baris Schlicht, Damir Korenčić, Berta Chulvi, et al. Detection of Medical Conspiracy Theories with Limited Resources: Using Data from Prior Epidemics and LLMs. Authorea . 21 April 2025. DOI: https://doi.org/10.22541/au.174522531.12427873/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.174522531.12427873/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9ffbd0040b3c593a',t:'MTc3OTQ1MjgyMQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();