Assessing News Credibility: Misinformation Content Indicators

Assessing News Credibility: Misinformation Content Indicators | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Assessing News Credibility: Misinformation Content Indicators Paula Carvalho, Danielle Caled, Mário J. Silva, Bruno Martins, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-173067/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The development of explainable news credibility prediction models is critical both for fighting the viral propagation of misinformation and improving media literacy. This work investigates a variety of content indicators approaching different semantic and discourse dimensions, such as title representativeness, reasoning errors, and sentiment intensity. These indicators were inspired by a previous study conducted for English news, aimed at reaching a collective consensus on which indicators could be widely used for predicting news credibility. This new study, performed by a multi-disciplinary team, relies on a corpus of 80 news articles from Portuguese mainstream and alternative news media, which were annotated by junior and senior journalists. The assessment of the corpus annotations provides insight into the prevalence of different indicators in each type of news source. The results obtained for Portuguese correlate in most cases with the ones reported for English, which motivates the adoption of common standards for supporting the collaborative development of interoperable automatic misinformation detection approaches. Computer Architecture and Engineering credibility indicators content indicators misinformation disinformation comparable news corpus media literacy Figures Figure 1 Full Text Due to technical limitations, full-text HTML conversion of this manuscript could not be completed. However, the latest manuscript can be downloaded and accessed as a PDF. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-173067","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":13129541,"identity":"ed87e627-0888-49c6-89a2-f6548e01f3fd","order_by":0,"name":"Paula Carvalho","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAqUlEQVRIiWNgGAWjYHACxgcJELpBglgtzAZIWgyI0sIGN5w4Lebsh59VPNxhY8/fwNx4g3HHH8JaLHvSzG4knklLnHGAsdmC8QwRthjc4GG7kdh2OMFA/mGbBGMbkVoKEtv+2xswMJKghSGx7QDjBqK1AP1iLJHYlgzxS+IZY8JagCH28OPPNjtgiLE/vPFxhxwRDkPhJTYQ1oGmhZEYLaNgFIyCUTDiAACumzSoVu+aqAAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0003-2884-1250","institution":"INESC-ID: Instituto de Engenharia de Sistemas e Computadores Investigacao e Desenvolvimento em Lisboa","correspondingAuthor":true,"prefix":"","firstName":"Paula","middleName":"","lastName":"Carvalho","suffix":""},{"id":13129542,"identity":"01f7d265-d774-400e-8480-eef8463eeca1","order_by":1,"name":"Danielle Caled","email":"","orcid":"","institution":"INESC-ID: Instituto de Engenharia de Sistemas e Computadores Investigacao e Desenvolvimento em Lisboa","correspondingAuthor":false,"prefix":"","firstName":"Danielle","middleName":"","lastName":"Caled","suffix":""},{"id":13129543,"identity":"23d42a5a-a090-4e18-b4b1-4a39153deb28","order_by":2,"name":"Mário J. Silva","email":"","orcid":"","institution":"INESC-ID: Instituto de Engenharia de Sistemas e Computadores Investigacao e Desenvolvimento em Lisboa","correspondingAuthor":false,"prefix":"","firstName":"Mário","middleName":"J.","lastName":"Silva","suffix":""},{"id":13129544,"identity":"4f6b704b-ceca-4427-b0a9-4ec886c828d4","order_by":3,"name":"Bruno Martins","email":"","orcid":"","institution":"INESC-ID: Instituto de Engenharia de Sistemas e Computadores Investigacao e Desenvolvimento em Lisboa","correspondingAuthor":false,"prefix":"","firstName":"Bruno","middleName":"","lastName":"Martins","suffix":""},{"id":13129545,"identity":"870c4e01-a558-4cc1-b48c-ee7638dbcab4","order_by":4,"name":"João Paulo Carvalho","email":"","orcid":"","institution":"INESC-ID: Instituto de Engenharia de Sistemas e Computadores Investigacao e Desenvolvimento em Lisboa","correspondingAuthor":false,"prefix":"","firstName":"João","middleName":"Paulo","lastName":"Carvalho","suffix":""},{"id":13129546,"identity":"71ce5d41-7dee-416c-9e78-7c46592674b2","order_by":5,"name":"Joaquim Carreira","email":"","orcid":"","institution":"Lusa - Agência de Notícias de Portugal","correspondingAuthor":false,"prefix":"","firstName":"Joaquim","middleName":"","lastName":"Carreira","suffix":""},{"id":13129547,"identity":"408732f3-71b6-4b83-9d26-7f29d54a1713","order_by":6,"name":"João Pedro Fonseca","email":"","orcid":"","institution":"Lusa - Agência de Notícias de Portugal","correspondingAuthor":false,"prefix":"","firstName":"João","middleName":"Pedro","lastName":"Fonseca","suffix":""},{"id":13129548,"identity":"4cb58cf0-be07-4ca9-bcb3-f33517df21f1","order_by":7,"name":"Teresa Gomes","email":"","orcid":"","institution":"​​​​​​​​​​​​​​​​​​​Lusa - Agência de Notícias de Portugal","correspondingAuthor":false,"prefix":"","firstName":"Teresa","middleName":"","lastName":"Gomes","suffix":""},{"id":13129549,"identity":"4bfe2e51-cfce-4766-bc90-568366162651","order_by":8,"name":"Pedro Camacho","email":"","orcid":"","institution":"Lusa - Agência de Notícias de Portugal","correspondingAuthor":false,"prefix":"","firstName":"Pedro","middleName":"","lastName":"Camacho","suffix":""}],"badges":[],"createdAt":"2021-01-28 10:52:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-173067/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-173067/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":6625664,"identity":"4fd0c319-51ce-4660-aadf-75aeba8a8d2b","added_by":"auto","created_at":"2021-03-04 22:11:12","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":256742,"visible":true,"origin":"","legend":"MAXQDA interface for text annotation.","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-173067/v1/8d7895907a51f47bafa96f72.png"},{"id":13675444,"identity":"63093dc4-4b75-4623-a965-74d4e87354a8","added_by":"auto","created_at":"2021-09-17 11:26:41","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":314500,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-173067/v1/1736e215-8b2c-4ef5-82b5-f09b312c43a4.pdf"}],"financialInterests":"","formattedTitle":"Assessing News Credibility: Misinformation Content Indicators","fulltext":[{"header":"Full Text","content":"Due to technical limitations, full-text HTML conversion of this manuscript could not be completed. However, the latest manuscript can be downloaded and \u003ca href='/article/rs-173067/latest.pdf' target='_blank'\u003e accessed as a PDF.\u003c/a\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"credibility indicators, content indicators, misinformation, disinformation, comparable news corpus, media literacy","lastPublishedDoi":"10.21203/rs.3.rs-173067/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-173067/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The development of explainable news credibility prediction models is critical both for fighting the viral propagation of misinformation and improving media literacy. This work investigates a variety of content indicators approaching different semantic and discourse dimensions, such as title representativeness, reasoning errors, and sentiment intensity. These indicators were inspired by a previous study conducted for English news, aimed at reaching a collective consensus on which indicators could be widely used for predicting news credibility. This new study, performed by a multi-disciplinary team, relies on a corpus of 80 news articles from Portuguese mainstream and alternative news media, which were annotated by junior and senior journalists. The assessment of the corpus annotations provides insight into the prevalence of different indicators in each type of news source. The results obtained for Portuguese correlate in most cases with the ones reported for English, which motivates the adoption of common standards for supporting the collaborative development of interoperable automatic misinformation detection approaches.","manuscriptTitle":"Assessing News Credibility: Misinformation Content Indicators","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-03-04 22:08:10","doi":"10.21203/rs.3.rs-173067/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"eb93f532-6abd-4d0f-8aac-1e11c97829df","owner":[],"postedDate":"March 4th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":2777546,"name":"Computer Architecture and Engineering"}],"tags":[],"updatedAt":"2021-03-04T22:08:10+00:00","versionOfRecord":[],"versionCreatedAt":"2021-03-04 22:08:10","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-173067","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-173067","identity":"rs-173067","version":["v1"]},"buildId":"J0_U0BvcaRcwD8yVFaRlm","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}