Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts

Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts Nikesh Gyawali, Doina Caragea, Alex Vasenkov, Cornelia Caragea This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7411930/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 12 You are reading this latest preprint version Abstract Financial narratives from U.S. Securities and Exchange Commission (SEC) filing reports and quarterly earnings call transcripts (ECTs) are very important for investors, auditors, and regulators. However, their length, financial jargon, and nuanced language make fine-grained analysis difficult. Prior sentiment analysis in the financial domain required a large, expensive labeled dataset, making the sentence-level stance towards specific financial targets challenging. In this work, we introduce a sentence-level corpus for stance detection focused on three core financial metrics: debt, earnings per share (EPS), and sales. The sentences were extracted from Form 10-K annual reports and ECTs, and labeled for stance (positive, negative, neutral) using the advanced ChatGPT-o3-pro model under rigorous human validation. Using this corpus, we conduct a systematic evaluation of modern large language models (LLMs) using zero-shot, few-shot, and Chain-of-Thought (CoT) prompting strategies. Our results show that few-shot with CoT prompting performs best compared to supervised baselines, and LLMs' performance varies across the SEC and ECT datasets. Our findings highlight the practical viability of leveraging LLMs for target-specific stance in the financial domain without requiring extensive labeled data. Physical sciences/Engineering Business and commerce/Information systems and information technology Physical sciences/Mathematics and computing Full Text Additional Declarations No competing interests reported. Supplementary Files supplementary.pdf Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 12 Dec, 2025 Reviews received at journal 10 Dec, 2025 Reviews received at journal 10 Dec, 2025 Reviews received at journal 05 Dec, 2025 Reviewers agreed at journal 04 Dec, 2025 Reviewers agreed at journal 03 Dec, 2025 Reviewers agreed at journal 03 Dec, 2025 Reviewers invited by journal 03 Dec, 2025 Editor invited by journal 01 Sep, 2025 Editor assigned by journal 22 Aug, 2025 Submission checks completed at journal 20 Aug, 2025 First submitted to journal 19 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7411930","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":555525472,"identity":"5811a215-10b2-43f5-8474-2632cc454bf6","order_by":0,"name":"Nikesh Gyawali","email":"","orcid":"","institution":"Kansas State University","correspondingAuthor":false,"prefix":"","firstName":"Nikesh","middleName":"","lastName":"Gyawali","suffix":""},{"id":555525473,"identity":"60e4c547-01d5-4c47-8600-95ad49b02298","order_by":1,"name":"Doina Caragea","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAwklEQVRIiWNgGAWjYDCCA1CanwdMMZOgRbKHZC0GZ4jVwnf88LMPP37ZyBufOf5MgqHCOrGBkBbJM2nGM3v70gy3ne0xk2A4k05Yi8ENBmMG3p7DjNvO87BJMLYdJkYL+2fGvz3/7Tf3sz+TYPxHlBYeY2aeHwcSN/A2mEkwNhChRfJMTjGzbENy8owzZ4wtEo6lGxPUwnf8+GbGN3/sbPt70h/e+FBjLUtQCxgwtkEZCUQpB4M/xCsdBaNgFIyCEQgAGXtBsj4DbCYAAAAASUVORK5CYII=","orcid":"","institution":"Kansas State University","correspondingAuthor":true,"prefix":"","firstName":"Doina","middleName":"","lastName":"Caragea","suffix":""},{"id":555525474,"identity":"aa54a585-d83e-40f8-8ad4-38b893cae622","order_by":2,"name":"Alex Vasenkov","email":"","orcid":"","institution":"Mathinvestments, Inc.","correspondingAuthor":false,"prefix":"","firstName":"Alex","middleName":"","lastName":"Vasenkov","suffix":""},{"id":555525475,"identity":"c0535aa5-1773-4f88-a84a-7a6ffed7e860","order_by":3,"name":"Cornelia Caragea","email":"","orcid":"","institution":"University of Illinois at Chicago","correspondingAuthor":false,"prefix":"","firstName":"Cornelia","middleName":"","lastName":"Caragea","suffix":""}],"badges":[],"createdAt":"2025-08-19 22:23:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7411930/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7411930/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":97500526,"identity":"2d749d54-4cc6-46dc-95ea-6175b2f4d581","added_by":"auto","created_at":"2025-12-05 05:56:53","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":6005,"visible":true,"origin":"","legend":"","description":"","filename":"c0e16e9d51a8419abc1ce86a53103498.json","url":"https://assets-eu.researchsquare.com/files/rs-7411930/v1/71c9531db08fa8323b94bf0e.json"},{"id":97669993,"identity":"48c4978b-b40a-4cfe-a88a-7fd4d6f162bc","added_by":"auto","created_at":"2025-12-08 09:29:32","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2120199,"visible":true,"origin":"","legend":"","description":"","filename":"ScientificReports.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7411930/v1_covered_0054b0fc-9dd7-4045-9f3e-ec06c4efa2a7.pdf"},{"id":97500527,"identity":"1e04f93f-d398-4372-98fd-67e8138ad967","added_by":"auto","created_at":"2025-12-05 05:56:53","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":138250,"visible":true,"origin":"","legend":"","description":"","filename":"supplementary.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7411930/v1/5bc94616e6c43ba62aa41443.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7411930/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7411930/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Financial narratives from U.S. Securities and Exchange Commission (SEC) filing reports and quarterly earnings call transcripts (ECTs) are very important for investors, auditors, and regulators. However, their length, financial jargon, and nuanced language make fine-grained analysis difficult. Prior sentiment analysis in the financial domain required a large, expensive labeled dataset, making the sentence-level stance towards specific financial targets challenging. In this work, we introduce a sentence-level corpus for stance detection focused on three core financial metrics: debt, earnings per share (EPS), and sales. The sentences were extracted from Form 10-K annual reports and ECTs, and labeled for stance (positive, negative, neutral) using the advanced ChatGPT-o3-pro model under rigorous human validation. Using this corpus, we conduct a systematic evaluation of modern large language models (LLMs) using zero-shot, few-shot, and Chain-of-Thought (CoT) prompting strategies. Our results show that few-shot with CoT prompting performs best compared to supervised baselines, and LLMs' performance varies across the SEC and ECT datasets. Our findings highlight the practical viability of leveraging LLMs for target-specific stance in the financial domain without requiring extensive labeled data. ","manuscriptTitle":"Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-05 05:56:48","doi":"10.21203/rs.3.rs-7411930/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-12-12T07:28:47+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-10T17:19:43+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-10T07:47:50+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-05T07:15:05+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"324623559809121581252884732434453843079","date":"2025-12-05T01:21:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"286638123908871490741232677033421080601","date":"2025-12-03T16:13:30+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"22686083706709957948505619721319360106","date":"2025-12-03T13:24:48+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-12-03T12:09:13+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-09-01T14:08:10+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-22T07:31:36+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-20T07:09:10+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-08-19T22:18:37+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"db548786-7103-4957-894a-ae791a9a3597","owner":[],"postedDate":"December 5th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[{"id":59125799,"name":"Physical sciences/Engineering"},{"id":59125800,"name":"Business and commerce/Information systems and information technology"},{"id":59125801,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2025-12-12T07:38:49+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-05 05:56:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7411930","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7411930","identity":"rs-7411930","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}