Assessing the Efficacy of Pre-trained Large Language Model for Intersection Crash Severity Classification

Assessing the Efficacy of Pre-trained Large Language Model for Intersection Crash Severity Classification | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Assessing the Efficacy of Pre-trained Large Language Model for Intersection Crash Severity Classification Swaranjit Roy, Sherif M. Gaweesh, Ph.D., P.E., RSP This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7894857/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Traffic safety analysis plays a critical role in preventing road crashes by identifying and addressing risk factors before accidents occur. Crash severity prediction is a critical aspect of traffic safety research, yet traditional machine learning (ML) models often demand labor-intensive preprocessing and produce results that are often complex to interpret for non-technical stakeholders. This study investigates the potential of large language models (LLMs), specifically GPT-4o, for crash severity classification using a few-shot prompting approach that bypasses resource-intensive fine-tuning. Structured crash data from the North Dakota Department of Transportation (2013–2023) were transformed into narrative-style prompts through a tabular-to-text framework, enabling GPT-4o to process real-world crash reports in natural language. The model’s performance was benchmarked against five classical ML algorithms Logistic Regression, Random Forest, Decision Tree, Naive Bayes, and CatBoost using accuracy, precision, recall, and F1 score. Results show that GPT-4o achieved the highest performance across all metrics (accuracy: 0.50, precision: 0.68, recall: 0.50, F1 score: 0.50), outperforming traditional ML baselines under minimal preprocessing. Compared to a prior study using a fine-tuned LLaMA-2 70B model, GPT-4o demonstrated a relative improvement of 11.1% in accuracy and F1, 4.2% in recall, and a substantial 51.1% in precision. These findings highlight the practicality of LLMs for crash severity analysis in low-resource settings, offering scalable, interpretable, and plug-and-play alternatives to conventional ML methods. With simplified deployment and interpretation, this approach provides transportation agencies a practical pathway to adopt AI for safety analysis, as the use of natural language processing lowers technical barriers significantly. Civil Engineering Artificial Intelligence and Machine Learning Large Language Models (LLMs) Crash Severity GPT-4o Few-Shot Learning Traffic Safety Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7894857","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":531684119,"identity":"a4267156-4077-4721-89a7-c8c99f1812c0","order_by":0,"name":"Swaranjit Roy","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5ElEQVRIiWNgGAWjYPACNh4IXQHEBxgYJIjSAtFzhngtDAxgLYxtRGiRn5H87MPPHXwy9gy8Dx/zzjucx3eA+eBtHjxaDG6kGc/sPQNyGLuxMe+2w8WSB9iSrfFqkU4wZuBtA2lhY5MGaknccIDHTBqfFvnZ6Z8Z/8K1zAFp4f+GVwvD7RxjZoQtDWBb2PBqMbj/pphZFqTlMBuz4Zxj6YkzD7MZW87B57Ce45sZ37Yds2dvb2N88KbGOrHvePPDG2/wOQwCjjEwMDMwMIHdw0xYOQjUgEnGH8SpHgWjYBSMghEGAJ/7QvGEx9UnAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0009-0003-4003-5451","institution":"University of North Dakota","correspondingAuthor":true,"prefix":"","firstName":"Swaranjit","middleName":"","lastName":"Roy","suffix":""},{"id":531684120,"identity":"e7e4cd0f-d222-4353-8f8c-f188f49e196a","order_by":1,"name":"Sherif M. Gaweesh, Ph.D., P.E., RSP","email":"","orcid":"https://orcid.org/0000-0001-7977-6378","institution":"University of North Dakota","correspondingAuthor":false,"prefix":"","firstName":"P.","middleName":"Ph.D. Sherif M.","lastName":"Gaweesh","suffix":"Ph.D."}],"badges":[],"createdAt":"2025-10-18 17:08:30","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7894857/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7894857/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93988218,"identity":"bb13034a-5201-4852-9cba-01226e0d2ac2","added_by":"auto","created_at":"2025-10-21 04:50:29","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":563301,"visible":true,"origin":"","legend":"","description":"","filename":"LLMPaperFinal.docx","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/bda5226dabe808ee2676af36.docx"},{"id":93988059,"identity":"144b08d7-b965-4da3-b9db-29c02858dfd2","added_by":"auto","created_at":"2025-10-21 04:42:29","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":342,"visible":true,"origin":"","legend":"","description":"","filename":"rs7894857.json","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/18204e28899d09c8173ff27d.json"},{"id":93988913,"identity":"ccbc5d3d-62d5-4ac2-8159-574adde5866b","added_by":"auto","created_at":"2025-10-21 04:58:29","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":116032,"visible":true,"origin":"","legend":"","description":"","filename":"rs78948570enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/b39e652b43ae7f2571b271c2.xml"},{"id":93988062,"identity":"7ffa6c92-275a-4fae-b5cc-4ba748545f14","added_by":"auto","created_at":"2025-10-21 04:42:29","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":86083,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/989db3e2727bd670d326850c.png"},{"id":93988060,"identity":"2eedb235-2b7e-48b0-a359-48de9da88b67","added_by":"auto","created_at":"2025-10-21 04:42:29","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":189638,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/589652ef04c81457e12a8fce.png"},{"id":93988064,"identity":"0940f13f-4c03-46f1-8d05-fb41be158a39","added_by":"auto","created_at":"2025-10-21 04:42:29","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":38758,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/d1a8b501d9f3447c29c277c8.png"},{"id":93988058,"identity":"67ca0854-b873-4e4e-bdee-5b58a2de6403","added_by":"auto","created_at":"2025-10-21 04:42:29","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":51736,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/33dec35d70a513e6f764c1c9.png"},{"id":93988065,"identity":"4b808d6a-e1ca-40f0-81d7-c44fc7a3b575","added_by":"auto","created_at":"2025-10-21 04:42:30","extension":"xml","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":114684,"visible":true,"origin":"","legend":"","description":"","filename":"rs78948570structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/a44e84a45a8cadd5b1807d59.xml"},{"id":93988220,"identity":"368d1ff7-b70a-465f-bc00-cab6c008c34e","added_by":"auto","created_at":"2025-10-21 04:50:30","extension":"html","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":125464,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1/2b0b442821c09ac167e37b6d.html"},{"id":93989176,"identity":"ae5ddce1-a35d-4a1e-ad0b-c2d4ca49e3ee","added_by":"auto","created_at":"2025-10-21 05:06:30","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":566653,"visible":true,"origin":"","legend":"","description":"","filename":"LLMPaperFinal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7894857/v1_covered_94de400a-17b9-4732-b67b-ac32961ff4af.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eAssessing the Efficacy of Pre-trained Large Language Model for Intersection Crash Severity Classification\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of North Dakota","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Large Language Models (LLMs), Crash Severity, GPT-4o, Few-Shot Learning, Traffic Safety","lastPublishedDoi":"10.21203/rs.3.rs-7894857/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7894857/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTraffic safety analysis plays a critical role in preventing road crashes by identifying and addressing risk factors before accidents occur. Crash severity prediction is a critical aspect of traffic safety research, yet traditional machine learning (ML) models often demand labor-intensive preprocessing and produce results that are often complex to interpret for non-technical stakeholders. This study investigates the potential of large language models (LLMs), specifically GPT-4o, for crash severity classification using a few-shot prompting approach that bypasses resource-intensive fine-tuning. Structured crash data from the North Dakota Department of Transportation (2013\u0026ndash;2023) were transformed into narrative-style prompts through a tabular-to-text framework, enabling GPT-4o to process real-world crash reports in natural language. The model\u0026rsquo;s performance was benchmarked against five classical ML algorithms Logistic Regression, Random Forest, Decision Tree, Naive Bayes, and CatBoost using accuracy, precision, recall, and F1 score. Results show that GPT-4o achieved the highest performance across all metrics (accuracy: 0.50, precision: 0.68, recall: 0.50, F1 score: 0.50), outperforming traditional ML baselines under minimal preprocessing. Compared to a prior study using a fine-tuned LLaMA-2 70B model, GPT-4o demonstrated a relative improvement of 11.1% in accuracy and F1, 4.2% in recall, and a substantial 51.1% in precision. These findings highlight the practicality of LLMs for crash severity analysis in low-resource settings, offering scalable, interpretable, and plug-and-play alternatives to conventional ML methods. With simplified deployment and interpretation, this approach provides transportation agencies a practical pathway to adopt AI for safety analysis, as the use of natural language processing lowers technical barriers significantly.\u003c/p\u003e","manuscriptTitle":"Assessing the Efficacy of Pre-trained Large Language Model for Intersection Crash Severity Classification","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-21 04:42:25","doi":"10.21203/rs.3.rs-7894857/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"11fd5e32-e973-46b8-9f93-55bcbcf94b83","owner":[],"postedDate":"October 21st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":56526467,"name":"Civil Engineering"},{"id":56526468,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-10-21T04:42:25+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-21 04:42:25","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7894857","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7894857","identity":"rs-7894857","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}