E2ETrADS: End-to-End Transformer Based Autonomous Driving System for Adverse Weather Conditions

E2ETrADS: End-to-End Transformer Based Autonomous Driving System for Adverse Weather Conditions | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article E2ETrADS: End-to-End Transformer Based Autonomous Driving System for Adverse Weather Conditions Sotirios Spanogianopoulos, Kenan Ahiska This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8768278/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Adverse weather conditions, such as snow,heavy rain, fog or limited illumination create significant challenges for autonomous vehicles (AVs) by diminishing the reliability of their sensors. This paper proposes a Transformer-based navigation strategy that could help autonomous vehicles navigate better in adverse weather conditions. The CARLA (Car Learning to Act) simulation tool makes a dataset by changing the environment to make it seem like visibility is lower and sensors don't work as well. The dataset that was created includes sensor data from multiple sources that is affected by changes in the weather. This includes LiDAR point clouds, depth maps, and pictures. We suggest using a deep learning model with transformer-based architectures to combine data from different sensors to help with decision-making. Using imitation learning, the model is trained using the control actions of the hybrid MPC-PID controller. While the weather-adaptive MPC optimizes control instructions using an environmental risk-aware cost structure, its PID component manages low-level actuation. According to the findings, sensor fusion can greatly improve the resilience of autonomous driving systems in inclement weather, especially when combined with transformer models. The suggested model was found to operate better in fog, rain, and low-visibility situations than the Transfuser baseline and to have far fewer violations. Autonomous driving Transformer CARLA Adverse Weather Imitation Learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8768278","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":593424639,"identity":"df1d6c16-93e7-4429-bbca-b336d35c4c32","order_by":0,"name":"Sotirios Spanogianopoulos","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABTUlEQVRIie2RMUvDQBTHLzi4pGSTKwHvK7xQaDtU8lUSAro4ZJJOJSLYpbVrQvwGLoFAwC0hoEtS14gdAoVOCnWRIB28pKmYcu6C9xvuHsf/l7x3hxCH80cJEXzXuN5NhKRy10yGcMBU6EnbKhVgKixoEsK62IOMZ1E8NEeIuFerfP3ZJ73DabQUYaB2nu70dQ6ISEdh42tJjKIEYgSLh57i3GDlfjI3OiKc6kH25mPamOK4WkPBBoos2gNgrSu3JljwsvOuLEKsdbPUKxUNXhoKsSuFNmaffZSKulPUjp36BUNBWaXQO6iSBdZ3iuBJ04D1FzoLUCUWYZFctB0LG14yN5RbOoudtYK+Bnh/FjK+XL5bm9ExcccBXm9GJ97jNMpfhwNVmqX+czEcEEluNlYjVo8oXG/rLbhKYla8DpTL5qcihb+nORwO5z/xBWv5fAmkUSuFAAAAAElFTkSuQmCC","orcid":"","institution":"University of Portsmouth","correspondingAuthor":true,"prefix":"","firstName":"Sotirios","middleName":"","lastName":"Spanogianopoulos","suffix":""},{"id":593424640,"identity":"30e08012-2c70-47d8-bf7e-c3d924681fc3","order_by":1,"name":"Kenan Ahiska","email":"","orcid":"","institution":"Middle East Technical University","correspondingAuthor":false,"prefix":"","firstName":"Kenan","middleName":"","lastName":"Ahiska","suffix":""}],"badges":[],"createdAt":"2026-02-02 18:54:44","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8768278/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8768278/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":103049668,"identity":"59ec1d50-5451-4ae1-b44a-80f9c43cab19","added_by":"auto","created_at":"2026-02-20 07:44:33","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1500829,"visible":true,"origin":"","legend":"","description":"","filename":"EndtoEndTransformerBasedAutonomousDrivingSystemforAdverseWeatherConditionsv.1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8768278/v1_covered_b0baaebe-daaf-44d7-adab-1244d5686131.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"E2ETrADS: End-to-End Transformer Based Autonomous Driving System for Adverse Weather Conditions","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Autonomous driving, Transformer, CARLA, Adverse Weather, Imitation Learning","lastPublishedDoi":"10.21203/rs.3.rs-8768278/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8768278/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e Adverse weather conditions, such as snow,heavy rain, fog or limited illumination create significant challenges for autonomous vehicles (AVs) by diminishing the reliability of their sensors. This paper proposes a Transformer-based navigation strategy that could help autonomous vehicles navigate better in adverse weather conditions. The CARLA (Car Learning to Act) simulation tool makes a dataset by changing the environment to make it seem like visibility is lower and sensors don't work as well. The dataset that was created includes sensor data from multiple sources that is affected by changes in the weather. This includes LiDAR point clouds, depth maps, and pictures. We suggest using a deep learning model with transformer-based architectures to combine data from different sensors to help with decision-making. Using imitation learning, the model is trained using the control actions of the hybrid MPC-PID controller. While the weather-adaptive MPC optimizes control instructions using an environmental risk-aware cost structure, its PID component manages low-level actuation. According to the findings, sensor fusion can greatly improve the resilience of autonomous driving systems in inclement weather, especially when combined with transformer models. The suggested model was found to operate better in fog, rain, and low-visibility situations than the Transfuser baseline and to have far fewer violations.\u003c/p\u003e","manuscriptTitle":"E2ETrADS: End-to-End Transformer Based Autonomous Driving System for Adverse Weather Conditions","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-19 18:37:02","doi":"10.21203/rs.3.rs-8768278/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"573330f6-7391-47da-a59a-1d3fb75fb8a2","owner":[],"postedDate":"February 19th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-04-21T10:11:22+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-19 18:37:02","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8768278","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8768278","identity":"rs-8768278","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}