When AI Tells the Truth? Evaluating Different LLM Approaches to Reliable Trip Planning

When AI Tells the Truth? Evaluating Different LLM Approaches to Reliable Trip Planning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article When AI Tells the Truth? Evaluating Different LLM Approaches to Reliable Trip Planning Vitalii Morskyi, Dawid Jaworski, Paweł Kuraś, Patryk Organiściak This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7389856/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This paper presents a hybrid architecture for the automatic generation of multi-day, personalized travel itineraries that balance strict logistical constraints with individual traveler preferences. The system combines Large Language Models (LLMs) with retrieval-augmented generation, web search, and geospatial APIs in an iterative plan-execute-reflect loop. Three agent configurations - LLM-only, LLM + Search, and geospatially-aware LLM + Maps - were evaluated across 371 scenarios differing in location, trip length, and transport mode. Five quantitative metrics captured temporal realism, preference alignment, hallucination rate, and computational efficiency. Results show that grounding LLMs in verified, real-time data sources - especially via Google Maps - virtually eliminates hallucinations and unrealistic timing, producing feasible itineraries. The best overall performance was achieved by a geospatially-grounded agent using Claude 3.5 Sonnet v2, highlighting the role of LLMs as high-level semantic orchestrators rather than autonomous planners. Large Language Models Grounded AI Travel Itinerary Generation Geospatial APIs Retrieval-Augmented Generation Hallucination Mitigation Hybrid AI Systems Trip Personalization Route Optimization Recommender Systems Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7389856","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":524593032,"identity":"8737e59c-6715-4d19-8c0b-43b049284ad8","order_by":0,"name":"Vitalii Morskyi","email":"","orcid":"","institution":"Rzeszów University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Vitalii","middleName":"","lastName":"Morskyi","suffix":""},{"id":524593033,"identity":"fc993ece-9060-4399-a816-7038ed6afcc7","order_by":1,"name":"Dawid Jaworski","email":"","orcid":"","institution":"Rzeszów University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Dawid","middleName":"","lastName":"Jaworski","suffix":""},{"id":524593034,"identity":"64279db3-167a-419b-8072-db0bea92fc83","order_by":2,"name":"Paweł Kuraś","email":"","orcid":"","institution":"AGH University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Paweł","middleName":"","lastName":"Kuraś","suffix":""},{"id":524593035,"identity":"2aef4d25-2c3c-4467-9e73-5f551a1ae98d","order_by":3,"name":"Patryk Organiściak","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAsElEQVRIiWNgGAWjYHACxgcQOgGIC4jTwmyA0GJAnBY2CdK08Esfv1ZduKeOgZ89x4C5gBgtkn05ZbdnPGNjkOx5Y8A8gxgtBmd40m7zHOBhMLgBtIWHWC3FPAckGOxJ0MJ+jJnngAGDgQSxWiR7eJilZxxI4JE486zgMFF+4edhf/i54ECdHH978sbHBRVEaGFg4DFgBpEg5mGiNDAwsD9ghjGZ8SgbBaNgFIyCEQwAFBEsCSwGuucAAAAASUVORK5CYII=","orcid":"","institution":"Rzeszów University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Patryk","middleName":"","lastName":"Organiściak","suffix":""}],"badges":[],"createdAt":"2025-08-17 01:23:04","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7389856/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7389856/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92826676,"identity":"800fbf24-392c-4c9f-a83c-9ae5f6f30ebf","added_by":"auto","created_at":"2025-10-06 04:35:12","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5348,"visible":true,"origin":"","legend":"","description":"","filename":"18bda8acc4fe46069a5428c7047a8d09.json","url":"https://assets-eu.researchsquare.com/files/rs-7389856/v1/353a5727ef359d91bf8d21d5.json"},{"id":96246258,"identity":"bb0d8fc5-4f49-4029-9856-8c32f2b82e7c","added_by":"auto","created_at":"2025-11-19 07:25:13","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1042652,"visible":true,"origin":"","legend":"","description":"","filename":"WhenAITellstheTruth1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7389856/v1_covered_001ca9ec-0b98-4c63-a2f9-b9281874d5d4.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"When AI Tells the Truth? Evaluating Different LLM Approaches to Reliable Trip Planning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Large Language Models, Grounded AI, Travel Itinerary Generation, Geospatial APIs, Retrieval-Augmented Generation, Hallucination Mitigation, Hybrid AI Systems, Trip Personalization, Route Optimization, Recommender Systems","lastPublishedDoi":"10.21203/rs.3.rs-7389856/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7389856/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"This paper presents a hybrid architecture for the automatic generation of multi-day, personalized travel itineraries that balance strict logistical constraints with individual traveler preferences. The system combines Large Language Models (LLMs) with retrieval-augmented generation, web search, and geospatial APIs in an iterative plan-execute-reflect loop. Three agent configurations - LLM-only, LLM + Search, and geospatially-aware LLM + Maps - were evaluated across 371 scenarios differing in location, trip length, and transport mode. Five quantitative metrics captured temporal realism, preference alignment, hallucination rate, and computational efficiency. Results show that grounding LLMs in verified, real-time data sources - especially via Google Maps - virtually eliminates hallucinations and unrealistic timing, producing feasible itineraries. The best overall performance was achieved by a geospatially-grounded agent using Claude~3.5 Sonnet~v2, highlighting the role of LLMs as high-level semantic orchestrators rather than autonomous planners.","manuscriptTitle":"When AI Tells the Truth? Evaluating Different LLM Approaches to Reliable Trip Planning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-06 04:35:07","doi":"10.21203/rs.3.rs-7389856/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ce16f11c-20e9-4ac0-ac6c-01fb50b4ce17","owner":[],"postedDate":"October 6th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-11-17T07:54:29+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-06 04:35:07","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7389856","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7389856","identity":"rs-7389856","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}