Transferring population group knowledge from multimodal large language model to small model: using urban safety perception evaluation as case study

Transferring population group knowledge from multimodal large language model to small model: using urban safety perception evaluation as case study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Transferring population group knowledge from multimodal large language model to small model: using urban safety perception evaluation as case study Ce Hou, Fan Zhang, Yuhao Kang, Zhuangyuan Fan, Sen Li This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6351052/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Human-centric urban planning curates environments that address behaviors, perceptions, and feelings. Conventional methods of understanding people's perceptions and feelings depend on questionnaires and surveys that are time-consuming and labor-intensive. Recent studies developed deep-learning-based computer vision tools to decipher the safety perception. Yet, this method only provides a generalized approximation of human perception and fails to incorporate personalized and local assessments.Large language models (LLM), trained with immersive human knowledge, have demonstrated the potential to simulate human understanding of images, space, and place.In view of these challenges and motivations, our study proposes an LLM-based framework that zero-shot adapts general safety perception evaluation models to specific demographics and perception scenarios. The framework first generates (1) general and (2) demographic and scenario-specific textual descriptions for street view images (SVIs) in specific areas. Then SVIs and general descriptions are used to train a safety perception model, and specific descriptions are used to fine-tune it via a pseudo-labeling strategy.To validate our method's effectiveness, we conduct experiments using individual-level safety perception data from Stockholm. The results show that general model's accuracy decreased by 19.7–25 percent when evaluating safety perception with a specific population group, while the accuracy of fine-tuned model with our method improved by 14.9–24 percent. We further employ this framework to map safety perceptions of four predefined demographic groups (middle-aged, elderly, women, and men) in Hong Kong across two perception scenarios: traffic accidents and crime.Our framework provides governments with a new tool for large-scale automated evaluation of urban perceptions across different groups. Social science/Geography Scientific community and society/Social sciences/Interdisciplinary studies Social science/Science technology and society Large language model safety perception street view image urban visual intelligence Full Text Additional Declarations No competing interests reported. Supplementary Files supplementarymaterials.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6351052","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":453343719,"identity":"d22f7452-d6bc-42bb-9e7f-46f526506008","order_by":0,"name":"Ce Hou","email":"","orcid":"","institution":"Peking University","correspondingAuthor":false,"prefix":"","firstName":"Ce","middleName":"","lastName":"Hou","suffix":""},{"id":453343720,"identity":"c11cb206-9717-4839-a00a-6fd5b06fe051","order_by":1,"name":"Fan Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+UlEQVRIiWNgGAWjYBAC9gYGBmYwi5mB8QHDARArAb8WngMILcwGJGphYGCTIE4L+9nDnwtq7tjNd2d/Vl1w5jADP3uOAcPPHXi08OQlGM849ix542Ees9szbhxmkOx5Y8DYewa3FnuGHINkHrbDyYbNPGy3eT4cZjC4kWPAzNiGxxb+NwaHef6BtLA/KwZpsSeoRSLHsJm37bCdPDODGTMP0GEGEgS1vDFm5u07nGDAzGMsPeNMOo/EmWcFB3vxOizH+DPPt8P28v3HH34uOGYtx9+evPHBTzxaYCBxwwFIBPGAeAcIawCGnHwDIk5HwSgYBaNgFKAAAFftT7s+E2a5AAAAAElFTkSuQmCC","orcid":"","institution":"Peking University","correspondingAuthor":true,"prefix":"","firstName":"Fan","middleName":"","lastName":"Zhang","suffix":""},{"id":453343721,"identity":"793899e8-0358-4838-801f-7bfbc2b190f2","order_by":2,"name":"Yuhao Kang","email":"","orcid":"","institution":"The University of Texas at Austin","correspondingAuthor":false,"prefix":"","firstName":"Yuhao","middleName":"","lastName":"Kang","suffix":""},{"id":453343722,"identity":"26fc5c0e-3a7c-48ec-b011-5f2145f0733a","order_by":3,"name":"Zhuangyuan Fan","email":"","orcid":"","institution":"University of Hong Kong","correspondingAuthor":false,"prefix":"","firstName":"Zhuangyuan","middleName":"","lastName":"Fan","suffix":""},{"id":453343723,"identity":"60e353c4-3197-4a8c-bbaa-4bd6a2b9580e","order_by":4,"name":"Sen Li","email":"","orcid":"","institution":"Hong Kong University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Sen","middleName":"","lastName":"Li","suffix":""}],"badges":[],"createdAt":"2025-04-01 08:23:23","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6351052/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6351052/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92475733,"identity":"de3fe369-70be-40a0-907b-7880722bd1bb","added_by":"auto","created_at":"2025-09-30 07:17:50","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6463052,"visible":true,"origin":"","legend":"","description":"","filename":"manuscriptsubmission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6351052/v1_covered_83e4b99f-e9f1-4943-9df8-1ff1a87aace5.pdf"},{"id":82229870,"identity":"7f5e1639-73dc-4545-a6cc-7f7dd9f10c2a","added_by":"auto","created_at":"2025-05-08 05:40:17","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":137236,"visible":true,"origin":"","legend":"","description":"","filename":"supplementarymaterials.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6351052/v1/2a876d5c4933bd8461b627ea.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Transferring population group knowledge from multimodal large language model to small model: using urban safety perception evaluation as case study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Large language model, safety perception, street view image, urban visual intelligence","lastPublishedDoi":"10.21203/rs.3.rs-6351052/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6351052/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Human-centric urban planning curates environments that address behaviors, perceptions, and feelings. Conventional methods of understanding people's perceptions and feelings depend on questionnaires and surveys that are time-consuming and labor-intensive. Recent studies developed deep-learning-based computer vision tools to decipher the safety perception. Yet, this method only provides a generalized approximation of human perception and fails to incorporate personalized and local assessments.Large language models (LLM), trained with immersive human knowledge, have demonstrated the potential to simulate human understanding of images, space, and place.In view of these challenges and motivations, our study proposes an LLM-based framework that zero-shot adapts general safety perception evaluation models to specific demographics and perception scenarios. The framework first generates (1) general and (2) demographic and scenario-specific textual descriptions for street view images (SVIs) in specific areas. Then SVIs and general descriptions are used to train a safety perception model, and specific descriptions are used to fine-tune it via a pseudo-labeling strategy.To validate our method's effectiveness, we conduct experiments using individual-level safety perception data from Stockholm. The results show that general model's accuracy decreased by 19.7–25 percent when evaluating safety perception with a specific population group, while the accuracy of fine-tuned model with our method improved by 14.9–24 percent. We further employ this framework to map safety perceptions of four predefined demographic groups (middle-aged, elderly, women, and men) in Hong Kong across two perception scenarios: traffic accidents and crime.Our framework provides governments with a new tool for large-scale automated evaluation of urban perceptions across different groups.","manuscriptTitle":"Transferring population group knowledge from multimodal large language model to small model: using urban safety perception evaluation as case study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-05-08 05:40:13","doi":"10.21203/rs.3.rs-6351052/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3d1487ca-9b1c-4e2d-88f0-c33358d3918a","owner":[],"postedDate":"May 8th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":48211218,"name":"Social science/Geography"},{"id":48211219,"name":"Scientific community and society/Social sciences/Interdisciplinary studies"},{"id":48211220,"name":"Social science/Science technology and society"}],"tags":[],"updatedAt":"2025-09-30T07:09:21+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-08 05:40:13","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6351052","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6351052","identity":"rs-6351052","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}