Generative text-to-image diffusion for automated map production based on geosocial media data

Abstract The state of generative AI has taken a leap forward with the availability of open source diffusion models. Here, we demonstrate an integrated workflow that uses text-to-image Stable Diffusion at its core to automatically generate icon maps such as for the area of the Großer Garten, a tourist hotspot in Dresden, Germany. The provided workflow is based on the aggregation of geosocial media data from Twitter, Flickr, Instagram, and iNaturalist. This data is used to create diffusion prompts, to account for the collective attribution of meaning and importance by the population in map generation. Specifically, we contribute methods for simplifying the variety of contexts communicated on social media, through spatial clustering and semantic filtering, for use in prompts, and then demonstrate how this human-contributed baseline data can be used in prompt engineering to automatically generate icon maps. Replacing labels on maps with expressive graphics has the general advantage of reaching a broader audience, such as children and other illiterate groups. For example, the resulting maps can be used to inform tourists of all backgrounds about important activities, points of interest, and landmarks without the need for translation. Several challenges are identified and possible future optimizations are described for different steps of the process. The code and data are fully provided and shared in several Jupyter notebooks, allowing for transparent replication of the workflow and adoption to other areas or datasets.