Urban Dynamics through Social Media Lens: A Computational Approach to Identifying and Analyzing Civic Issues in Smart Cities

In this study, we conducted a comprehensive analysis of urban challenges in smart cities by categorizing issues such as Traffic, Pollution, Accidents, Garbage, and Potholes, through the lens of Twitter data. Recognizing the limitations of traditional physical sensor surveillance in monitoring extensive urban road networks, we leveraged the dynamic and real-time nature of social media data, specifically from Twitter, to extract insights relevant to urban management and planning. This analysis was particularly focused on Delhi, a key smart city in India.To effectively segregate and classify tweets into these distinct urban categories, we developed an innovative methodology using dense word embeddings facilitated by the word2vec model. This approach enabled us to capture and analyze the semantic context of tweets, leading to the generation of lists of keywords that are semantically related, enhancing the precision of our categorization process. Moreover, we conducted a detailed spatial analysis to pinpoint the most vulnerable locations within each identified category. Complementing this, our temporal analysis shed light on the peak periods of public discourse related to these urban challenges, allowing us to explore and understand the key characteristics of these peak activity times. Our findings, demonstrating the application of advanced natural language processing techniques in urban issue identification and analysis, were corroborated and validated through a comparison with reports issued by government bodies and media publications. This not only underlines the efficacy of our approach but also highlights its potential utility in urban governance and policy-making.