Beyond Raw Performance: Neural Machine Translation Based Multi-lingual Classification of Tweets for Automated Disease Surveillance

Abstract Twitter and social media as a whole have great potential as a source of disease surveillance data however the general messiness of tweets presents several challenges for standard information extraction methods. Most deployed systems employ approaches that rely on simple keyword matching and do not distinguish between relevant and irrelevant keyword mentions making them susceptible to false positives as a result of the fact that keyword volume can be influenced by several social phenomena that may be unrelated to disease occurrence. Furthermore, most solutions are intended for a single language and those meant for multilingual scenarios do not incorporate semantic context. In this paper we experimentally examine a translation based approach that allows for incorporation of semantic context in multi-lingual disease surveillance in the social web.