Extracting Semantic Representations from Word Co-occurrence Statistics: Polysemes and Homonyms

Vectors of word co-occurrence statistics from large corpora of natural language have previously been shown to provide good representations of lexical semantics and are now widely used resources in numerous language applications and models of human behaviour. This paper explores the open question of how the blending of multiple word meanings associated with polysemy and homonymy affect the quality of those corpus-derived semantic representations. Given the obvious difficulties involved in extracting the necessary information for real polysemes and homonyms, a general method is developed for evaluating the potential performance degradation by creating artificial polysemes and homonyms that can be expected to behave in the same way as their real counterparts. It is shown that words with multiple semantically similar senses (i.e., many polysemes) do not exhibit any problems – in fact they can have improved quality semantic vectors. However, it is confirmed that words with multiple semantically dissimilar senses (i.e., homonyms and some polysemes) can sometimes result in poorer quality semantic vectors that involve interesting interactions with the frequencies associated with the multiple senses, though often the quality degradation is less than might be anticipated. A series of computational experiments is designed and results presented to provide clear explanations of the various issues affecting these outcomes, and to provide a thorough systematic evaluation of potential methods for improving the quality of poorly performing homonym vectors. The general pattern of artificial homonym outcomes and the optimal method discovered for improving vector performance are then tested successfully on real homonyms.