A hitchhiker’s guide to information theoretical measures in psychology

In psychology, as in other sciences, information theory can be used as a tool tocomplement more standard regression-based methods of data analysis. It is important tosee the potential of information theoretical measures as statistical tools without implying aconnection to their origins in communication theory and engineering. The use of thesemeasures may provide us with additional insights due to their sensitivity to non-linearrelationships, their flexibility to the mixing of data types, and their more straightforwardgeneralization towards investigating higher-order interactions. We briefly reintroduceinformation theory and compare several measures such as mutual information andco-information with correlation and regression-based methods for the investigation ofvariable dependence.