Rényi Entropy Estimation via Affine Transformations of Generalized Means

Rényi entropy is a generalization of Shannon entropy that enables the analysis of different aspects of the informative structure of probability distributions. This family of entropy measures is particularly valuable in scientific and technological contexts requiring the classification, comparison, or characterization of probability distributions, discrete stochastic processes, or discretized dynamical systems. Despite their theoretical appeal, Rényi entropies are challenging to estimate from empirical data—especially in settings involving limited samples, high dimensionality, or highly irregular behaviors. To address these difficulties, we propose a general-purpose estimation method applicable across the entire Rényi family. The method is based on affine transformations that enhance generality, robustness, and accuracy. In particular, the estimation of collision entropy achieves optimality in a single step. To help visualize and understand the effect of the transformation, we also introduce a geometric framework that represents probability distributions from the point of view of their entropic state. This provides intuitive insight into how our estimator works and why it is effective in practice.