Rényi entropy and guessing: Old and new results

Research output: Contribution to conferencePaperpeer-review


We introduce in this talk the Rényi entropy and the Arimoto-Rényi conditional entropy, together with asymptotically tight
bounds on the guessing moments by Arikan (1996) which are expressed as a function of these Rényi information measures. For a
discrete random variable X which takes a finite n of possible values, the problem of maximizing the Rényi entropy of a function
of X over all functions which are mappings from a set of cardinality n to a set of a smaller cardinality m (with fixed values of
m < n) is strongly NP-hard. We provide an upper bound on this maximal Rényi entropy with a guarantee on its largest possible
gap from the exact value, together with a simple algorithm to construct this function. This work was inspired by the recently
published paper by Cicalese et al. (IEEE Trans. on IT, 2018), which is focused on the Shannon entropy, and it strengthens and
generalizes the results of that paper to Rényi entropies of arbitrary positive orders by the use of majorization theory. We discuss
the implications of these results in the context of the guessing problem.
The new findings in this talk are based on the paper: I. Sason, “Tight bounds on the Renyi entropy via majorization with
applications to guessing and compression,” Entropy, vol. 20, no. 12, paper 896, pp. 1–25, November 2018.
Original languageEnglish
StatePublished - 2019
EventWorkshop on Mathematical Data Science - , Austria
Duration: 14 Oct 201915 Oct 2019


ConferenceWorkshop on Mathematical Data Science
Abbreviated titleMDS
Internet address


Dive into the research topics of 'Rényi entropy and guessing: Old and new results'. Together they form a unique fingerprint.

Cite this