Abstract
We introduce in this talk the Rényi entropy and the Arimoto-Rényi conditional entropy, together with asymptotically tight
bounds on the guessing moments by Arikan (1996) which are expressed as a function of these Rényi information measures. For a
discrete random variable X which takes a finite n of possible values, the problem of maximizing the Rényi entropy of a function
of X over all functions which are mappings from a set of cardinality n to a set of a smaller cardinality m (with fixed values of
m < n) is strongly NP-hard. We provide an upper bound on this maximal Rényi entropy with a guarantee on its largest possible
gap from the exact value, together with a simple algorithm to construct this function. This work was inspired by the recently
published paper by Cicalese et al. (IEEE Trans. on IT, 2018), which is focused on the Shannon entropy, and it strengthens and
generalizes the results of that paper to Rényi entropies of arbitrary positive orders by the use of majorization theory. We discuss
the implications of these results in the context of the guessing problem.
The new findings in this talk are based on the paper: I. Sason, “Tight bounds on the Renyi entropy via majorization with
applications to guessing and compression,” Entropy, vol. 20, no. 12, paper 896, pp. 1–25, November 2018.
bounds on the guessing moments by Arikan (1996) which are expressed as a function of these Rényi information measures. For a
discrete random variable X which takes a finite n of possible values, the problem of maximizing the Rényi entropy of a function
of X over all functions which are mappings from a set of cardinality n to a set of a smaller cardinality m (with fixed values of
m < n) is strongly NP-hard. We provide an upper bound on this maximal Rényi entropy with a guarantee on its largest possible
gap from the exact value, together with a simple algorithm to construct this function. This work was inspired by the recently
published paper by Cicalese et al. (IEEE Trans. on IT, 2018), which is focused on the Shannon entropy, and it strengthens and
generalizes the results of that paper to Rényi entropies of arbitrary positive orders by the use of majorization theory. We discuss
the implications of these results in the context of the guessing problem.
The new findings in this talk are based on the paper: I. Sason, “Tight bounds on the Renyi entropy via majorization with
applications to guessing and compression,” Entropy, vol. 20, no. 12, paper 896, pp. 1–25, November 2018.
| Original language | American English |
|---|---|
| State | Published - 2019 |
| Event | Workshop on Mathematical Data Science - , Austria Duration: 14 Oct 2019 → 15 Oct 2019 https://www.mins.ee.ethz.ch/mds2019/program.html |
Conference
| Conference | Workshop on Mathematical Data Science |
|---|---|
| Abbreviated title | MDS |
| Country/Territory | Austria |
| Period | 14/10/19 → 15/10/19 |
| Internet address |