Entropy bounds for discrete random variables via coupling

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This work provides new bounds on the difference between the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal couplings, and the bounds apply to discrete random variables which are defined over finite or countably infinite alphabets. Loosened versions of these bounds are demonstrated to reproduce some previously reported results. The use of the new entropy bounds is exemplified for the Poisson approximation, where bounds on the local and total variation distances follow from Stein's method. The full paper version for this work is available at http://arxiv.org/abs/1209.5259.

Original languageEnglish
Title of host publication2013 IEEE International Symposium on Information Theory, ISIT 2013
Pages414-418
Number of pages5
DOIs
StatePublished - 2013
Event2013 IEEE International Symposium on Information Theory, ISIT 2013 - Istanbul, Turkey
Duration: 7 Jul 201312 Jul 2013

Publication series

NameIEEE International Symposium on Information Theory - Proceedings

Conference

Conference2013 IEEE International Symposium on Information Theory, ISIT 2013
Country/TerritoryTurkey
CityIstanbul
Period7/07/1312/07/13

Keywords

  • Entropy
  • Poisson approximation
  • Stein's method
  • local distance
  • maximal coupling
  • total variation distance

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modelling and Simulation
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Entropy bounds for discrete random variables via coupling'. Together they form a unique fingerprint.

Cite this