On the entropy of sums of Bernoulli random variables via the Chen-Stein method

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper considers the entropy of the sum of (possibly dependent and non-identically distributed) Bernoulli random variables. Upper bounds on the error that follows from an approximation of this entropy by the entropy of a Poisson random variable with the same mean are derived. The derivation of these bounds combines elements of information theory with the Chen-Stein method for Poisson approximation. The resulting bounds are easy to compute, and their applicability is exemplified. This conference paper presents in part the first half of the paper entitled 'An information-theoretic perspective of the Poisson approximation via the Chen-Stein method' (see: http://arxiv.org/abs/1206.6811). A generalization of the bounds that considers the accuracy of the Poisson approximation for the entropy of a sum of non-negative, integer-valued and bounded random variables is introduced in the full paper. It also derives lower bounds on the total variation distance, relative entropy and other measures that are not considered in this conference paper.

Original languageEnglish
Title of host publication2012 IEEE Information Theory Workshop, ITW 2012
Pages542-546
Number of pages5
DOIs
StatePublished - 2012
Event2012 IEEE Information Theory Workshop, ITW 2012 - Lausanne, Switzerland
Duration: 3 Sep 20127 Sep 2012

Publication series

Name2012 IEEE Information Theory Workshop, ITW 2012

Conference

Conference2012 IEEE Information Theory Workshop, ITW 2012
Country/TerritorySwitzerland
CityLausanne
Period3/09/127/09/12

Keywords

  • Chen-Stein method
  • Poisson approximation
  • entropy
  • information theory
  • total variation distance

All Science Journal Classification (ASJC) codes

  • Information Systems

Fingerprint

Dive into the research topics of 'On the entropy of sums of Bernoulli random variables via the Chen-Stein method'. Together they form a unique fingerprint.

Cite this