Rate-distortion function via minimum mean square error estimation

Research output: Contribution to journalArticlepeer-review


We derive a simple general parametric representation of the rate-distortion function of a memoryless source, where both the rate and the distortion are given by integrals whose integrands include the minimum mean square error (MMSE) of the distortion Δ =d(X,Y) based on the source symbol X, with respect to a certain joint distribution of these two random variables. At first glance, these relations may seem somewhat similar to the IMMSE relations due to Guo, Shamai and Verd, but they are, in fact, quite different. The new relations among rate, distortion, and MMSE are discussed from several aspects, and more importantly, it is demonstrated that they can sometimes be rather useful for obtaining nontrivial upper and lower bounds on the rate-distortion function, as well as for determining the exact asymptotic behavior for very low and for very large distortion. Analogous MMSE relations hold for channel capacity as well.

Original languageEnglish
Article number5773042
Pages (from-to)3196-3206
Number of pages11
JournalIEEE Transactions on Information Theory
Issue number6
StatePublished - Jun 2011


  • Estimation
  • Legendre transform
  • minimum mean square error
  • rate-distortion function

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Rate-distortion function via minimum mean square error estimation'. Together they form a unique fingerprint.

Cite this