Estimating differential entropy using recursive copula splitting

Gil Ariel, Yoram Louzoun

Research output: Contribution to journalArticlepeer-review

Abstract

A method for estimating the Shannon differential entropy of multidimensional random variables using independent samples is described. The method is based on decomposing the distribution into a product of marginal distributions and joint dependency, also known as the copula. The entropy of marginals is estimated using one-dimensional methods. The entropy of the copula, which always has a compact support, is estimated recursively by splitting the data along statistically dependent dimensions. The method can be applied both for distributions with compact and non-compact supports, which is imperative when the support is not known or of a mixed type (in different dimensions). At high dimensions (larger than 20), numerical examples demonstrate that our method is not only more accurate, but also significantly more efficient than existing approaches.

Original languageEnglish
Article number236
JournalEntropy
Volume22
Issue number2
DOIs
StatePublished - 1 Feb 2020

Keywords

  • Copulas
  • Entropy estimation
  • Multivariate continuous distributions

All Science Journal Classification (ASJC) codes

  • General Physics and Astronomy

Fingerprint

Dive into the research topics of 'Estimating differential entropy using recursive copula splitting'. Together they form a unique fingerprint.

Cite this