Abstract

Consider a channel W along with a given input distribution PX. In certain settings, such as in the construction of polar codes, the output alphabet of W is 'too large', and hence we replace W by a channel Q having a smaller output alphabet. We say that Q is upgraded with respect to W if W is obtained from Q by processing its output. In this case, the mutual information I(PX,W) between the input and output of W is upper-bounded by the mutual information I(PX,Q) between the input and output of Q. In this paper, we present an algorithm that produces an upgraded channel Q from W, as a function of PX and the required output alphabet size of Q, denoted L. We show that the difference in mutual informations is 'small'. Namely, it is O(L-2/(|X|-1), where |X| is the size of the input alphabet. This power law of L is optimal. We complement our analysis with numerical experiments which show that the developed algorithm improves upon the existing state-of-the-art algorithms also in non-asymptotic setups.

Original languageEnglish
Pages (from-to)7822-7836
Number of pages15
JournalIEEE Transactions on Information Theory
Volume67
Issue number12
DOIs
StatePublished - 1 Dec 2021

Keywords

  • Approximation algorithms
  • Channel upgradation
  • Costs
  • Markov processes
  • Mutual information
  • Probability distribution
  • Random variables
  • channel degradation
  • polar codes
  • quantization

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'An Upgrading Algorithm with Optimal Power Law'. Together they form a unique fingerprint.

Cite this