TY - JOUR
T1 - Greedy-merge degrading has optimal power-law
AU - Kartowsky, Assaf
AU - Tal, Ido
N1 - Funding Information: Manuscript received April 23, 2017; revised March 31, 2018; accepted September 28, 2018. Date of publication November 6, 2018; date of current version January 18, 2019. This work was supported in part by the Israel Science Foundation under Grant 1769/13 and in part by the United States– Israel Binational Science Foundation under Grant 2012016. This paper was presented at ISIT 2017. A. Kartowsky is with Vayyar Imaging Ltd., Yehud 5621717, Israel (e-mail: [email protected]). I. Tal is with the Department of Electrical Engineering, Technion, Haifa 32000, Israel (e-mail: [email protected]). Communicated by M. Lentmaier, Associate Editor for Coding Theory. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIT.2018.2879802 Publisher Copyright: © 2018 IEEE.
PY - 2019/2
Y1 - 2019/2
N2 - Consider a channel with a given input alphabet size and input distribution. Our aim is to degrade or upgrade it to a channel with at most L output letters. Channel Q is degraded with respect to a channel W if Q can be obtained from W by processing the output of W. Upgrading is the inverse relation. This paper contains four main results. The first result, from which the paper title is derived, deals with the so called 'greedy-merge' algorithm. We derive an upper bound on the reduction in mutual information between input and output, as a function of L. This upper bound is within a constant factor of an algorithm-independent lower bound. Thus, we establish that greedy-merge is optimal in the power-law sense (i.e. the power of L ). The other main results deal with upgrading. The second result shows that a certain sequence of channels, which was previously shown to be 'hard' for degrading, displays the same hardness in the context of upgrading. That is, suppose we are given such a channel and a corresponding input distribution. If we upgrade (degrade) to a new channel with L output letters, we incur an increase (decrease) in mutual information between input and output. We show that a previously derived bound on the decrease in mutual information for the degrading case is also a lower bound on the increase for the upgrading case. The third result is an efficient algorithm for optimal upgrading, in the binary-input case. That is, we are given a channel and an input distribution. We must find an upgraded channel with L output letters, for which the increase in mutual information is minimal. We give a simple characterization of such a channel, which implies an efficient algorithm. The fourth result is an analog of the first result for the upgrading case when the input is binary. That is, we first present a sub-optimal algorithm for the setting considered in the third result. The main advantage of the sub-optimal algorithm is that it is amenable to analysis. We carry out the analysis and show that the increase incurred in mutual information is within a constant factor of the lower bound derived in the second result.
AB - Consider a channel with a given input alphabet size and input distribution. Our aim is to degrade or upgrade it to a channel with at most L output letters. Channel Q is degraded with respect to a channel W if Q can be obtained from W by processing the output of W. Upgrading is the inverse relation. This paper contains four main results. The first result, from which the paper title is derived, deals with the so called 'greedy-merge' algorithm. We derive an upper bound on the reduction in mutual information between input and output, as a function of L. This upper bound is within a constant factor of an algorithm-independent lower bound. Thus, we establish that greedy-merge is optimal in the power-law sense (i.e. the power of L ). The other main results deal with upgrading. The second result shows that a certain sequence of channels, which was previously shown to be 'hard' for degrading, displays the same hardness in the context of upgrading. That is, suppose we are given such a channel and a corresponding input distribution. If we upgrade (degrade) to a new channel with L output letters, we incur an increase (decrease) in mutual information between input and output. We show that a previously derived bound on the decrease in mutual information for the degrading case is also a lower bound on the increase for the upgrading case. The third result is an efficient algorithm for optimal upgrading, in the binary-input case. That is, we are given a channel and an input distribution. We must find an upgraded channel with L output letters, for which the increase in mutual information is minimal. We give a simple characterization of such a channel, which implies an efficient algorithm. The fourth result is an analog of the first result for the upgrading case when the input is binary. That is, we first present a sub-optimal algorithm for the setting considered in the third result. The main advantage of the sub-optimal algorithm is that it is amenable to analysis. We carry out the analysis and show that the increase incurred in mutual information is within a constant factor of the lower bound derived in the second result.
KW - Channel degrading
KW - channel upgrading
KW - greedy merge
KW - greedy split
KW - polar codes
KW - quantization
UR - https://www.scopus.com/pages/publications/85056312351
U2 - 10.1109/TIT.2018.2879802
DO - 10.1109/TIT.2018.2879802
M3 - Article
SN - 0018-9448
VL - 65
SP - 917
EP - 934
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
IS - 2
M1 - 8525262
ER -