The Double-Sided Information Bottleneck Function †

A double-sided variant of the information bottleneck method is considered. Let (Formula presented.) be a bivariate source characterized by a joint pmf (Formula presented.). The problem is to find two independent channels (Formula presented.) and (Formula presented.) (setting the Markovian structure (Formula presented.)), that maximize (Formula presented.) subject to constraints on the relevant mutual information expressions: (Formula presented.) and (Formula presented.). For jointly Gaussian (Formula presented.) and (Formula presented.), we show that Gaussian channels are optimal in the low-SNR regime but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low and are suboptimal for high correlations. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., (Formula presented.)) and provide supporting numerical evidence. Furthermore, we present a Blahut–Arimoto type alternating maximization algorithm and demonstrate its performance for a representative setting. This problem is closely related to the domain of biclustering.