Dynamic Programming for Quantization of q-ary Input Discrete Memoryless Channels

Abstract

In this paper, under a general cost function, we present a dynamic programming (DP) method to obtain an optimal sequential deterministic quantizer (SDQ) for q-ary input discrete memoryless channel (DMC). The DP method has complexity O(q(N − M ) 2 M ), where N and M are the alphabet sizes of the DMC output and quantizer output, respectively. Then, starting from the quadrangle inequality (QI), two techniques are applied to reduce the DP method's complexity. One technique makes use of the SMAWK algorithm and achieves complexity O(q(N − M )M ). The other technique is much easier to be implemented and achieves complexity O(q(N 2 − M 2 )). We further derive a sufficient condition under which the optimal SDQ is optimal among all quantizers and the two techniques are also applicable. This generalizes the results in the literature for binary-input DMC. Next, we show that the cost function of α-mutual information (α-MI)-maximizing quantizer belongs to the general cost function we adopt earlier. We further prove that under a weaker condition than the sufficient condition we derived, the aforementioned two techniques are applicable to the design of α-MI-maximizing quantizer. Finally, we propose a new algorithm called iterative DP (IDP). Theoretical analysis and simulation results demonstrate that IDP can improve the quantizer design over the state-of-the-art methods in the literature.Index Terms-α-mutual information (α-MI), discrete memoryless channel (DMC), dynamic programming (DP), quadrangle inequality (QI), sequential deterministic quantizer (SDQ). P Z|Y (z|y) = 1 z = z , 0 z = z , or equivalently, we say y's quantization result Q(y) is a deterministic element in Z. In this paper, a general cost function given by (2) is considered, which is widely adopted in the literature [12]-[14]. Under such a condition, we show that there always exists at least one DQ that is optimal among all quantizers. Due to this reason as well as that DQ is more practical than non-deterministic quantizer, we focus only on DQs in this paper. For any DQ Q : Y → Z, denote Q −1 (z) ⊂ Y as the preimage of z ∈ Z.For binary-input DMC, dynamic programming (DP) [15, Section 15.3] was applied by Kurkoski and Yagi [16] to design quantizers that maximize the mutual information (MI) between X and Z, i.e., I(X; Z). The complexity (refer to the computational complexity throughout this paper unless the storage complexity is specified) of this DP method was reduced [17], [18] by applying the SMAWK algorithm [19]. However, for the general q-ary input DMC with q > 2, design of the optimal quantizers that maximize I(X; Z) is an NP-hard problem [14], [20]. Up till now, only the necessary condition [12], rather than the sufficient condition, has been established for the optimal quantizer; meanwhile, there only exist some suboptimal design methods in practice [5], [14], [21]-[23]. Moreover, to the best of our knowledge, so far no work has arXiv:1901.01659v3 [cs.IT] 28 Oct 2019

On Mutual Information-Maximizing Quantized Belief Propagation Decoding of LDPC Codes

“…A method based on dynamic programming (DP) [22,Section 15.3] was proposed in [20] to find Q * with complexity O((N − M ) 2 M ). Moreover, a general framework has been developed in [21] for applying DP to find an optimal SDQ Λ * to maximize I(X; Z), for cases that the labeling of the elements in Y is fixed and Λ * is an SDQ. The quantization model in Fig.…”

“…where 1 ≤ i ≤ |B| and P L,S|X (l, s|x) is given by (9). Third, based on B and P B|X , we can apply the general DP method proposed in [21] to find an SDQ…”

A Recursive Quantizer Design Algorithm for Binary-Input Discrete Memoryless Channels

“…This algorithm has complexity O(JKM T ) where T is the number of iterations that the algorithm is run to converge to a local optimum. Another example is a dynamic programming method [6] with complexity O(JK(M − K) 2 ) to find an optimal sequential deterministic quantizer under a general cost function. The authors also derive a sufficient condition for general optimality of this method and under a condition for the DMC channel, they propose two techniques to reduce the complexity of their algorithm.…”

Optimal quantizer structure for binary discrete input continuous output channels under an arbitrary quantized-output constraint

“…Motivated by many applications in designing of the communication decoder i.e., polar code decoder [1] and LDPC code decoder [2], designing the optimal quantizer that maximizes the mutual information between input and quantized-output recently has received much attention from both information theory and communication theory society. Over a past decade, many algorithms was proposed [3], [4], [5], [6], [7], [8], [9], [10], [11]. Due to the non-linearity of quantization/partition problem, finding the global optimal quantizer is an extremely hard problem [12].…”

“…However, it is well-known that if the channel input is binary, then the optimal quantizer has a structure of convex cells in the space of posterior distribution and the global optimal quantizer can be found efficiently in a polynomial time by using dynamic programming technique [3]. In [5] and [11], the time complexity can be further reduced to a linear time complexity using the famous SMAWK algorithm.…”