Computable Lower Bounds for Capacities of Input-Driven Finite-State Channels

Abstract

This paper studies the capacities of input-driven finitestate channels, i.e., channels whose current state is a time-invariant deterministic function of the previous state and the current input. We lower bound the capacity of such a channel using a dynamic programming formulation of a bound on the maximum reverse directed information rate. We show that the dynamic programmingbased bounds can be simplified by solving the corresponding Bellman equation explicitly. In particular, we provide analytical lower bounds on the capacities of (d, k)-runlength-limited inputconstrained binary symmetric and binary erasure channels. Furthermore, we provide a single-letter lower bound based on a class of input distributions with memory.

On the Performance of Reed-Muller Codes Over $(d,\infty)$-RLL Input-Constrained BMS Channels

“…Theorem III.2 states that for the (𝑑, ∞)-RLL input-constrained BEC, a rate of 1 𝑑+1 (1 − 𝜖) is achievable when 𝑑 = 2 𝑡 − 1, for some 𝑡 ∈ N, and a rate of 1 2(𝑑+1) (1 − 𝜖) is achievable, otherwise. We note, however, that using random coding arguments, or using the techniques in [17] or [18], it holds that a rate of 𝐶 (𝑑) 0 (1 − 𝜖) is achievable over the (𝑑, ∞)-RLL input-constrained BEC, where 𝐶 (𝑑) 0 is the noiseless capacity of the input constraint (for example, 𝐶 (1) 0 ≈ 0.6942 and 𝐶 (2) 0 ≈ 0.5515). For the (𝑑, ∞)-RLL input-constrained BSC, similarly, a rate of 1 𝑑+1 (1 − ℎ 𝑏 ( 𝑝)) is achievable when 𝑑 = 2 𝑡 − 1, for some 𝑡 ∈ N, and a rate of 1 2(𝑑+1) (1 − ℎ 𝑏 ( 𝑝)) is achievable, otherwise.…”

“…Thus, Pr 𝑆 𝑚−2−𝑖 ≤ 𝑥 − Pr[𝑍 ≤ 𝑥] ≤ 𝑚 holds for all 𝑥 ∈ R and 𝑖 ∈ [𝑡 𝑚 ]. This yieldsPr 𝑍 ≤ 𝑄 −1 (1 − 𝑅) − 𝜈 𝑚 𝑚 − 2 − 𝑡 𝑚 − 𝑚 − 2 − 𝑡 𝑚 ≤ Pr[𝑆 𝑚−2−𝑖 ≤ 𝑟 𝑚 ] ≤ Pr 𝑍 ≤ 𝑄 −1 (1 − 𝑅) + 𝜈 𝑚 𝑚 − 2 − 𝑡 𝑚 + 𝑚 − 2 − 𝑡 𝑚 .Since 𝑡 𝑚 and 𝜈 𝑚 are both 𝑜( √ 𝑚), we deduce that, as 𝑚 → ∞,Pr[𝑆 𝑚−2−𝑖 ≤ 𝑟 𝑚 ] = 𝑚−2−𝑖 𝑚−2−𝑖 ≤𝑟 𝑚 converges to 𝑅 uniformly in 𝑖 ∈ [𝑡 𝑚 ].Hence, for any 𝛿 ∈ (0, 1) and 𝑚 large enough, it holds for all 𝑖∈ [𝑡 𝑚 ] that 1 2 𝑚−2−𝑖 𝑚 − 2 − 𝑖 ≤ 𝑟 𝑚 ≥ (1 − 𝛿)𝑅,so that, carrying on from(18),ln 1 1 − 𝑅 •2 𝑚−𝑖 − 𝑚 − 2 − 𝑖 ≤ 𝑟 𝑚 ≤ 2 𝑚−3 • 4 ln 1 1 − 𝑅 − (1 − 𝛿)𝑅 .…”

On the Performance of Reed-Muller Codes Over (d, ∞)-RLL Input-Constrained BMS Channels

“…From the discussion in Section II-C, we see that by Theorem III.1, using linear constrained subcodes of RM codes over the (𝑑, ∞)-RLL input-constrained BEC, a rate of 1 𝑑+1 (1−𝜖) is achievable when 𝑑 = 2 𝑡 −1, for some 𝑡 ∈ N, and a rate of 1 2(𝑑+1) (1−𝜖) is achievable, otherwise. We note, however, that using random coding arguments, or using the techniques in [10] or [16], it holds that a rate of 𝐶 (𝑑) 0 (1 − 𝜖) is achievable over the (𝑑, ∞)-RLL input-constrained BEC, where 𝐶 (𝑑) 0 is the noiseless capacity of the input constraint (for example, 𝐶 (1) 0 ≈ 0.6942 and 𝐶 (2) 0 ≈ 0.5515). For the (𝑑, ∞)-RLL input-constrained BSC, similarly, a rate of…”

“…, where 𝐶 is the capacity of the unconstrained BMS channel. Figure 4 shows comparisons, for the specific case of the (1, ∞)-RLL input-constrained BEC, of the upper bound of min 7 8 • (1 − 𝜖), 𝐶 (1) 0 , obtained by sub-optimal decoding, in Theorem III.4, with the achievable rate of 𝐶 (1) 0 • (1 − 𝜖) (from [11] and [16]), and the numerically computed achievable rates using the Monte-Carlo method in [40] (or the stochastic approximation scheme in [15]). For large values of the erasure probability 𝜖, we observe that the upper bound of min 7 8 • (1 − 𝜖), 𝐶 (1) 0 lies below the achievable rates of [40], thereby indicating that it is not possible to achieve the capacity of the (1, ∞)-RLL input-constrained BEC, using (1, ∞)-RLL subcodes of {C 𝑚 (𝑅)} 𝑚≥1 , when the bit-MAP decoders of {C 𝑚 (𝑅)} 𝑚≥1 are used for decoding.…”

“…We mention also that unlike the case of the unconstrained DMC, whose capacity is characterized by Shannon's single-letter formula, 𝐶 DMC = sup 𝑃 ( 𝑥) 𝐼 (𝑋; 𝑌 ), the computation of the capacity of a DFSC, given by the maximum mutual information rate between inputs and outputs, is a much more difficult problem to tackle, and is open for many simple channels. In fact, the computation of the mutual information rate even for the simple case of Markov inputs, over input-constrained DMCs, reduces to the computation of the entropy rate of a hidden Markov process-a well-known hard problem (see [10]- [16]).…”