On the coding theorem and its converse for finite-memory channels

Feinstein, Amiel

doi:10.1016/s0019-9958(59)90066-x

Cited by 42 publications

(28 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proof of C Shannon ≤ C S . The proof is similar to the one in [10], so we just outline the main steps.…”

Section: Resultsmentioning

confidence: 96%

“…For this, we observe that the asymptotic mean stationarity [13] of the output process makes it possible to apply tools from ergodic theory to establish the existence of the mutual information rate of our channel and further the asymptotic equipartition property of the output process. Another issue is to mix the "blocked" processes to obtain a stationary process achieving the information capacity, for which we find an adaptation of Feinstein's method [10] as a solution.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the capacity of multilevel NAND flash memory channels

Lü¹,

Kavcic²,

Han³

2016

2016 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

In this paper, we initiate a first information-theoretic study on multilevel NAND flash memory channels [2] with intercell interference. More specifically, for a multilevel NAND flash memory channel under mild assumptions, we first prove that such a channel is indecomposable and it features asymptotic equipartition property; we then further prove that stationary processes achieve its information capacity, and consequently, as the order tends to infinity, its Markov capacity converges to its information capacity; eventually, we establish that its operational capacity is equal to its information capacity. Our results suggest that it is highly plausible to apply the ideas and techniques in the computation of the capacity of finite-state channels, which are relatively better explored, to that of the capacity of multilevel NAND flash memory channels.

show abstract

“…Proof of C Shannon ≤ C S . The proof is similar to the one in [10], so we just outline the main steps.…”

Section: Resultsmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

On the capacity of multilevel NAND flash memory channels

Lü¹,

Kavcic²,

Han³

2016

2016 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…Subsequently H (Z n |Z n−k ) is given by Eq. (15). Putting everything together, we obtain the lemma.…”

Section: B the Second Termmentioning

confidence: 86%

“…With an abuse of notation, we therefore drop all subscripts and use p to denote the respective probability measure. Let W n = Y n , X The main proof is a modification of Feinstein's work [15], which is under the ergodic-theoretic framework. We exploit his construction of an SE probability measure from a finitedimensional probability measure.…”

Section: Appendix Bmentioning

confidence: 99%

See 1 more Smart Citation

On Capacity Formulation With Stationary Inputs and Application to a Bit-Patterned Media Recording Channel Model

Nguyen

Armand

2015

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

In this correspondence, we illustrate among other things the use of the stationarity property of the set of capacityachieving inputs in capacity calculations. In particular, as a case study, we consider a bit-patterned media recording channel model and formulate new lower and upper bounds on its capacity that yield improvements over existing results. Inspired by the observation that the new bounds are tight at low noise levels, we also characterize the capacity of this model as a series expansion in the low-noise regime.The key to these results is the realization of stationarity in the supremizing input set in the capacity formula. While the property is prevalent in capacity formulations in the ergodictheoretic literature, we show that this realization is possible in the Shannon-theoretic framework where a channel is defined as a sequence of finite-dimensional conditional probabilities, by defining a new class of consistent stationary and ergodic channels.Index Terms-Channel capacity, stationary inputs, stationary and ergodic channel, bit-symmetry, bit-patterned media recording, lower/upper bounds, series expansion. I. BACKGROUNDThe fundamental limit of information transmission through noisy channels, the channel capacity, has been a holy grail in information theory. The capacity problem of a general pointto-point channel has been well resolved with the informationspectrum framework [1]. Such general formula for the capacity, however, does not lend itself to computation in general, since it requires one to scrutinize the distribution of the information density at the limit of infinite block length. To overcome this problem, a common approach is to find an alternative expression that, instead of being described by an information-spectrum quantity, contains a mutual information quantity (or entropy quantities). While an expression of this kind is not as general, it may cover a sufficiently large class of channels for many practical purposes.There are two popular forms of such expression. One is Dobrushin's information-stable channel capacity [2]:where the supremum is over all possible sequences of distributions P (n) : X n ∼ P (n) . This formula holds for the class of information-stable channels. A similar formula also appears in the context of (decomposable or indecomposable) finitestate channels [3]. The other form swaps the supremum and the limit in the above formula, with the supremum being taken over a smaller set of input distributions with special structures. This type of formula appears in the ergodic-theoretic literature of information theory. For example, ford-continuous discrete stationary and ergodic (SE) two-sided channels, the capacity was shown to be [4]where µ is a probability measure that describes the input process. (See Section I-B for the distinction between the two sets being supremized over in the above formulas.) Such capacity formulations that involve supremization over stationary inputs are common in the ergodic-theoretic setting, where a channel only admits infinitely long input sequences,...

show abstract

Information Theory of Stochastic Processes

Kieffer

1999

Wiley Encyclopedia of Electrical and Electronics Engineering

View full text Add to dashboard Cite

show abstract

On the coding theorem and its converse for finite-memory channels

Cited by 42 publications

References 3 publications

On the capacity of multilevel NAND flash memory channels

On the capacity of multilevel NAND flash memory channels

On Capacity Formulation With Stationary Inputs and Application to a Bit-Patterned Media Recording Channel Model

Information Theory of Stochastic Processes

Contact Info

Product

Resources

About