State-Dependent Generalizations of Nonanticipatory Epsilon Entropy of Partially Observable Processes

Abstract

Two fundamental generalizations of Gorbunov's and Pinsker's nonanticipatory ϵ−entropy are formulated and analyzed for a tuple of partially observable, finite-dimensional, random processes (X n , Y n ), where X n △ = {X1, . . . , Xn} is the unobserved state process and Y n △ = {Y1, . . . , Yn} is the observable process-a noisy version of X n , subject to a fidelity between X n , and its reproduction X n △ = { X1, . . . , Xn}. The encoder observes causally Y n and past reproductions X n may or may not be available to both the encoder and the decoder. Theorem 1 gives a tight lower bound on the operational rate of zero-delay codes, when X n is causally available to the decoder only, in terms of a state-dependent nonanticipatory ϵ−entropy of a state process Z n , which is fundamentally different from a corresponding nonanticipatory ϵ−entropy, when X n is causally available to both the encoder and the decoder. Theorem 2 identifies sufficient conditions for the two nonanticipatory ϵ−entropies to coincide. Theorem 3 identifies the information structure of the optimal test-channel distributions. The paper also discusses applications to jointly Gaussian partially observable processes (X n , Y n ) with a square-error fidelity criterion, and derives characterizations of the two nonanticipatory ϵ−entropies.