Generalizations of Nonanticipative Rate Distortion Function to Multivariate Nonstationary Gaussian Autoregressive Processes

Charalambous, Charalambos D.; Kourtellaris, Christos K.; Charalambous, Themistoklis; Schuppen, Jan H. van

doi:10.1109/cdc40024.2019.9029859

Cited by 6 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For case p 1 = p 2 = 1, our results reproduce the value of the RDF derived by Xiao and Luo [3]. The tools used in this paper have been used to derive structural properties of distributed RDFs [9] and the nonanticipative RDF of multivariate Gaussian Markov [10] and autoregressive [11] processes.…”

Section: Main Contributionssupporting

confidence: 79%

Joint Rate Distortion Function of a Tuple of Correlated Multivariate Gaussian Sources with Individual Fidelity Criteria

Stylianou

Charalambous

2021

2021 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

In this paper we analyze the joint rate distortion function (RDF), for a tuple of correlated sources taking values in abstract alphabet spaces (i.e., continuous) subject to two individual distortion criteria. First, we derive structural properties of the realizations of the reproduction Random Variables (RVs), which induce the corresponding optimal test channel distributions of the joint RDF. Second, we consider a tuple of correlated multivariate jointly Gaussian RVs, X 1 : Ω → R p 1 , X 2 : Ω → R p 2 with two square-error fidelity criteria, and we derive additional structural properties of the optimal realizations, and use these to characterize the RDF as a convex optimization problem with respect to the parameters of the realizations. We show that the computation of the joint RDF can be performed by semidefinite programming. Further, we derive closed-form expressions of the joint RDF, such that Gray's [1] lower bounds hold with equality, and verify their consistency with the semidefinite programming computations. I. LITERATURE REVIEW, PROBLEM FORMULATION, AND MAIN CONTRIBUTIONS A. Literature ReviewGray [1, Theorem 3.1, Corollary 3.1] derived lower bounds on the joint rate distortion functions (RDFs), of a tuple of Random Variables (RVs) taking values in arbitrary, abstract spaces, X 1 : Ω → X 1 , X 2 : Ω → X 2 , with a weighted distortion, expressed in terms of conditional RDFs, and marginal RDFs. Gray and Wyner in [2], characterized the rate distortion region of a tuple of correlated RVs, using the joint, conditional and marginal RDFs. Xiao and Luo [3, Theorem 6] derived the closed-form expression of the joint RDF for a tuple of scalar-valued correlated Gaussian RVs, with two square-error distortion criteria, while Lapidoth and Tinguely [4] re-derived Xiao's and Luo's joint RDF using an alternative method. Xu, Liu and Chen [5] and Viswanatha, Akyol and Rose [6], generalized Wyner's common information [7] to its lossy counterpart, as the minimum common message rate on the Gray and Wyner rate region with sum rate equal to the joint RDF with two individual distortion functions. The analysis in [5], [6], includes the application of a tuple of scalar-valued, jointly Gaussian RVs. More recent work on rates that lie on the Gray and Wyner rate region are found in [8]. SourceEncoder: f E (•)

show abstract

Section: Main Contributionssupporting

confidence: 79%

Joint Rate Distortion Function of a Tuple of Correlated Multivariate Gaussian Sources with Individual Fidelity Criteria

Stylianou

Charalambous

2021

2021 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Kostina and Hassibi in [8] revisited the framework of [5] and derived bounds on the optimal rate-cost tradeoffs in control for time-invariant fully observable multivariate Markov processes under the assumption of uniform cost (or distortion) allocation. Recently, Charalambous et al in [13] used a state augmentation technique to extend the characterization of the Gaussian nonanticipatory −entropy derived in [2] to nonstationary multivariate Gaussian autoregressive models of any finite order.…”

Section: Introductionmentioning

confidence: 99%

Indirect NRDF for Partially Observable Gauss-Markov Processes with MSE Distortion: Complete Characterizations and Optimal Solutions

Stavrou¹,

Skoglund²

2021

Preprint

View full text Add to dashboard Cite

In this paper we study the problem of characterizing and computing the nonanticipative rate distortion function (NRDF) for partially observable multivariate Gauss-Markov processes with hard mean squared error (MSE) distortion constraints. For the finite time horizon case, we first derive the complete characterization of this problem and its corresponding optimal realization which is shown to be a linear functional of the current time sufficient statistic of the past and current observations signals. We show that when the problem is strictly feasible, it can be computed via semidefinite programming (SDP) algorithm. For time-varying scalar processes with average total MSE distortion we derive an optimal closed form expression by means of a dynamic reverse-waterfilling solution that we also implement via an iterative scheme that convergences linearly in finite time, and a closed-form solution under pointwise MSE distortion constraint. For the infinite time horizon, we give necessary and sufficient conditions to sure that asymptotically the sufficient statistic process of the observation signals achieves a steady-state solution for the corresponding covariance matrices and impose conditions that allow existence of a time-invariant solution. Then, we show that when a finite solution exists in the asymptotic limit, it can be computed via SDP algorithm. We also give strong structural properties on the characterization of the problem in the asymptotic limit that allow for an optimal solution via a reverse-waterfilling algorithm that we implement via an iterative scheme that converges linearly under a finite number of spatial components. Subsequently, we compare the computational time needed to execute for both SDP and reverse-waterfilling algorithms when these solve the same problem to show that the latter is a scalable optimization technique. Our results are corroborated with various simulation studies and are also compared with existing results in the literature.

show abstract

“…Gaussian zero mean noise processes. Recently the authors of [13] used a state augmentation technique to extend the characterization of the Gaussian nonanticipatory −entropy derived in [3] to nonstationary multivariate Gaussian autoregressive models of any finite order.…”

Section: Introductionmentioning

confidence: 99%

A novel sequential RDF to compute partially observable Markov processes with MSE distortion

Stavrou¹,

Skoglund²

2020

Preprint

View full text Add to dashboard Cite

<div>We develop a new sequential rate distortion function</div><div>to compute lower bounds on the average length of all causal prefix free codes for partially observable multivariate Markov processes with mean-squared error distortion constraint. Our information measure is characterized by a variant of causally conditioned directed information and is utilized in various application examples. First, it is used to optimally characterize a finite dimensional optimization problem for jointly Gaussian processes and to obtain the corresponding optimal linear encoding and decoding policies.</div><div>Under the assumption that all matrices commute by pairs,</div><div>we show that our problem can be cast as a convex program</div><div>which achieves its global minimum. We also derive sufficient</div><div>conditions which ensure that our assumption holds. We then</div><div>solve the KKT conditions and derive a new reverse-waterfilling algorithm that we implement. If our assumption is violated, one can still use our approach to derive sub-optimal (upper bound) waterfilling solutions. For scalar-valued Gauss-Markov processes with additional observation noise, we derive a new closed form solution and we compare it with known results in the literature. For partially observable time-invariant Markov processes driven by additive i:i:d: system noise only, we recover using an alternative approach and thus strengthening a recent result by Kostina and Hassibi in [1, Theorem 9] whereas for timeinvariant and spatially IID Markov processes driven by additive noise process we also derive new analytical lower bounds.</div>

show abstract

Generalizations of Nonanticipative Rate Distortion Function to Multivariate Nonstationary Gaussian Autoregressive Processes

Cited by 6 publications

References 8 publications

Joint Rate Distortion Function of a Tuple of Correlated Multivariate Gaussian Sources with Individual Fidelity Criteria

Joint Rate Distortion Function of a Tuple of Correlated Multivariate Gaussian Sources with Individual Fidelity Criteria

Indirect NRDF for Partially Observable Gauss-Markov Processes with MSE Distortion: Complete Characterizations and Optimal Solutions

A novel sequential RDF to compute partially observable Markov processes with MSE distortion

Contact Info

Product

Resources

About