Bivariate Markov Processes and Their Estimation

Abstract

A bivariate Markov process comprises a pair of random processes which are jointly Markov. One of the two processes in that pair is observable while the other plays the role of an underlying process. We are interested in three classes of bivariate Markov processes. In the first and major class of interest, the underlying and observable processes are continuous-time with finite alphabet; in the second class, they are discrete-time with finite alphabet; and in the third class, the underlying process is continuous-time with uncountably infinite alphabet, and the observable process is continuous-time with countably or uncountably infinite alphabet. We refer to processes in the first two classes as bivariate Markov chains. Important examples of continuoustime bivariate Markov chains include the Markov modulated Poisson process, and the batch Markovian arrival process. A hidden Markov model with finite alphabet is an example of a discrete-time bivariate Markov chain. In the third class we have diffusion processes observed in Brownian motion, and diffusion processes modulating the rate of a Poisson process. Bivariate Markov processes play central roles in the theory and applications of estimation, control, queuing, biomedical engineering, and reliability. We review properties of bivariate Markov processes, recursive estimation of their statistics, and recursive and iterative parameter estimation.

Information Landscape and Flux, Mutual Information Rate Decomposition and Connections to Entropy Production

“…We use numerical simulations, which evaluate R x , I(X, S) and I B (X, S) directly from the typical sequences of Z (see [7,8]). The corresponding results can be given by:…”

“…Although the BMC has been studied for decades, there are still challenges on quantifying the dynamics of the whole, as well as the two subsystems. This is because neither of them needs to be a Markovian chain in general [7], and the quantifications of the probabilities (densities) for the trajectories of the two subsystems involve the complicated random matrix multiplications [8]. This leads to the problem not exactly being analytically solvable.…”

“…There is growing interest in studying two interacting information systems in the fields of control theory, information theory, communication theory, nonequilibrium physics and biophysics [1][2][3][4][5][6][7][8][9]. Significant progresses has been made recently towards the understanding of the information system in terms of information thermodynamics [10][11][12][13].…”

Explicit Causal Recursive Estimators for Continuous-Time Bivariate Markov Chains

“…The process is assumed observable, while the process is the underlying process. Such a bivariate Markov chain may be obtained by applying an aggregating deterministic function to a Markov chain [13], [22]. The processes and may jump simultaneously.…”

“…For example, the MMPP has an HMM representation, as was first shown in [31]. A thorough review of bivariate Markov processes, in discrete and continuous time, with finite as well as continuous alphabet, may be found in [13]. In this paper, we exclusively focus on finite-state continuous-time bivariate Markov chains which, for brevity, we shall simply refer to as "bivariate Markov chains.…”