Averaging, aggregation and optimal control of singularly perturbed stochastic hybrid systems

Abstract

This paper develops methodologies for the averaging, aggregation and optimal control of stochastic hybrid systems whose state equations depend on continuoustime, nearly completely decomposable ® nite state Markov chains. Such Markovian switching models are assumed to have several identi® ed groups of strongly interacting states. The random sample solution process of the system state can be well approximated by a deterministic trajectory, for the duration of an interval of which the switching process sojourns in a group. Aggregated models are obtained by utilizing the aggregation method over the in® nite time interval. By using the perturbation approach, necessary and su cient conditions of stochastic stabilizability and controllability proposed by Ji and Chizeck (1990) are presented for the systems. With the aggregation models and the known results of stochastic stabilizability, near-optimum ® nite time and in® nite time Markovian jump linear quadratic control laws are also investigated. The limiting behaviour of corresponding minimized averaged cost functions is studied. Finally, an illustrative example is performed to illustrate the feasibility and e ectiveness of the proposed techniques.

Empirical averaging in deterministic optimal control

“…Averaging has been also used is stochastic optimal control, notably of Markov chains (see for instance 20,21,22 ). Indeed, when there exists a cycle in a discrete state Markov chain that has high transition probabilities, then this cycle is gone through very fast and averaging can be applied.…”

“…Proposition 2. Let 𝑥 defined by the ODE (21) and let 𝑥 0 defined by the averaged ODE (22). We define the signal 𝐼[𝑓 , 𝑥 0 ] by…”

Stabilization of Markovian Systems via Probability Rate Synthesis and Output Feedback

“…In [21], stabilizing switching signals have been constructed when none of the individual subsystems is asymptotically stable. Closed-loop and open-loop switching control strategies have been investigated in [22] and [23]. State-feedback and output-feedback switching design have been considered in [27], [28].…”

Robust Control for Singularly Perturbed Markov Jumping Linear Systems

“…The Markovian jump linear system (MJLS) has become popular for modeling these changes. In [4], a special class of stochastic hybrid systems whose Markov switching models are singularly perturbed was studied.…”