The stability of iterations of affine linear maps $\Psi_{n}(x)=A_{n}x+B_{n}$,
$n=1,2,\ldots$, is studied in the presence of a Markovian environment, more
precisely, for the situation when $(A_{n},B_{n})_{n\ge 1}$ is modulated by an
ergodic Markov chain $(M_{n})_{n\ge 0}$ with countable state space
$\mathcal{S}$ and stationary distribution $\pi$. We provide necessary and
sufficient conditions for the a.s. and the distributional convergence of the
backward iterations $\Psi_{1}\circ\ldots\circ\Psi_{n}(Z_{0})$ and also describe
all possible limit laws as solutions to a certain Markovian stochastic
fixed-point equation. As a consequence of the random environment, these limit
laws are stochastic kernels from $\mathcal{S}$ to $\mathbb{R}$ rather than
distributions on $\mathbb{R}$, thus reflecting their dependence on where the
driving chain is started. We give also necessary and sufficient conditions for
the distributional convergence of the forward iterations
$\Psi_{n}\circ\ldots\circ\Psi_{1}$. The main differences caused by the
Markovian environment as opposed to the extensively studied case of independent
and identically distributed (iid) $\Psi_{1},\Psi_{2},\ldots$ are that: (1)
backward iterations may still converge in distribution, if a.s. convergence
fails, (2) the degenerate case when $A_{1}c_{M_{1}}+B_{1}=c_{M_{0}}$ a.s. for
suitable constants $c_{i}$, $i\in\mathcal{S}$, is by far more complex than the
degenerate case for iid $(A_{n},B_{n})$ when $A_{1}c+B_{1}=c$ a.s. for some
$c\in\mathbb{R}$, and (3) forward and backward iterations generally have
different laws given $M_{0}=i$ for $i\in\mathcal{S}$ so that the former ones
need a separate analysis. Our proofs draw on related results for the iid-case,
notably by Vervaat, Grincevi\v{c}ius, and Goldie and Maller, in combination
with recent results by the authors on fluctuation theory for Markov random
walks.Comment: 36 page