Synchronization of stochastically coupled chaotic oscillators is a topic of intensive research for its ubiquitous application across natural and technological systems. Several breakthroughs have been made over the last decade in understanding the underpinnings of stochastic synchronization. Yet, most of the literature has focused on memoryless switching, where the coupling between the oscillators intermittently changes independently of the switching history. Here, we analytically investigate the synchronization of two one-dimensional coupled nonlinear maps under Markovian switching. We linearize the system in the vicinity of the synchronous solution and examine the mean square asymptotic stability of the error dynamics. By leveraging state-of-the-art techniques in jump linear systems, fundamentals of ergodic theory, and perturbation analysis, we elucidate the potential of Markovian switching in manipulating the stability of synchronization. We focus on chaotic tent maps, for which we compute exact, closed-form expressions to measure the error dynamics. The hypothesis of memoryless switching has often been challenged in practical applications; this study makes a first, necessary step toward unraveling the role of switching memory in stochastic synchronization.