We first review fundamentals of the theory of stochastic processes. The system dynamics are specified by the set of its states, {S}, and the transitions between them, S → S , where S, S ∈ {S}. For example, the state S can denote the position of a Brownian particle, the numbers of molecules of different chemical species, or any other variable that characterizes the state of the system of interest. Here we restrict ourselves to processes for which the transition rates depend only on the system's instantaneous state, and not the entirety of its history. Such memoryless processes are known as Markovian and are applicable to a wide range of systems. We also assume that the transition rates do not explicitly depend on time, a condition known as stationarity. In this review we make the standard assumption that the transitions between the states are Poisson distributed random processes. In other words, the probability of transitioning from state S to state S in an infinitesimal interval, dt, is α(S, S )dt, where α(S, S ) is the transition rate.Examples. For a Brownian particle diffusing along a line, the state S is defined by the particle position; the transition rate is 2D/d 2 , where D is the diffusion coefficient and d is the step length. For a set of radioactive atoms undergoing decay with rate κ per atom, the state S is defined by the number, n, of atoms that have not decayed yet, and the transition rate from state n to state n − 1 is κ n. For a system with N reacting chemical species, the system state is defined by the concentrations of each reactant, (x 1 . . . x N ), and the transition rates are functions of these concentrations.B. Time evolution equation(s) for the system.We now summarize the equations that govern the dynamical evolution of the probability that the system is in state S at time t, which we denote by P (S, t). Typically, such equations are written in one of three formalisms: the Master Equation, the Fokker-Planck Equation, or the Stochastic Differential Equation; each is summarized below in turn. Details of the