Half century has past since the pioneering works of Anderson and Kubo on the stochastic theory of spectral line shape were published in J. Phys. Soc. Jpn. 9 (1954) 316 and 935, respectively. In this review, we give an overview and extension of the stochastic Liouville equation focusing on its theoretical background and applications to help further the development of their works. With the aid of path integral formalism, we derive the stochastic Liouville equation for density matrices of a system. We then cast the equation into the hierarchy of equations which can be solved analytically or computationally in a nonperturbative manner including the effect of a colored noise. We elucidate the applications of the stochastic theory from the unified theoretical basis to analyze the dynamics of a system as probed by experiments. We illustrate this as a review of several experimental examples including NMR, dielectric relaxation, Mössbauer spectroscopy, neutron scattering, and linear and nonlinear laser spectroscopies. Following the summary of the advantage and limitation of the stochastic theory, we then derive a quantum Fokker-Planck equation and a quantum master equation from a system-bath Hamiltonian with a suitable spectral distribution producing a nearly Markovian random perturbation. By introducing auxiliary parameters that play a role as stochastic variables in an expression for reduced density matrix, we obtain the stochastic Liouville equation including temperature correction terms. The auxiliary parameters may also be interpreted as a random noise that allows us to derive a quantum Langevin equation for non-Markovian noise at any temperature. The results afford a basis for clarifying the relationship between the stochastic and dynamical approaches. Analytical as well as numerical calculations are given as examples and discussed.