We review some recent contributions of the authors regarding the numerical approximation of stochastic problems, mostly based on stochastic differential equations modeling random damped oscillators and stochastic Volterra integral equations. The paper focuses on the analysis of selected stability issues, i.e., the preservation of the long-term character of stochastic oscillators over discretized dynamics and the analysis of mean-square and asymptotic stability properties of ϑ -methods for Volterra integral equations.