Delay-based optoelectronic oscillators (OEOs) use the large propagation delays of long optical fibers to produce ultra-low phase noise (PN) radio frequency (RF) oscillations. Most of the OEO-PN analysis approaches in the literature predict a singular value for the output variable's PN-induced power spectral density (PSD) at zero offset frequency from the carrier. The current paper aims to resolve this issue. First the perturbation theory of classical oscillators is generalized to extract the stochastic delay differential equation (SDDE) governing the PN of OEOs. Then the well-known small delay approximation approach is used to derive a stochastic ordinary differential equation (SODE) governing the PN. Using the previously published solutions to this SODE, the PN-PSD of the OEO is extracted in presence of both white and colored noise sources. As the small delay approximation is only valid for small frequency offsets, the obtained PSDs are only accurate at these offsets. This approach avoids the non-physically large values encountered in classical approaches. For larger frequency offsets one can use the classical noise analysis techniques in the literature. The validity of this approach is verified by comparing its results with those in the literature and direct Monte-Carlo simulations.