Dynamics of a nonlinear second order system with a stochastically varying mass is considered in the article. The system is under an external time-harmonic loading, which is assumed to be near resonant. Damping, excitation strength, nonlinearity, and mass variations are considered to be of the same order of smallness. Under these assumptions, an explicit approximate solution of the problem is obtained using the multiple scales perturbation method. The leading (zero) order component of the solution is not affected by noise, whereas the first order component is stochastic and is energetically unstable because of mass variations. This implies that the considered oscillator will not vibrate in the near resonant regime because of mass variations. Instead, its vibrations will feature a considerable, but limited in amplitude, stochastic component. These vibrations will be stable, which illustrates a qualitative difference of the phenomenon from the motion instability described previously for linear stochastic oscillators for lower values of damping. The presence of the nonlinearity does not considerably affect the stability. To avoid the phenomenon, damping in the system should be increased so as to be much larger than mass variations. The obtained results were validated by a series of numerical experiments. The oscillator can be considered as a simplified model of machines used for processing of granular materials, such as vibrating screens, and the results are relevant for various applications, including mining industry.