We consider non-stationary free and forced transverse oscillation of an infinite taut string on the Winkler foundation subjected to a discrete mass-spring system non-uniformly moving at a given sub-critical speed. The speed of the mass-spring system is assumed to be a slowly time-varying function less than the critical speed. To describe a non-vanishing free oscillation we use an analytic approach based on the method of stationary phase and the method of multiple scales. The moving oscillator is characterized by a partial frequency, which can be greater or less than the cut-off frequency. Accordingly, a sub-critical uniformly accelerated motion generally has two stages. At the first stage there exists a trapped mode, and, therefore, a part of the wave energy is localized near the moving oscillator and does not propagate away. For this stage we obtain the analytic solution in a simple form describing non-vanishing free oscillation and verify it numerically. For the second stage there is no trapped mode, and all the wave energy propagate away. This stage is investigated numerically, and some unexpected results are obtained. Additionally, we consider the case of the oscillator with a destabilizing spring. The dynamics of the system in the latter case is quite different from the case of commonly used stabilizing spring, since the system loses the stability during an accelerated motion. We also take into consideration the forced oscillation caused by an external load being a superposition of harmonics with time-varying parameters (the amplitude and the frequency).