We consider dynamics of a quantum nonlinear oscillator subjected to non-Markovian pumping. Models of this kind can describe formation of exciton-polariton Bose-Einstein condensates in course of stimulated scattering and relaxation of reservoir excitons. Using the Markovian embedding techniques, we obtain stochastic differential equations of motion with an additional degree of freedom corresponding to dynamical memory. It is shown that the oscillator asymptotically tends to the intrinsically non-Markovian stable fixed point corresponding to constant product of oscillator amplitude and modulo of the memory variable. The state corresponding to this point exhibits unlimited growth of population, with the growth rate that decreases with time. Our results show that the Markovian behavior could be observed only within some limited early stage of oscillator evolution provided that decay of dynamical memory is sufficiently fast. Transition from the Markovian regime to non-Markovian one with increasing time is linked to phase shift of the pumping term. Coherence properties of the oscillator are studied. It was found that interaction between particles delimits growth of condensate population, thereby impeding formation of Bose-Einstein condensate.