We study the quantum transport and the nonequilibrium vibrational states of a quantum dot embedded between a normal-conducting and a superconducting lead with the charge on the quantum dot linearly coupled to a harmonic oscillator of frequency ω. To the leading order in the charge-vibration interaction, we calculate the current and the nonequilibrium phonon occupation by the Keldsyh Green's function technique. We analyze the inelastic, vibration-assisted tunneling processes in the regime ω < , with the superconducting energy gap , and for sharp resonant transmission through the dot. When the energy ε 0 of the dot's level is close to the Fermi energy μ, i.e., |ε 0 − μ| , inelastic vibration-assisted Andreev reflections dominate up to voltage eV . The inelastic quasiparticle tunneling becomes the leading process when the dot's level is close to the superconducting gap |ε 0 − μ| ∼ ± ω. In both cases, the inelastic tunneling processes appear as sharp and prominent peaks-not broadened by temperature-in the current-voltage characteristic and pave the way for inelastic spectroscopy of vibrational modes even at temperatures T ω. We also found that inelastic vibration-assisted Andreev reflections as well as quasiparticle tunneling induce a strong nonequilibrium state of the oscillator. In different ranges on the dot's level, we found that the current produces: (i) ground-state cooling of the oscillator with phonon occupation n 1, (ii) accumulation of energy in the oscillator with n 1, and (iii) a mechanical instability characterized by a negative damping coefficient. We show that ground-state cooling is achieved simultaneously for several modes of different frequencies. Finally, we discuss how the nonequilibrium vibrational state can be readily detected by the asymmetric behavior of the inelastic current peaks with respect to the gate voltage.