The collective multipole excitations are studied in the framework of relativistic random-phase approximation with the vacuum polarization. First, we show for the nuclear ground state that the leading order of derivative expansion of the effective action arising from the vacuum correction agrees with the exact calculation using the Green function method very well. The derivative expansion makes us easy to perform a fully self-consistent calculation, even for the random-phase approximation. A remarkable effect of the inclusion of the vacuum polarization is the increase of the effective mass meff/mN ~ 0.9, which gives, for all multipole modes, smaller energy-weighted sum rule values than those of the typical relativistic model. Also, the large effective mass constrained by the vacuum polarization can give an excellent agreement with experimental data on the excitation energy for the isoscalar quadrupole resonances. It is shown, further, that the change of the shell structure due to the vacuum polarization plays an important role in the dipole compression modes.