The reduction of the photoluminescence (PL) decay time of a colloidal quantum dot (QD) inserted into an Ag or Au surface nanohole and the efficiency enhancement of the Förster resonance energy transfer (FRET) from a green-emitting QD into a red-emitting QD are first experimentally demonstrated. Besides the factor of metal dissipation in the induced surface plasmon (SP) coupling process, the reduced PL decay time is attributed to the QD emission efficiency increase caused by the SP-coupling involved nanoscale-cavity effect. Numerical simulation studies are undertaken to confirm the feasible enhancements of QD emission, FRET, and color conversion efficiencies. In particular, by artificially changing the dielectric constant of Ag based on the Drude model, the effects of cavity resonance and SP coupling in producing the enhanced radiated power peaks can be differentiated. Such a peak can be formed when both conditions of cavity resonance and SP resonance are satisfied. In the case of a weaker (stronger) SP resonance, the combined resonance can lead to a stronger and sharper (weaker and broader) radiated power peak. The results in this paper indicate that a nanoscale metal cavity can be used for enhancing the emission and color conversion efficiencies of inserted light emitters.