The use of nanostructures to enhance the emission of single-photon sources has attracted some attention in the last decade due to the development of quantum technologies. In particular, the use of metallic and high-refractive-index dielectric materials has been proposed. However, the utility of moderate-refractive-index dielectric nanostructures to achieve more efficient single-photon sources remains unexplored. Here, a systematic comparison of various metallic, high-refractive-index and moderate-refractive-index dielectric nanostructures was performed to optimize the excitation and emission of a CdSe/ZnS single quantum dot in the visible spectral region. Several geometries were evaluated in terms of electric field enhancement and Purcell factor, considering the combination of metallic, high-refractive-index and moderate-refractive-index dielectric materials conforming to homogeneous and hybrid nanoparticle dimers. Our results demonstrate that moderate-refractive-index dielectric nanoparticles can enhance the photoluminescence signal of quantum emitters due to their broader electric and magnetic dipolar resonances compared to high-refractive-index dielectric nanoparticles. However, hybrid combinations of metallic and high-refractive-index dielectric nanostructures offer the largest intensity enhancement and Purcell factors at the excitation and emission wavelengths of the quantum emitter, respectively. The results of this work may find applications in the development of single-photon sources.