The finite-difference time-domain (FDTD) method was employed to simulate the electric field distribution for noble metal (Au or Ag)/semiconductor (Ge or Si) substrates. The simulation showed that noble metal/Ge had stronger SERS enhancement than noble metal/Si, which was mainly attributed to the different dielectric constants of semiconductors. In order to verify the simulation, Ag nanoparticles with the diameter of ca. 40 nm were grown on Ge or Si wafer (Ag/Ge or Ag/Si) and employed as surface-enhanced Raman scattering substrates to detect analytes in solution. The experiment demonstrated that both the two substrates exhibited excellent performance in the low concentration detection of Rhodamine 6G. Besides, the enhancement factor (1.3 × 109) and relative standard deviation values (less than 11%) of Ag/Ge substrate were both better than those of Ag/Si (2.9 × 107 and less than 15%, respectively), which was consistent with the FDTD simulation. Moreover, Ag nanoparticles were grown in-situ on Ge substrate, which kept the nanoparticles from aggregation in the detection. To data, Ag/Ge substrates showed the best performance for their sensitivity and uniformity among the noble metal/semiconductor ones.