Surface‐enhanced Raman scattering (SERS) spectroscopy has renowned its fame for the ultra‐high sensitivity and single‐molecule detection ability, and listed as a fingerprint spectrum representative in various trace detection fields. Considerable efforts have been made by researchers to design high‐sensitive SERS‐active substrates ranging from noble metals to semiconductors. This review summarizes the fundamental theories for SERS technique, that is, the electromagnetic enhancement mechanism and chemical enhancement mechanism and the state‐of‐the‐art design strategies for noble metal and semiconductor substrates. It also sheds light on the effective approaches to improve the SERS activity for noble metal substrates, that is, tuning the localized surface plasmon resonance position, the assembling of hot spots, and precise controlling of nanogaps. Although charge transfer is considered as the main reason for the enhancement mechanism for semiconductors at the present stage, the underlying theoretical basis remains mysterious. This review summarized the critical points for SERS‐active substrates design and prospected the future development direction of SERS technology.