A comparative study of the solid substrates used in surface-enhanced Raman scattering (SERS) based immunoassay is made in this paper. Five different substrates were prepared and divided into two groups with and without SERS activity. They are (1) a poly-L-lysine slide, (2) a glutaraldehyde (GA)-aminosilane slide, (3) a substrate assembled with silver nanoparticles, (4) a substrate assembled with silver nanoparticles and functionalized with GA-aminosilane and (5) a substrate assembled with gold nanoparticles, of which the first two are substrates are without SERS activity and the latter three are with SERS activity because of the existence of the metallic nanoparticles. The SERS experimental results show that the immunoassay performed on an SERS-active substrate is more effective than that employing the inactive substrate. Among the inactive substrates, the GA-aminosilane slide with a better ability for antibody immobilization leads to a more sensitive immunoassay than the poly-L-lysine slide. Moreover, for SERS-based immunoassay, the substrate with assembled silver nanoparticles has an advantage of higher SERS enhancement capacity over the substrate assembled with gold nanoparticles. This work indicates that SERS-active substrates play important and positive roles in sensitive SERS-based immunoassay.