This study aimed to use gold nanorods and graphene as key materials to fabricate high-performance substrates for the detection of pesticides by surface enhanced Raman spectroscopy (SERS). Three types of pesticides (azinphos-methyl, carbaryl, and phosmet) were selected. Gold nanorods have great potential to be used as a SERS substrate because it is easy to tune the surface plasmon resonance of the nanorods to the laser excitation wavelength of Raman spectroscopy. Graphene is a promising nanoscale material that can be used for supporting metal nanostructures. Three types of novel SERS substrates were fabricated, including graphene-gold film-gold nanorod (G-Au-AuNR) substrate, gold film-gold nanorod (Au-AuNR) substrate, and graphene coupled with gold nanorods (G-AuNR). The results demonstrate that G-Au-AuNR substrates exhibited the strongest Raman signals of the selected pesticides, followed by the Au-AuNR substrates. G-AuNR exhibited the weakest Raman signals, and no characteristic spectral features of the analytes were obtained. A partial least-squares method was used to develop quantitative models for the analysis of spectral data (R = 0.94, 0.87, and 0.86 for azinphos-methyl, carbaryl, and phosmet, respectively). The G-Au-AuNRs substrate was able to detect all three types of pesticides at the parts per million level with limits of detection at around 5, 5, and 9 ppm for azinphos-methyl, carbaryl, and phosmet, respectively. These results indicate that combining gold nanorods and graphene has great potential in the fabrication of sensitive, lightweight, and flexible substrates for SERS applications to improve food safety.