Malathion, phoxim, and thiram are organophosphates and organosulfur pesticides widely used in agricultural products. The residues of these pesticides present a direct threat to human health. Rapid and on-site detection is critical for minimizing such risks. In this work, a simple approach was introduced using a flexible surface-enhanced Raman spectroscopy (SERS) substrate. The prepared Ag nanoparticles-polydimethylsiloxane (AgNPs-PDMS) substrate showed high SERS activity, good precision (relative standard deviation = 5.33%), and stability (30 days) after optimization. For target pesticides, the linear relationship between characteristic SERS bands and concentrations were achieved in the range of 10~1000, 100~5000, and 50~5000 μg L−1 with LODs down to 3.62, 41.46, and 15.69 μg L−1 for thiram, malathion, and phoxim, respectively. Moreover, SERS spectra of mixed samples indicated that three pesticides can be identified simultaneously, with recovery rates between 96.5 ± 3.3% and 118.9 ± 2.4%, thus providing an ideal platform for detecting more than one target. Pesticide residues on orange surfaces can be simply determined through swabbing with the flexible substrate before acquiring the SERS signal. This study demonstrated that the prepared substrate can be used for the rapid detection of pesticides on real samples. Overall, this method greatly simplified the pre-treatment procedure, thus serving as a promising analytical tool for rapid and nondestructive screening of malathion, phoxim, and thiram on various agricultural products.