Azo compounds such as the Sudan dyes I–IV are frequently used illegally as colorants and added to a wide range of foods. These compounds have been linked to a number of food safety hazards. Several methods have been proposed to detect food contamination by azo compounds and most of these are laboratory based; however, the development of reliable and portable methods for the detection and quantification of food contaminated by these chemicals in low concentration is still needed due to their potentially carcinogenic properties. In this study, we investigated the ability of surface enhanced Raman scattering (SERS) combined with chemometrics to quantify Sudan I–IV dyes. SERS spectra were acquired using a portable Raman device and gold nanoparticles were employed as the SERS substrate. As these dyes are hydrophobic, they were first dissolved in water: acetonitrile (1:10, v/v) as single Sudan dyes (I–IV) at varying concentrations. SERS was performed at 785 nm and the spectra were analyzed by using partial least squares regression (PLS-R) with double cross-validations. The coefficient of determination (Q2) were 0.9286, 0.9206, 0.8676 and 0.9705 for Sudan I to IV, respectively; the corresponding limits of detection (LOD) for these dyes were estimated to be 6.27 × 10−6, 5.35 × 10−5, 9.40 × 10−6 and 1.84 × 10−6 M. Next, quadruplex mixtures were made containing all four Sudan dyes. As the number of possible combinations needed to cover the full concentration range at 5% intervals would have meant collecting SERS spectra from 194,481 samples (214 combinations) we used a sustainable solution based on Latin hypercubic sampling and reduced the number of mixtures to be analyzed to just 90. After collecting SERS spectra from these mixture PLS-R models with bootstrapping validations were employed. The results were slightly worse in which the Q2 for Sudan I to IV were 0.8593, 0.7255, 0.5207 and 0.5940 when PLS1 models (i.e., one model for one dye) was employed and they changed to 0.8329, 0.7288, 0.5032 and 0.5459 when PLS2 models were employed (i.e., four dyes were modelled simultaneously). These results showed the potential of SERS to be used as a high-throughput, low-cost, and reliable methods for detecting and quantifying multiple Sudan dyes in low concentration from illegally adulterated samples.