In this study, we developed a colloidal gold immunochromatographic strip (CGIS) method that used the matrix-matched calibration curves of contamination ratio models to quantitatively determine the total aflatoxin in five herbal medicines. This approach addresses issues related to false results and poor accuracy associated with conventional methods. The CGIS was analyzed using a Vertu touch reader, and the matrix-matched calibration was established based on the absorbance ratios of the T and C lines, as well as the logarithmic values of the total aflatoxin concentrations. The total aflatoxins could be accurately and digitally detected from 2.5 to 40 μg/kg, and the LOD of total aflatoxins was 1 μg/kg in the five herbal medicines. The recovery rates from the spiked samples ranged from 65.1% to 98.6%, and the RSD was less than 16.9%. A total of 229 samples were analyzed by both CGIS and HPLC-FLD, with agreement ranging from 78.4% to 132.6% (Arecae semen), 82.6% to 133.0% (Nelumbinis semen), 79.9% to 117.9% (Coicis semen), 78.1% to 119.0% (Platycladi semen), and 76.1% to 123.0% (Ziziphi spinosae semen). This process for the discrimination of the CGIS results was established to assess if samples met the requirement of aflatoxin limits, which could save approximately 75% in time and reduce the workload of retesting by a designated confirmatory reference method to less than 10%. This study demonstrated that the application of matrix-matched calibration curves based on contamination ratio models to CGIS can effectively enhance the rapid quantitative determination capability of total aflatoxins in herbal medicine matrices.