Histamine produced via the secretion of histidine decarboxylase by the bacteria in fish muscles is a toxic biogenic amine and of significant concern in food hygiene, since a high intake can cause poisoning in humans. This study proposed a fluorometric and colorimetric dual-mode specific method for the detection of histamine in fish, based on the fluorescence labeling of a histamine specific aptamer via the quenching and optical properties of gold nanoparticles (AuNPs). Due to the fluorescence resonance energy transfer phenomenon caused by the proximity of AuNPs and NaYF4:Ce/Tb, resulting in the quenching of the fluorescence signal in the detection system, the presence of histamine will compete with AuNPs to capture the aptamer and release it from the AuNP surface, inducing fluorescence recovery. Meanwhile, the combined detection of the two modes showed good linearity with histamine concentration, the linear detection range of the dual-mode synthesis was 0.2–1.0 μmol/L, with a detection limit of 4.57 nmol/L. Thus, this method has good selectivity and was successfully applied to the detection of histamine in fish foodstuffs with the recoveries of 83.39~102.027% and 82.19~105.94% for Trichiurus haumela and Thamnaconus septentrionalis, respectively. In addition, this method was shown to be simple, rapid, and easy to conduct. Through the mutual verification and combined use of the two modes, a highly sensitive, rapid, and accurate dual-mode detection method for the analysis of histamine content in food was established, thereby providing a reference for the monitoring of food freshness.