Fish is one of the highly demanded aquatic products, and its quality and safety play a pivotal role in daily diet. However, the possible hazardous substance in perishable fish both in pre-and postharvest periods may decrease their values and pose a threat to public health. Laborious and expensive traditional methods drive the need of developing effective tools for detecting fish quality and safety properties in a rapid, nondestructive, and effective manner. Recent advances in Raman spectroscopy (RS) and surface-enhanced Raman scattering (SERS) have shown enormous potential in various aspects, which largely boost their applications in fish quality and safety evaluation. They have incomparable merits such as providing molecule fingerprint information and allowing for rapid, sensitive, and noninvasive detection with simple sample preparation. This review provides a comprehensive overview focusing on the applications of RS and SERS for fish quality assessment and safety inspection, highlighting the hazardous substance and illegal behavior both in preharvest (veterinary drug residues and environmental pollutants) and postharvest (freshness and illegal behavior) particularly. Moreover, challenges and prospects are also proposed to facilitate the vigorous development of RS and SERS. This review is aimed to emphasize potential opportunities for applying RS and SERS as promising techniques for routine food quality and safety detection.