The excessive use of pesticides and drugs, coupled with environmental pollution, has resulted in the persistence of contaminants on food. These pollutants tend to accumulate in humans through the food chain, posing a significant threat to human health. Therefore, it is crucial to develop rapid, low-cost, portable, and on-site biosensors for detecting food contaminants. Among various biosensors, polymer-based biosensors have emerged as promising probes for detection of food contaminants in recent years, due to their various functions such as target binding, enrichment, and simple signal reading. This paper aims to discuss the characteristics of five types of food pollutants—heavy metals, pesticide residues, pathogenic bacteria, allergens, and antibiotics—and their adverse effects on human health. Additionally, this paper focuses on the principle of polymer-based biosensors and their latest applications in detecting these five types of food contaminants in actual food samples. Furthermore, this review briefly examines the future prospects and challenges of biosensors for food safety detection. The insights provided in this review will facilitate the development of biosensors for food safety detection.