Tea is the second most popular nonalcoholic beverage consumed in the world, made from the buds and young leaves of the tea plants (Camellia sinensis). Tea trees, perennial evergreen plants, contain abundant specialized metabolites and suffer from severe herbivore and pathogen attacks in nature. Thus, there has been considerable attention focusing on investigating the precise function of specialized metabolites in plant resistance against pests and diseases. In this review, firstly, the responses of specialized metabolites (including phytohormones, volatile compounds, flavonoids, caffeine, and L-theanine) to different attacks by pests and pathogens were compared. Secondly, research progress on the defensive functions and action modes of specialized metabolites, along with the intrinsic molecular mechanisms in tea plants, was summarized. Finally, the critical questions about specialized metabolites were proposed for better future research on phytohormone-dependent biosynthesis, the characteristics of defense responses to different stresses, and molecular mechanisms. This review provides an update on the biological functions of specialized metabolites of tea plants in defense against two pests and two pathogens.