Phenolic compounds are natural substances that can be obtained from plants. Many of them are potent growth inhibitors of foodborne pathogenic microorganisms, however, phenolic activities against spoilage yeasts are rarely studied. In this study, planktonic and biofilm growth, and the adhesion capacity of Pichia anomala, Saccharomyces cerevisiae, Schizosaccharomyces pombe and Debaryomyces hansenii spoilage yeasts were investigated in the presence of hydroxybenzoic acid, hydroxycinnamic acid, stilbene, flavonoid and phenolic aldehyde compounds. The results showed significant anti-yeast properties for many phenolics. Among the tested molecules, cinnamic acid and vanillin exhibited the highest antimicrobial activity with minimum inhibitory concentration (MIC) values from 500 µg/mL to 2 mg/mL. Quercetin, (−)-epicatechin, resveratrol, 4-hydroxybenzaldehyde, p-coumaric acid and ferulic acid were also efficient growth inhibitors for certain yeasts with a MIC of 2 mg/mL. The D. hansenii, P. anomala and S. pombe biofilms were the most sensitive to the phenolics, while the S. cerevisiae biofilm was quite resistant against the activity of the compounds. Fluorescence microscopy revealed disrupted biofilm matrix on glass surfaces in the presence of certain phenolics. Highest antiadhesion activity was registered for cinnamic acid with inhibition effects between 48% and 91%. The active phenolics can be natural interventions against food-contaminating yeasts in future preservative developments.