This study investigated phytoplankton and water environmental factors in Yazhou Bay, South China Sea, during the winter, spring, and summer of 2023. It examined phytoplankton community structure, subgroup heterogeneity, and key environmental drivers. Phytoplankton abundance ranged from 0.08 to 14.30 × 10⁴ cells·L−1, with high concentrations in estuary and nearshore zones. In summer, currents carry phytoplankton offshore, with stratification leading to high sedimentation in southern offshore waters. RDA results indicated that in winter and spring, inorganic nitrogen mainly influences phytoplankton distribution, while silicate is the primary factor in summer. Although seasonal differences in total phytoplankton abundance are minimal, significant horizontal and vertical distribution variations exist. Diverse preferences of different phytoplankton species for temperature, salinity, nitrogen, and phosphorus result in high species diversity. The Shannon–Wiener diversity index (H′) averages 3.96 ± 0.09, and the Pielou evenness index (J) averages 0.82 ± 0.01. Dominant species include Pseudo-nitzschia pungens, Skeletonema costatum, and Rhizosolenia sinica. Influenced by external oceanic water masses, estuary input, and islands, phytoplankton subgroups show regional and seasonal variations. Despite recorded harmful algal blooms (HABs) in adjacent waters, Yazhou Bay’s high biodiversity and low cell density suggest a low HAB risk, though future risks due to climate change and human activities remain.