Forming solid solutions is one of the most effective strategies to suppress the thermal conductivity of thermoelectric materials. However, the accompanying increase in impurity ion scattering usually results in an undesirable loss in hall mobility, negatively impacting the electrical transport properties. In this work, a tellurium–selenium (Te-Se) solid solution with trace antimony (Sb) doping was synthesized via the high pressure and high temperature method. It was found that slight Se doping into the Te sites not only had no impact on the hall mobility and carrier concentration, but also enhanced the density-of-state effective mass of Sb0.003Te0.997, leading to an enhanced power factor near room temperature. Additionally, the presence of Se doping caused a significant reduction in the phonon thermal conductivity of Te due to fluctuations in the mass and strain field. The lowest phonon thermal conductivity was as low as ~0.42 Wm−1K−1 at 600 K for Sb0.003Se0.025Te0.972, which approached the theoretical minimum value of Te (~0.28 Wm−1K−1). The effects of Se doping suppressed thermal conductivity, while Sb doping enhanced the power factor, resulting in a larger ZT of ~0.94 at 600 K. Moreover, these findings demonstrate that Sb and Se doping can effectively modulate the electrical and thermal transport properties of Te in a synergistic manner, leading to a significant increase in the average ZT across a wide temperature range.