Identifying the trade-offs and synergies (TOSs) of ecosystem services (ESs) and their responses to landscape patterns at various scales, especially in mountainous areas, could benefit the strategies of ES management and landscape optimization. In this study, the southern Zhejiang Province, a hilly region in eastern China, was chosen as the study area. Five ESs, including food production (FP), carbon sequestration (CS), flood mitigation (FM), water conservation (WC), and soil retention (SR) in 2020 were quantified. The TOSs of these ESs were identified at four spatial scales (i.e., grid, watershed, town, and county scales) through Pearson correlation analysis and the spatial overlay method. The effects of landscape patterns on the TOSs of ESs were analyzed by applying a logistic regression model. Results showed that FP and other ESs were trade-offs, while the other ES pairs were synergies. Spatial overlay results showed that weak synergies increased significantly, while strong synergies decreased significantly with the increase of the scale. The direction of the influence of landscape pattern on TOSs did not change, but the magnitudes of the impacts were scale-dependent. Landscape composition (i.e., cropland%, forest%, construction land%) had more significant effects on the trade-offs of ESs than spatial configuration (i.e., LSI, PD, COHE, and SHDI). The magnitudes of impact of landscape composition were strengthened at larger scales, while the effects of landscape configuration on the TOSs of ESs became complex as the scale changed. The results of this study could contribute to understanding how landscape patterns affect TOSs across scales, which will promote the hierarchical governance of ESs in mountainous areas.