The vocal structure of gibbons has long been believed to be under strong genetic control and consistent with phylogeny. Gibbon vocalizations show hierarchical structures and song coordination behaviors, which may have different functions and convey distinct information compared to basic notes. However, previous studies have not compared vocalizations at multiple levels across species. We compared the vocal structures of three Nomascus species in the wild in South China, namely the western black gibbon (Nomascus concolor), Cao vit gibbon (N. nasutus), and Hainan gibbon (N. hainanus) at the levels of note (six features of four types of notes), male sequence (eight features), and song coordination (four features of the overlap pattern between male and female songs). We selected 15 recordings for each species: N. concolor (six groups, 2006 to 2021), N. nasutus (seven groups, 2008 to 2021), and N. hainanus (six groups, 2020 to 2021). We used permuted random forest analysis to test for species differences in vocal structure and the contribution of acoustic features to species differences at each level. We used acoustic features to construct a hierarchical clustering tree, and compared it with phylogenetic relationships. We found significant differences at all levels except the boom note. Acoustic similarity did not match genetic similarity in boom and pre-modulated notes, suggesting that genetic differences are not the only factor leading to species differences in vocal structures. We also found different contributions of acoustic features to differentiation in vocal structure at different levels. Our study suggests that we need to compare gibbon vocalizations at different levels to understand their differentiation and evolution.