Chinese herbal polysaccharides (CHPs) are natural polymers composed of monosaccharides, which are widely found in Chinese herbs and work as one of the important active ingredients. Its biological activity is attributed to its complex chemical structure with diverse spatial conformations. However, the structural elucidation is the foundation but a bottleneck problem because the majority of CHPs are heteropolysaccharides with more complex structures. Similarly, the studies on the relationship between structure and function of CHPs are even more scarce. Therefore, this review summarizes the structure-activity relationship of CHPs. Meanwhile, we reviewed the structural elucidation strategies and some new progress especially in the advanced structural analysis methods. The characteristics and applicable scopes of various methods are compared to provide reference for selecting the most efficient method and developing new hyphenated techniques. Additionally, the principle structural modification methods of CHPs and their effects on activity are summarized. The shortcomings, potential breakthroughs, and developing directions of the study of CHPs are discussed. We hope to provide a reference for further research and promote the application of CHPs.