The sway frequency is an important component of the dynamic characteristics of trees. Video-based methods can be used to measure the sway frequencies of trees. The key to successfully measuring tree sway frequency using video methods lies in whether the tracking method employed is appropriate. Based on six algorithms, i.e., Boosting, TLD, MIL, KCF, MOSSE and CSR-DCF, the tracking performance and accuracy of tree sway frequency measurements were investigated under two conditions: artificial excitation and environmental excitation. The results show that the following: (1) In terms of the tracking speed of tree sway, MOSSE > KCF > CSR-DCF > Boosting > MIL > TLD. (2) The TLD algorithm is not suitable for tree sway tracking. Boosting, MIL, MOSSE, KCF and CSR-DCF can be used for tree sway tracking. (3) Boosting, MIL and MOSSE are suitable for measuring the sway frequency of artificially excited branches and environmentally excited trees. (4) KCF and CSR-DCF algorithms are not suitable for the measurement of branch sway frequency under artificial excitation conditions but can be used for the measurement of tree sway frequency under environmental excitation conditions. However, it should be noted that this experiment only takes a Betula platyphylla Sukaczev tree and a Betula platyphylla Sukaczev branch as the research object to verify the effectiveness and feasibility of each tracking method, and does not verify the generalization ability of the above methods (on multiple tree species and multiple trees).