Forests, as the main body of the terrestrial ecosystem, have long been focal points for accurate structural parameter extraction. Among these parameters, tree height is a fundamental measurement factor that plays an important role in monitoring forest structure and biomass. The emergence of unmanned aerial vehicle light detection and ranging (UAV-LiDAR) technology has provided a strong guarantee of the acquisition of forest tree height parameters. However, UAV-LiDAR point cloud data have problems such as a large volume and data redundancy, and different point cloud data processing methods have different effects. Based on voxel filtering (VF) and statistical outlier removal (SOR)point cloud data processing experimental analysis, this study explored the influence of different filtering methods on the forest tree height inversion efficiency and accuracy. First, the point cloud data processed by VF is significantly better than that of SOR in terms of point cloud number, file size, running time, etc. The number of point clouds for VF decreased by an average of 96.91% compared with the original point clouds. Second, the VF tree height inversion accuracy was better than the tree height inversion data using SOR. The average accuracy of VF was 96.24%, while that of SOR was 94.17%. In summary, VF can effectively reduce data redundancy and improve tree height inversion accuracy.