Forest canopy structure is very important to measure forest change and forest coverage. And TerraSAR-X data are well suited for inversion applications at tree height. Based on the Random Volume over Ground model, the three-stage algorithm and its PSO improvement are studied in this paper. Taking the TerraSAR-X data of Mengla County in Yunnan Province China as the data source, the forest height inversion algorithm were compared in the experiment part. Finally, the results are verified with the field measured data. The results show that the precision of forest height inversion based on the PSO intelligent algorithm is better than the traditional three-stage algorithm, and the correlation coefficient is improved by more than 20%.