Aboveground biomass (AGB) and leaf area index (LAI) are important indicators to measure crop growth and development. Rapid estimation of AGB and LAI is of great significance for monitoring crop growth and agricultural site-specific management decision-making. As a fast and non-destructive detection method, unmanned aerial vehicle (UAV)-based imaging technologies provide a new way for crop growth monitoring. This study is aimed at exploring the feasibility of estimating AGB and LAI of mung bean and red bean in tea plantations by using UAV multispectral image data. The spectral parameters with high correlation with growth parameters were selected using correlation analysis. It was found that the red and near-infrared bands were sensitive bands for LAI and AGB. In addition, this study compared the performance of five machine learning methods in estimating AGB and LAI. The results showed that the support vector machine (SVM) and backpropagation neural network (BPNN) models, which can simulate non-linear relationships, had higher accuracy in estimating AGB and LAI compared with simple linear regression (LR), stepwise multiple linear regression (SMLR), and partial least-squares regression (PLSR) models. Moreover, the SVM models were better than other models in terms of fitting, consistency, and estimation accuracy, which provides higher performance for AGB (red bean: R2 = 0.811, root-mean-square error (RMSE) = 0.137 kg/m2, normalized RMSE (NRMSE) = 0.134; mung bean: R2 = 0.751, RMSE = 0.078 kg/m2, NRMSE = 0.100) and LAI (red bean: R2 = 0.649, RMSE = 0.36, NRMSE = 0.123; mung bean: R2 = 0.706, RMSE = 0.225, NRMSE = 0.081) estimation. Therefore, the crop growth parameters can be estimated quickly and accurately using the models established by combining the crop spectral information obtained by the UAV multispectral system using the SVM method. The results of this study provide valuable practical guidelines for site-specific tea plantations and the improvement of their ecological and environmental benefits.