There is ongoing interest in developing remote sensing technology to map and monitor the spatial distribution and carbon stock of mangrove forests. Previous research has demonstrated that the relationship between remote sensing derived parameters and aboveground carbon (AGC) stock varies for different species types. However, the coarse spatial resolution of satellite images has restricted the estimated AGC accuracy, especially at the individual species level. Recently, the availability of unmanned aerial vehicles (UAVs) has provided an operationally efficient approach to map the distribution of species and accurately estimate AGC stock at a fine scale in mangrove areas. In this study, we estimated mangrove AGC in the core area of northern Shenzhen Bay, South China, using four kinds of variables, including species type, canopy height metrics, vegetation indices, and texture features, derived from a low-cost UAV system. Three machine-learning algorithm models, including Random Forest (RF), Support Vector Regression (SVR), and Artificial Neural Network (ANN), were compared in this study, where a 10-fold cross-validation was used to evaluate each model’s effectiveness. The results showed that a model that used all four type of variables, which were based on the RF algorithm, provided better AGC estimates (R2 = 0.81, relative RMSE (rRMSE) = 0.20, relative MAE (rMAE) = 0.14). The average predicted AGC from this model was 93.0 ± 24.3 Mg C ha−1, and the total estimated AGC was 7903.2 Mg for the mangrove forests. The species-based model had better performance than the considered canopy-height-based model for AGC estimation, and mangrove species was the most important variable among all the considered input variables; the mean height (Hmean) the second most important variable. Additionally, the RF algorithms showed better performance in terms of mangrove AGC estimation than the SVR and ANN algorithms. Overall, a low-cost UAV system with a digital camera has the potential to enable satisfactory predictions of AGC in areas of homogenous mangrove forests.