The frequent occurrence of geological hazards will not only cause peoples' property loss and deterioration of living environments, but will also endanger peoples' lives. Therefore, rapid and accurate evaluation of geological hazards susceptibility can provide an important scientific basis for emergency rescue and disaster reduction and prevention. In this paper, ten effective variables including slope, aspect, curvature, normalized differential vegetation index, annual precipitation, strata lithology, tectonic complexity, residential density, road network density, and land use/land cover were selected as evaluation indexes. Meanwhile, random forest (RF) model is improved by the optimization of unbalanced geological hazards dataset, differentiation of continuous geological hazards evaluation factors, sample similarity calculation, and iterative method for finding optimal random characteristics by calculating out-of-bagger errors. The geological hazards susceptibility evaluation model based on optimized RF (OPRF) was established and used to assess the susceptibility level of geological hazards for Lingyun County. Then, receiver operating characteristics (ROC) curves and field investigation were performed to verify the efficiency for five models. Analysis and comparison of the results denoted that the model based on OPRF has the highest prediction accuracy of 93.4%, which is far better than the other four models. Furthermore, the evaluation results can provide reference for geological hazards prediction and prevention, and can also provide decision support for land use development and rational utilization of resources and environment in Lingyun County. Based on these results, the OPRF model could be extended to other regions with similar geological environment backgrounds for geological hazards susceptibility assessment and prediction.