In this paper a ground safety assessment model is introduced based on the probability estimation of possible impact positions when unmanned aerial vehicle (UAV) crashes on the ground. By incorporating the random uncertainties during the descending process, risks associated with UAV’s ground crash are estimated accurately. The number of victims on the ground per flight hour is selected as the indicative index to evaluate the risk levels of the corresponding ground area. We mainly focus on the analysis of uncertainties that usually appear in drag coefficient which would generate a great amount of effects on the travelled horizontal distance from the failure point to the impact point on the ground, which further influences the possible impact positions. The drag force in the air, failure velocity of a UAV, and wind effects in the local area are all considered in the proposed model, as well as ground features, including sheltering effects on the ground, UAV parameter settings, and distribution of local population. Uncertainties in drag force when a UAV descends, UAV’s initial horizontal and vertical speeds at failure point, and local wind patterns are all considered as the indispensable factors in the proposed model. Especially the probability of fatality once hit by the UAV’s debris is explored to make the safety assessment more reliable and valuable. In the end, the actual UAV parameters and official historical weather data are used to estimate the risks in a real operation environment when a failure event happens at a legal flying height. Experimental results are given based on different types of UAVs and random effects in the descent. The results show that all the operations of all kinds of UAVs selected in the validation are so dangerous that the safety of people on the ground cannot be guaranteed, whose value is much bigger than the manned aircraft safety criterion 10−7.