The Open-Source Digital Elevation Model (DEM) is fundamental data of the geoscientific community. However, the variation of its accuracy with land cover type and topography has not been thoroughly studied. This study evaluates the accuracy of five globally covered and open-accessed DEM products (TanDEM-X90 m, SRTEM, NASADEM, ASTER GDEM, and AW3D30) in the mountain area using ICESat/GLAS data as the GCPs. The robust evaluation indicators were utilized to compare the five DEMs’ accuracy and explore the relationship between these errors and slope, aspect, landcover types, and vegetation coverage, thereby revealing the consistency differences in DEM quality under different geographical feature conditions. The Taguchi method is introduced to quantify the impact of these surface characteristics on DEM errors. The results show that the slope is the main factor affecting the accuracy of DEM products, accounting for about 90%, 81%, 85%, 83%, and 65% for TanDEM-X90, SRTM, NASADEM, ASTER GDEM, and AW3D30, respectively. TanDEM-X90 has the highest accuracy in very flat areas (slope < 2°), NASADEM and SRTM have the greatest accuracy in flat areas (2 ≤ slope < 5°), while AW3D30 accuracy is the best in other cases and shows the best consistency on slopes. This study makes a new attempt to quantify the factors affecting the accuracy of DEM, and the results can guide the selection of open-source DEMs in related geoscience research.