Building a high-resolution Digital Elevation Model (DEM) in urban areas has been a challenging task so far because of the complicated and trivial topographic character caused by dense buildings, dense artificial vegetation, such as deep manual intervention. The paper presents a heterogeneous urban DEM and modeling method based on a large-scale digital topographic map, Digital Surface Model (DSM) generated by popular UAV photogrammetry or laser radar point cloud data. The method is first to extract elevation points, contour lines, road edges, steep slopes, building outlines, retaining walls, steps, and such topographic linear elements as possible from a large-scale topographic map; then to merge elevation information from DEM, DSM, point clouds data to generate a series of 3d-lines or polygons with elevation; even to struct some necessary 3D auxiliary lines according to geometric features to some abrupt terrain features; finally to create a Triangulated Irregular Network (TIN), which is able to accurately represents the terrain, using constrained TIN create function of ArcGIS. And evaluation precision was not uniform in the data bound because of multi-source data with different precision, so it was called Heterogeneous DEM(HDEM). The experiment showed that the HDEM has high consistency with the topographic map and is more suitable for flood analysis, urban fine management, real three-dimensional construction, etc.