Aerial laser scanning or photogrammetric point clouds are often noisy at building boundaries. In order to produce regularized polygons from such noisy point clouds, this study proposes a hierarchical regularization method for the boundary points. Beginning with detected planar structures from raw point clouds, two stages of regularization are employed. In the first stage, the boundary points of an individual plane are consolidated locally by shifting them along their refined normal vector to resist noise, and then grouped into piecewise smooth segments. In the second stage, global regularities among different segments from different planes are softly enforced through a labeling process, in which the same label represents parallel or orthogonal segments. This is formulated as a Markov random field and solved efficiently via graph cut. The performance of the proposed method is evaluated for extracting 2D footprints and 3D polygons of buildings in metropolitan area. The results reveal that the proposed method is superior to the state-of-art methods both qualitatively and quantitatively in compactness. The simplified polygons could fit the original boundary points with an average residuals of 0.2 m, and in the meantime reduce up to 90% complexities of the edges. The satisfactory performances of the proposed method show a promising potential for 3D reconstruction of polygonal models from noisy point clouds.industrial standard datasets for various applications [10,11], including visualization, spatial analysis, urban planning, and navigation [12][13][14][15]. Although the extraction of points on the exterior building boundaries is gracefully handled by a standard convex hull or alpha-shapes for non-convex boundaries, simplification and regularization of the noisy boundaries are still non-trivial tasks and well-established cartographic algorithms do not produce acceptable results when applied to point cloud datasets.The inherent deficiencies of point clouds data, such as data anisotropy, insufficient sampling, and especially noise, make it challenging to retrieve compact 2D/3D polygons of buildings that have satisfactory geometric quality. For ALS data, which have high altimetric accuracy but relatively low point density [16], the initial boundary points of building plans are often jagged [17] and small structures are not well sampled. Meanwhile, DIM point clouds of aerial oblique images are generally inferior to those created from laser scanning in terms of noise level and the preservation of sharp features [9]. The forward intersected point clouds suffer from inaccurate positioning at the edge of building planes due to disparity discontinuities, and thus sharp features may degenerate at corners.In the boundary simplification process, traditional edge collapse-based methods are likely to eliminate sharp features, whereas regularization methods that adopt the Manhattan rule based on the dominant orientation tend to be too strict in many real-world applications, leading to large distortions when applied to polygons with ...