Extraction of vectorized building outlines from high-resolution remote sensing images is highly useful for various application fields, such as map creation and urban planning. However, this process is often complicated by external factors, such as trees and shadows, which cause issues, such as excessive node redundancy, jagged lines, and unclear corner points. In this study, a multilevel regularization method was designed for building outlines, including the “overall–local–detail” levels. First, overall regularization was performed by combining the minimum bounding rectangle of the building outline with the Hausdorff distance method. Next, based on the convex hull of the building outline and the distribution characteristics of nodes along the outline, the building outline was divided into multiple line chains and classified for local regularization. Finally, the details of the building outline were processed, with the parallel and perpendicular characteristics enhanced to obtain the final regularization results. The experimental results showed that the proposed method effectively enhances the edge representation accuracy of building outlines and significantly improves the accuracy and regularity of building edges. Furthermore, it strengthens the orthogonal characteristics of building outlines, providing more accurate representations of true building outlines.