The micro-topographic quality of a product’s surface directly affects the performance of the virtual assembly. The computational precision of the fractal dimension has a significant effect on the accuracy of the virtual assembly model. To address the imprecision and large deviation of the calculation method under some conditions, a virtual product assembly surface modeling method is proposed in this article based on the root mean square of the area residuals. First, the fractal dimension of the surface is calculated using the regional residual root mean square (3R method). The feasibility of the 3R method is verified by comparing with the existing methods on isotropic and anisotropic surfaces. Second, the fractal dimension of a real part surface is obtained using the 3R method. According to the Weierstrass–Mandelbrot function, a partial surface model of the machined part is established. The surface contours, roughness, waviness, and flatness of the part are extracted by wavelet transform. Then, the digital surface model for virtual assembly with the same profile parameters is generated by reconstructing the surface contours of the actual parts. Finally, the method proposed by taking the virtual assembly of a VTM200/F5 turn-milling composite machining center guideway is verified in this article.