Advancements in astronomical telescopes and cutting-edge technologies, including deep ultraviolet (DUV) and extreme ultraviolet (EUV) lithography, have escalated demands and imposed stringent surface quality requirements on optical system components. Achieving near-ideal optical components requires ultra-smooth surfaces with sub-nanometer roughness, no sub-surface damage, minimal surface defects, low residual stresses, and intact lattice integrity. This necessity has driven the rapid development and diversification of ultra-smooth surface fabrication technologies. This paper summarizes recent advances in ultra-smooth surface processing technologies, categorized by their material removal mechanisms. A subsequent comparative analysis evaluates the roughness and polishing characteristics of ultra-smooth surfaces processed on various materials, including fused silica, monocrystalline silicon, silicon carbide, and sapphire. To maximize each process’s advantages and achieve higher-quality surfaces, the paper discusses tailored processing methods and iterations for different materials. Finally, the paper anticipates future development trends in response to current challenges in ultra-smooth surface processing technology, providing a systematic reference for the study of the production of large-sized freeform surfaces.