Ion beam erosion can be used as a process for achieving surface smoothing at microscopic length scales and for the preparation of ultrasmooth surfaces, as an alternative to nanostructuring of various surfaces via self-organization. This requires that in the evolution of the surface topography different relaxation mechanisms dominate over the roughening, and smoothing of initially rough surfaces can occur. This contribution focuses on the basic mechanisms as well as potential applications of surface smoothing using low energy ion beams. In the first part, the fundamentals for the smoothing of III/V semiconductors, Si and quartz glass surfaces using low energy ion beams (ion energy: ≤2000 eV) are reviewed using examples. The topography evolution of these surfaces with respect to different process parameters (ion energy, ion incidence angle, erosion time, sample rotation) has been investigated. On the basis of the time evolution of different roughness parameters, the relevant surface relaxation mechanisms responsible for surface smoothing are discussed. In this context, physical constraints as regards the effectiveness of surface smoothing by direct ion bombardment will also be addressed and furthermore ion beam assisted smoothing techniques are introduced. In the second application-orientated part, recent technological developments related to ion beam assisted smoothing of optically relevant surfaces are summarized. It will be demonstrated that smoothing by direct ion bombardment in combination with the use of sacrificial smoothing layers and the utilization of appropriate broad beam ion sources enables the polishing of various technologically important surfaces down to 0.1 nm root mean square roughness level, showing great promise for large area surface processing. Specific examples are given for ion beam smoothing of different optical surfaces, especially for substrates used for advanced optical applications (e.g., in x-ray optics and components for extreme ultraviolet lithography).