In mold making, the mold surface roughness directly affects the surface roughness of the produced part. To achieve surface roughness below 0.8 μm, the cost of surface finish is high and time-consuming. One alternative to the different grinding and polishing steps is laser polishing (LP). This study investigates and models the LP of tool steel (X38CrMoV5-1-DIN 1.2343), typical for the mold industry, having an initial rough surface obtained by electrical discharge machining. The microstructures of the re-melted layer and heat-affected zone due to the LP process were also studied. Four parameters: the laser spot size, velocity, maximum melt pool temperature and overlapping were investigated via a design of experiments (DoE) approach, specifically a factorial design. The responses were line roughness (Ra), surface roughness (Sa), and waviness (Wa). The surface topography was measured before and after the LP process by white light profilometer or confocal microscopy. DoE results showed that the selected factors interact in a complex manner, including the interactions, and depend on the responses. The DoE analysis of the results revealed that the roughness is mainly affected by the velocity, temperature and overlap. Based on a first DoE model, an optimization of the parameters was performed and allowed to find optimum parameters for the LP of the rough samples. The optimum conditions to minimize the roughness are a spot size of 0.9 mm, a velocity of 50 mm/s, a temperature of 2080 °C and an overlap of 90%. By using these parameters, the roughness could be reduced by a factor of almost 8 from 3.8 µm to approximately 0.5 µm. Observations of the microstructure reveal that the re-melted layer consists of columnar grains of residual austenite. This can be explained by the carbon intake of the electro-machined surface that helps stabilize the austenitic phase.