Flexibility and simple tooling make the incremental sheet forming (ISF) a great process to create complex shapes from mild steel sheets. It is a significant issue to reduce the surface roughness (SR) which is a weakness in the manufacturing of the mild steel parts in ISF process. SR has an adverse effect on the esthetic aspects of the mild steel products or their painting appearance. The purpose of this study is to investigate the effects of the ISF process parameters on the SR of the mild steel sheets. Feed rate, tool diameter, vertical step and spindle speed are chosen as four input variables in the experimental tests. Taguchi design of experiment and the analysis of variance(ANOVA) are used to optimize the SR by investigating the parameters effects and their interactions. According to the obtained results, the vertical step reduction and increase in tool diameter, decrease the roughness on the surface of the mild steel sheets during the single-point incremental forming. In addition, the tool speed (rotation and feed) has little effect on the SR. The results of a validation test demonstrate that the Taguchi technique and the ANOVA, effectively optimize the level of each variable to ensure the best SR.