The evaluation of surface quality after milling still represents a challenge due to the stratified structure of wood. In this paper, the surface quality of beech wood obtained by profile milling was analyzed differentiating between tool marks, fuzzy grain, and accidental surface gaps overlapped onto the deep anatomical cavities. The samples were milled at two rotation speeds, 3308 rpm (n1) and 6594 rpm (n2), and two feed speeds of 6.53 m/min (vf1) and 23.74 m/min (vf2). After the samples were cut, approximately 600 m of beech wood were further processed before measuring the surface quality again. The surface waviness measured by Wa increased more than double when the feed speed increased from vf1 to vf2. Rk increased with the feed speed (with the feed per tooth, fz, for a constant n, respectively) for n2 by a significant 57%. An increase in the rotation speed from n1 to n2 has reduced Rk, but the effect was not significant. The processing quality, Rk, measured at different depths did not differ statistically, in spite of the local differences in the cutting speed. The moment of measuring the surface roughness, such as immediately after sharpening or after a working period, influenced the surface quality. After the tool processed 600 m of beech material, the surface quality improved by 30%. The statistical analysis showed that the most important factor affecting the surface quality was the feed speed (implicitly fz, for constant n and number of cutters).