Lightweight alloys made from aluminium are used to manufacture cars, trains and planes. The main parts most often manufactured from thin sheets requiring the use of milling in the manufacturing process are front panels for control systems, housing parts for electrical and electronic components. As a result of the final phase of the manufacturing process, cold rolling, residual stresses remain in the surface layers, which can influence the cutting processes carried out on these materials. The main aim of this study was to verify whether the strategy of removing the outer material layers of aluminium alloy sheets affects the surface roughness after the face milling process. EN AW-6082-T6 aluminium alloy thin plates with three different thicknesses and with two directions relative to the cold rolling process direction (longitudinal and transverse) were analysed. Three different strategies for removing the outer layers of the material by face milling were considered. Noticeable differences in surface roughness 2D and 3D parameters were found among all machining strategies and for both rolling directions, but these differences were not statistically significant. The lowest values of Ra = 0.34 µm were measured for the S#3 strategy, which asymmetrically removed material from both sides of the plate (main and back), for an 8-mm-thick plate in the transverse rolling direction. The highest values of Ra = 0.48 µm were measured for a 6-mm-thick plate milled with the S#2 strategy, which symmetrically removed material from both sides of the plate, in the longitudinal rolling direction. However, the position of the face cutter axis during the machining process was observed to have a significant effect on the surface roughness. A higher surface roughness was measured in the areas of the tool point transition from the up-milling direction to the down-milling direction (tool axis path) for all analysed strategies (Ra = 0.63–0.68 µm). The best values were obtained for the up-milling direction, but in the area of the smooth execution of the process (Ra = 0.26–0.29 µm), not in the area of the blade entry into the material. A similar relationship was obtained for analysed medians of the arithmetic mean height (Sa) and the root-mean-square height (Sq). However, in the case of the S#3 strategy, the spreads of results were the lowest.