The machining of the electron beam melting (EBM) produced parts is a challenging task because, upon machining, different part orientations (EBM layers’ orientations) produce different surface quality even when the same machining parameters are employed. In this paper, the EBM fabricated parts are machined in three possible orientations with regard to the tool feed direction, where the three orientations are “tool movement in a layer plane” (TILP), “tool movement perpendicular to layer planes” (TLP), and “tool movement parallel to layers planes” (TPLP). The influence of the feed rate, radial depth of cut, and cutting speed is studied on surface roughness, cutting force, micro-hardness, microstructure, chip morphology, and surface morphology of Ti6Al4V, while considering the EBM part orientations. It was found that different orientations have different effects on the machined surface during milling. The results show that the EBM parts can achieve good surface quality and surface integrity when milled along the TLP orientation. For instance, surface roughness (Sa) can be improved up to 29% when the milling tool is fed along the TLP orientation compared to the other orientations (TILP and TPLP). Furthermore, surface morphology significantly improves with lower micro-pits, redeposited chips, and feed marks in case of the TLP orientation.