The hot rolling of steel sheets is a highly energy-intensive process. There are technical and operational issues associated with this process, and the causes of these issues can be various. This study involved analysis of one issue that has a great influence on the resulting surface quality of rolled sheet metal: rolled foreign material. After the sheet cools, rolled foreign material tends to fall off and a hole then remain on the surface of the sheet. This paper focuses on the search for the root causes of the occurrence of foreign material rolling. The basic categorization of the causes of this issue was performed by experienced long-term operators. The 4M method (man, machine, method, and material) was used to categorize the causes. Pairwise comparison was used to verify the result. Using energy dispersive spectroscopy analysis, the origin of the foreign material was identified. The analysis confirmed that the foreign material was not derived from the primary material. Further research showed that the cause of the issue was the guide rulers, which are a structural part of the rolling mill. Measures were taken to significantly reduce the incidence of the problem, which also had the effect of reducing financial losses, which fell by a third in 18 weeks. In the future, it will be necessary to make design changes (modernization of the rolling mill), which will, however, require more financial investment.