Steel plays an important role in industry, and the surface defect detection for steel products based on machine vision has been widely used during the last two decades. This paper attempts to review state‐of‐art of vision‐based surface defect inspection technology of steel products by investigating about 170 publications. This review covers the overall aspects of vision‐based surface defect inspection for steel products including hardware system, automated vision‐based inspection method, existing problems and latest development. The types of steel product surface defects composition of visual inspection system are briefly described, and image acquisition system is introduced as well. The image processing algorithms for surface defect detection of steel products are reviewed, including image pre‐processing, region of interest (ROI) detection, image segmentation for ROI, feature extraction and selection and defect classification. The important problems such as small sample and real time of steel surface defect detection are discussed. Finally, the challenge and development trend of steel surface defect detection are prospected.