The article considers an approach to combine the FEM method with the machine learning method in modeling the process of sheet metal cooling in a laminar cooling unit. A model of rolled products cooling based on the FEM has been developed; it considers the variable properties of the material and phase transformations. The high importance of taking into account physical processes, which occur on the surface of rolled products during cooling, is shown, namely influence of the surface temperature of rolled surface on the heat transfer coefficient. In first iteration data from literature was used for this dependenсe, afterwards it was adapted for the concrete case using iteration method. Especial importance of this phenomenon for calculation of cooling processes for rolled heavy plates (with thickness more than 30 mm) is shown. Two ways to calculate heat dissipation from phase transformations based on the Avrami formula and using the curve of relationship between heat capacity and temperature are given; they are used in a model depending on availability of data for the examined steel grade. Heat transfer coefficient was determined using machine learning methods in order to increase accuracy of calculations. The training set was built on the basis of industrial data, cleared from serial production factor and errors in sensors data signals. Several machine learning models were examined, the model based on gradient boosting of the catboost library displayed the best results. The optimal model parameters were selected using the GridSearchCV method of the Sklearn library or other builtin methods. The most important factors (feature importance) were those that provide especial influence on the heat transfer coefficient -water flow, thickness of rolled products, temperature range of cooling, chemical composition of steel.