Laser welding is particularly relevant in the industry thanks to its simplicity, flexibility and final quality. The industry 4.0 and sustainable manufacturing framework gives massive attention to in situ and non‐destructive inspection methods to predict laser weld final quality. Literature often resorts to supervised Machine Learning approaches. However, selecting the ApTest method is non‐trivial and often decision making relies on diverse and unclearly defined criteria. This work addresses this task by proposing a statistical comparison method based on nonparametric tests. The method is applied to the most relevant supervised Machine Learning approaches exploited in literature to predict laser weld quality, specifically, considering the optimisation of a new production line, hence focussing on supervised Machine Learning methods that do not require massive data set, that is, Generalized Linear Model (GLM), Gaussian Process Regression, Support Vector Machine, Classification and Regression Tree, and Genetic Algorithms. The statistical comparison is carried out to select the best‐performing model, which is then exploited to optimise the production process. Additionally, an automatic process to optimise Machine Learning models and process parameters is resorted to, basing on Bayesian approaches, to reduce operator effect. This work provides quality and process engineers with a simple framework to compare Machine Learning approaches performances and select the most suitable process modelling technique.