Among many other sectors, the automobile industry relies heavily on resistance spot welding (RSW) to attach metal components. In order to determine if spot welded connections are weldable, this research develops a machine learning (ML) model and a parameter-less optimization method, and then suggests a data-driven welding analytics framework. It trains the ML model using data from experiments and simulations, which allows it to make accurate predictions about weld quality. In order to get the finest possible weld in the shortest amount of time, RL has been used to adjust factors like the FSW pin’s rotational and translational speeds. Due to their thorough exploration of the environment, these algorithms achieved extremely high accuracy rates, indicating that they are trustworthy in their ability to operate the agent optimally. However, there may be a large disparity in the precision with which different models provide predictions. Prioritizing the evaluation and selection of the most effective prediction model or models is of the highest significance. In addition, we build, test, and evaluate previous models of nugget width prediction using deep reinforcement learning (DRL). To begin with, we use bootstrapping to create sample distributions for all of the prediction models. Then, we evaluate their performances using statistical tests to see if there are any significant differences. This investigation shows that DRL which is built for RSW nugget width forecasting in this work, performs better than earlier models.