Friction stir welding is an advanced joining technology that is particularly suitable for aluminum alloys. Various studies have shown that welding quality depends significantly on the welding speed and the rotational speed of the tool. It is frequently possible to detect an unsuitable setting of these parameters by examining the resulting surface defects, such as increased flash formation or surface galling. In this work, Artificial Neural Networks were used to analyze process data in friction stir welding and predict the resulting quality of the weld surface. For this purpose, nine different variables were recorded during friction stir welding of EN AW-6082 T6 sheets: the forces and accelerations in three spatial directions, the spindle torque, and temperatures at the tool shoulder and tool probe. In Case 1, the welds were assigned to the classes good and defective on the basis of a human visual inspection of the weld surface. In Case 2, the welds were categorized into the two classes on the basis of a surface topography analysis. Subsequently, three different major Artificial Neural Network architectures were tested for their ability to predict the surface quality: Feed Forward Fully Connected Neural Networks, Recurrent Neural Networks and Convolutional Neural Networks. The highest classification accuracy was achieved when Convolutional Neural Networks were used. Thus, the evaluation of the force signal transverse to the welding direction yielded the highest accuracy of 99.1% for the prediction of the result of the human visual inspection. The result achieved for the prediction of the topography analysis was an accuracy of 87.4% when the spindle torque was evaluated. Using all nine different process variables to predict the topography analysis, the accuracy could be improved slightly to 88.0%. The sampling rate of the spindle torque was varied between 40 Hz and 9600 Hz and no significant influence was determined. The findings show that Convolutional Neural Networks are well suited for the interpretation of friction stir welding process data and can be used to predict the resulting surface quality. In future work, the results are to be used to develop a parameter optimization method for friction stir welding.