The weld defects are formed due to the incorrect welding patterns or wrong welding process. The defects in the weld may vary from size, shape and their projected quality. The most common weld defects occur during welding process is slag inclusions, porosity, lack of fusion and incomplete penetration. In this study, an effective method for weld defect classification using machine learning algorithm is presented. The system uses Speeded-up Robust Features (SURF) for feature extraction and one of the machine learning algorithms called Auto-Encoder Classifier (AEC) for classification. Initially, the features that distinguish weld defects and no defects in the weld image are extracted by SURF. Then, AEC is analyzed for weld image classification using different number of neurons in different hidden layers (2 and 3 hidden layers). The performance of the system is evaluated by GD X-ray weld image database. The results show that the weld images are correctly classified with 98% accuracy using SURF and AEC.