Endoscopic specialists performing gastroscopy, which relies on the naked eye, may benefit from a computer-aided diagnosis (CADx) system that employs deep learning. This report proposes utilizing a CADx system to classify normal and abnormal gastric cancer, gastritis, and gastric ulcer. The CADx system was trained using a deep learning algorithm known as a convolutional neural network (CNN). Specifically, Xception, which includes depth-wise separable convolution, was employed as the CNN. Image augmentation was applied to improve the disadvantages of medical data, which are difficult to collect. A class activation map (CAM), an algorithm that visualizes the classified region of interest in a CNN, was used to cut and paste the image area into another image. The CAM-identified lesion location in an abnormal image was augmented by pasting it into a normal image. The normal image was divided into nine equal parts and pasted where the variance difference from the lesion was minimal. Consequently, the number of abnormal images increased by 360,905. Xception was used to train the augmented dataset. A confusion matrix was used to evaluate the performance of the gastroscopy CADx system. The performance criteria were specificity, sensitivity, F1 score, harmonic average of precision, sensitivity (recall), and AUC. The F1 score of the CADx system trained with the original dataset was 0.792 and AUC was 0.885. The dataset augmentation approach using CAM presented in this report is shown to be an effective augmentation algorithm, with performance improved to 0.835, 0.903 in terms of F1 score and AUC respectively.