Geothermal surface manifestations (GSMs) are direct clues towards hydrothermal activities of a geothermal system in the subsurface and significant indications for geothermal resource exploration. It is essential to recognize various GSMs for potential geothermal energy exploration. However, there is a lack of work to fulfill this task using deep learning (DL), which has achieved unprecedented successes in computer vision and image interpretation. This study aims to explore the feasibility of using a DL model to fulfill the recognition of GSMs with photographs. A new image dataset was created for the GSM recognition by preprocessing and visual interpretation with expert knowledge and a high-quality check after downloading images from the Internet. The dataset consists of seven GSM types, i.e., warm spring, hot spring, geyser, fumarole, mud pot, hydrothermal alteration, crater lake, and one type of none GSM, including 500 images of different photographs for each type. The recognition results of the GoogLeNet model were compared with those of three machine learning (ML) algorithms, i.e., Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbor (KNN), by using the assessment metrics of overall accuracy (OA), overall F1 score (OF), and computational time (CT) for training and testing the models via cross-validation. The results show that the retrained GoogLeNet model using transfer learning has significant advantages of accuracies and performances over the three ML classifiers, with the highest OA, the biggest OF, and the fastest CT for both the validation and test. On the contrary, the three selected ML classifiers perform poorly for this task due to their low OA, small OF, and long CT. This suggests that transfer learning with a pretrained network be a feasible method to fulfill the recognition of the GSMs. Hopefully, this study provides a reference paradigm to help promote further research on the application of state-of-the-art DL in the geothermics domain.