The recognition and classification of rock lithology is an extremely important task of geological surveys. This paper proposes a new method for quickly identifying multiple types of rocks suitable for geological survey work field. Based on the two lightweight convolutional neural networks (CNNs), SqueezeNet and MobileNets, and combined with the transfer learning method, a rock lithology recognition model was established. The model was embedded into a smart phone for testing. This method was used to identify and classify the images of 28 kinds of rocks. Through a comprehensive comparison of the two models, the accuracy of SqueezeNet and MobileNets in the test dataset is 94.55% and 93.27%, respectively. Via the two models, the average recognition time of a single rock image is 557 and 836 milliseconds, and rock images with a recognition accuracy of over 96% accounted for 95% and 93% of the entire test dataset. Compared with the classification method based on rock thin section images, this method does not need to make rock thin sections. This paper meets the requirements of workers to quickly and accurately identify rocks in the work field, which improves the work efficiency and limits identification costs. INDEX TERMS Geological survey, lightweight convolutional neural networks, multiple types, rock recognition.