A signature is a useful human feature in our society, and determining the genuineness of a signature is very important. A signature image is typically analyzed for its genuineness classification; however, increasing classification accuracy while decreasing computation time is difficult. Many factors affect image quality and the genuineness classification, such as contour damage and light distortion or the classification algorithm. To this end, we propose a mobile computing method of signature image authentication (SIA) with improved recognition accuracy and reduced computation time. We demonstrate theoretically and experimentally that the proposed golden global-local (G-L) algorithm has the best filtering result compared with the methods of mean filtering, medium filtering, and Gaussian filtering. The developed minimum probability threshold (MPT) algorithm produces the best segmentation result with minimum error compared with methods of maximum entropy and iterative segmentation. In addition, the designed convolutional neural network (CNN) solves the light distortion problem for detailed frame feature extraction of a signature image. Finally, the proposed SIA algorithm achieves the best signature authentication accuracy compared with CNN and sparse representation, and computation times are competitive. Thus, the proposed SIA algorithm can be easily implemented in a mobile phone.