This paper mainly proposes two kinds of artificial neural network (ANN) models for predicting the plastic anisotropy properties of sheet metal using spherical indentation test, which minimizes measurement time, costs, and simplifies the process of obtaining the anisotropy properties than the conventional tensile test. The proposed ANN models for predicting anisotropic properties can replace the traditional complex dimensionless analysis. Moreover, this paper is not limited to the prediction of yield strength anisotropy but also further accurately predicts the Lankford coefficient in different orientations. We newly construct an FE spherical indentation model, which is suitable for sheet metal in consideration of actual compliance. To obtain a large dataset for training the ANN, the constructed FE model is utilized to simulate pure and alloyed engineering metals with one thousand elastoplastic parameter conditions. We suggest the specific variables of the residual indentation mark as input parameters, also with the indentation load–depth curve. The profile of the residual indentation, including the height and length in different orientations, are used to analyze the anisotropic properties of the material. Experimental validations have been conducted with three different sheet alloys, TRIP1180 steel, zinc alloy, and aluminum alloy 6063-T6, comparing the proposed ANN model and the uniaxial tensile test. In addition, machine vision was used to efficiently analyze the residual indentation marks and automatically measure the indentation profiles in different orientations. The proposed ANN model exhibits remarkable performance in the prediction of the flow curves and Lankford coefficient of different orientations.