Artificial neural networks (ANNs) have been an important approach for predicting the value of flow stress, which is dependent on temperature, strain, and strain rate. However, there is still a lack of sufficient knowledge regarding what structure of ANN should be used for predicting metal flow stress. In this paper, we train an ANN for predicting flow stress of In718 alloys at high temperatures using our experimental data, and the structure of the ANN is optimized by comparing the performance of four ANNs in predicting the flow stress of In718 alloy. It is found that, as the size of the ANN increases, the ability of the ANN to retrieve the flow stress results from a training dataset is significantly enhanced; however, the ability to predict the flow stress results absent from the training does not monotonically increase with the size of the ANN. It is concluded that the ANN with one hidden layer and four nodes possesses optimized performance for predicting the flow stress of In718 alloys in this study. The reason why there exists an optimized ANN size is discussed. When the ANN size is less than the optimized size, the prediction, especially the strain dependency, falls into underfitting and fails to predict the curve. When the ANN size is less than the optimized size, the predicted flow stress curves with the temperature, strain, and strain rate will contain non-physical fluctuations, thus reducing their prediction accuracy of extrapolation. For metals similar to the In718 alloy, ANNs with very few nodes in the hidden layer are preferred rather than the large ANNs with tens or hundreds of nodes in the hidden layers.