Predicting brain age of children accurately and quantitatively can give help in brain development analysis and brain disease diagnosis. Traditional methods to estimate brain age based on 3D magnetic resonance (MR), T1 weighted imaging (T1WI), and diffusion tensor imaging (DTI) need complex preprocessing and extra scanning time, decreasing clinical practice, especially in children. This research aims at proposing an end-to-end AI system based on deep learning to predict the brain age based on routine brain MR imaging. We spent over 5 years enrolling 220 stacked 2D routine clinical brain MR T1-weighted images of healthy children aged 0 to 5 years old and randomly divided those images into training data including 176 subjects and test data including 44 subjects. Data augmentation technology, which includes scaling, image rotation, translation, and gamma correction, was employed to extend the training data. A 10-layer 3D convolutional neural network (CNN) was designed for predicting the brain age of children and it achieved reliable and accurate results on test data with a mean absolute deviation (MAE) of 67.6 days, a root mean squared error (RMSE) of 96.1 days, a mean relative error (MRE) of 8.2%, a correlation coefficient (R) of 0.985, and a coefficient of determination (R 2) of 0.971. Specially, the performance on predicting the age of children under 2 years old with a MAE of 28.9 days, a RMSE of 37.0 days, a MRE of 7.8%, a R of 0.983, and a R 2 of 0.967 is much better than that over 2 with a MAE of 110.0 days, a RMSE of 133.5 days, a MRE of 8.2%, a R of 0.883, and a R 2 of 0.780.