We proposed an unsupervised end-to-end Affine and Deformable Medical Image Registration (ADMIR) method based on convolutional neural network (ConvNet). ADMIR includes three key components: an affine registration module for learning the affine transformation parameters, a deformable registration module for learning the displacement vector field, and a spatial transformer for getting the final warped image from both affine and deformable transformation parameters. To evaluate its performance, the magnetic resonance images of drug-addicted brains were used to train and test the model, and we compared it with two state-of-art methods in terms of Dice score, Hausdorff distance, and average symmetric surface distance. The experimental results demonstrated that our proposed ADMIR model outperforms existing methods even with the images without pre-alignment, which suggests that the ADMIR model can be used to achieve quick medical image registration with high accuracy. INDEX TERMS Deformable image registration, affine registration, convolutional neural network, unsupervised learning, end-to-end registration.