Aiming at the problem that convolutional neural network is difficult to deploy on small embedded devices due to its high complexity and large storage space requirement, this paper propose a convolutional neural network FPGA accelerator architecture based on binarization. Using the gray scale processing, binarization processing, threshold setting to reduce the number of parameters. Designing Parallel structures of convolution kernels, feature maps, and matrix blocks to accelerate. The designed architecture can be deployed on the AX7103 FPGA development platform with limited resources. The experimental results show that the convolutional neural network after parallel acceleration design can achieve a recognition accuracy rate of 98.73% on the premise of reducing the data bit width from 32 bits to 8 bits, the recognition speed is about 0.21 seconds/time.