The reimbursement process of invoices is very complicated, which requires manual input of key information in invoices, which wastes a lot of manpower and time. Therefore, it is particularly important to design an algorithm for intelligent identification of invoice information. This paper mainly carries on the invoice recognition system based on neural network. This paper firstly preprocessed the image and improved the Hough transform to detect the tilt Angle of the invoice image by taking the long horizontal line in the invoice layout as the target. After that, the stamp of the invoice image is removed to reduce the interference of text detection and recognition. Secondly, this paper improves invoice recognition based on YOLOv3 detection algorithm. In this paper, the invoice recognition system is constructed and compared with the other two systems. Through the experimental comparison results, it can be known that the system improves the efficiency of the staff in processing paper invoices and reduces the workload of their later registration and verification of invoices.