According to the survey, off-line examination is still the main examination method in universities, primary and secondary schools. The grading processing of off-line examination is time-consuming. Besides, since the off-line grading is subjective, it is error-prone. In order to address the challenges in off-line examinations of universities, primary and secondary schools, it is very urgent to improve the efficiency of off-line grading. In order to realize intelligent grading for off-line examinations, we exploit deep learning techniques to off-line grading. First, we propose an image processing method for English letters. Second, we propose a image recognition method based on deep learning for English letters. Third, we propose a lightweight framework for grading. Based on the above designs, we design an intelligent grading system based on deep learning. We implement the system and the result shows that the intelligent grading system can batch grading efficiently. Besides, compared with related designs, the proposed system is more flexible and intelligent.