Microelectromechanical System (MEMS) Inertial Measurement Unit (IMU) is popular in the community for constructing a navigation system, due to its small size and low power consumption. However, limited by the manufacturing technology, MEMS IMU experiences more complicated noises and errors. Thus, noise modeling and suppression is important for improving accuracy of the navigation system based on MEMS IMU. Motivated by this problem, in this paper, a deep learning method was introduced to MEMS gyroscope de-noising. Specifically, a recently popular Recurrent Neural Networks (RNN) variant Simple Recurrent Unit (SRU-RNN) was employed in MEMS gyroscope raw signals de-noising. A MEMS IMU MSI3200 from MT Microsystem Company was employed in the experiments for evaluating the proposed method. Following two problems were furtherly discussed and investigated: (1) the employed SRU with different training data length were compared to explore whether there was trade-off between the training data length and prediction performance; (2) Allan Variance was the most popular MEMS gyroscope analyzing method, and five basic parameters were employed to describe the performance of different grade MEMS gyroscope; among them, quantization noise, angle random walk, and bias instability were the major factors influencing the MEMS gyroscope accuracy, the compensation results of the three parameters for gyroscope were presented and compared. The results supported the following conclusions: (1) considering the computation brought from training dataset, the values of 500, 3000, and 3000 were individually sufficient for the three-axis gyroscopes to obtain a reliable and stable prediction performance; (2) among the parameters, the quantization noise, angle random walk, and bias instability performed 0.6%, 6.8%, and 12.5% improvement for X-axis gyroscope, 60.5%, 17.3%, and 34.1% improvement for Y-axis gyroscope, 11.3%, 22.7%, and 35.7% improvement for Z-axis gyroscope, and the corresponding attitude errors decreased by 19.2%, 82.1%, and 69.4%. The results surely demonstrated the effectiveness of the employed SRU in this application.