In the Ring Laser Gyro Inertial Navigation System (RLG INS), the temperature characteristics of the accelerometer can directly influence the measurement results. In order to improve navigation accuracy in long-endurance marine navigation, the operating temperature of the accelerometer should be precisely controlled. Based on thermal studies on the accelerometer, temperature control precision should be better than 0.01 °C to achieve 1 × 10−5 m/s2 output accuracy of the accelerometer. However, this conclusion is obtained by approximate calculations and cannot be directly applied to different inertial navigation systems. In order to verify this thermal conclusion and broaden its application, the Back Propagation Neural Network (BP-NN) algorithm is adopted to validate the feasibility of temperature control in this paper. In addition, a multi-level temperature control system is also set up and carefully designed to support the validation and experiments under different conditions. Test results of the temperature control system prove that operating temperature variation can be reduced to 0.01 °C. Meanwhile, the standard deviation per hundred seconds of the accelerometer outputs, after temperature control, reaches 1 × 10−5 m/s2. Static altitude and navigation results were improved by 41.97% and 62.91%, respectively, with the precision temperature control system, which meets the long-endurance marine navigation requirements.