In tunnel boring machine (TBM) construction, the advance rate is a crucial parameter that affects the TBM driving efficiency, project schedule, and construction cost. During the operation process, various types of indicators that are monitored in real-time can help to control the advance rate of TBM. Although some studies have already been carried out in advance rate prediction, the research is almost all based on statistical methods and shallow machine learning algorithms, thereby having difficulties in dealing with a very large amount of monitored data and in modeling the time-dependent characteristics of the parameters. To solve this problem, a deep learning model is proposed based on the CNN architecture, bidirectional Long Short-Term Memory module, and the attention mechanism, which is called the CNN-Bi-LSTM-Attention model. In the first step, the monitored data is processed, and the CNN architecture is adopted to extract features from the data sequence. Then the Bi-LSTM module is adopted to obtain the time-dependent indicators. The significant features can be addressed by the added attention mechanism. In the model training process, the rotation speed of the cutter head (N), thrust (F), torque (T), penetration rate (P), and chamber earth pressure (Soil_P) are adopted to predict the advance rate. The influence of the training periods on the model performance is also discussed. The result shows that not only the data amount, but also the data periods have an influence on the prediction. The long-term data may lead to a failure of the advance rate of TBM. The model evaluation result on the test data shows that the proposed model cannot predict the monitored data in the starting stage, which denotes that the working state of TBM in the starting stage is not stable. Especially when the TBM starts to work, the prediction error is big. The proposed model is also compared with several traditional machine methods, and the result shows the excellent performance of the proposed model.