Thermophilic proteins are proteins that can maintain good activity at high temperatures, they have important application value in the fields of enzyme engineering and biopharmaceuticals. Therefore, accurate identification of thermophilic proteins provides important information for their engineering applications. In this study, we proposed an improved predictor of thermophilic protein using ProtBert protein language model, called PB-TMP. Firstly, we used the ProtBert protein language model to extract the hidden features in protein sequences, and 1024-dimensional features are obtained. Secondly, Recursive Feature Elimination (RFE) are employed to remove the noise or redundant features, 31-dimensional features are selected as the final features set. Finally, Multi-Layer Perceptron (MLP) is trained on the final features set, then an identification model is obtained. The model yielded an accuracy of 98.53% on 10-fold cross-validation, and performed best among the existing methods. The model will accelerate the discovery of thermophilic protein and important application in protein engineering and biotechnology.