In recent years, machine learning algorithms had good performance in many fields. On the one hand, its predictive ability is greatly improved; on the other hand, with the increase of the model complexity, the interpretability of the algorithm is even worse. In this paper, we propose a novel method for improving the tree ensemble model by balancing predictive performance and interpretability. The rule extraction turns tree models into “if-then” rules. The rule pruning method removes the redundant constraints. And the rule selection method selects the optimal rule subset based on the genetic algorithm. An evaluation of the proposed method on the regression problem has been performed. Experiments on acute toxicity datasets demonstrate the effectiveness of the proposed approach.