Human drivers’ behavior, which is very difficult to model, is a very complicated stochastic system. To characterize a high-accuracy driver behavior model under different roadway geometries, the paper proposes a new algorithm of driver behavior model based on the whale optimization algorithm-restricted Boltzmann machine (WOA-RBM) method. This method establishes an objective optimization function first, which contains the training of RBM deep learning network based on the real driver behavior data. Second, the optimal training parameters of the restricted Boltzmann machine (RBM) can be obtained through the whale optimization algorithm. Finally, the well-trained model can be used to represent the human drivers’ operation effectively. The MATLAB simulation results showed that the driver model can achieve an accuracy of 90%.