In order to depict the effect of driver’s memory on car-following behavior, a new kind of car-following model is proposed by using fractional order differential equation in this paper. Its dynamic equation is defined by Caputo fractional order derivative. And the order of derivative is the measurement of driver’s memory. In addition, discrete formulas of the position and velocity of the new model are given. The Optimal Velocity (OV) model is taken as an example to introduce how to get the fractional order car-following model from an ordinary model. The simulation results show that the Fractional Order Optimal Velocity (FOOV) model is more stable, and it can avoid unrealistic acceleration values of the OV model in the cases of starting and braking processes. Moreover, magnitudes of the speed and headway fluctuation of the FOOV model with a suitable order are smaller than those of the OV model. This indicates that the memory characteristic of drivers increases the stability of traffic flow.