Fractional differential equations are more and more used in modeling memory (history-dependent, nonlocal, or hereditary) phenomena. Conventional initial values of fractional differential equations are defined at a point, while recent works define initial conditions over histories. We prove that the conventional initialization of fractional differential equations with a Riemann-Liouville derivative is wrong with a simple counter-example. The initial values were assumed to be arbitrarily given for a typical fractional differential equation, but we find one of these values can only be zero. We show that fractional differential equations are of infinite dimensions, and the initial conditions, initial histories, are defined as functions over intervals. We obtain the equivalent integral equation for Caputo case. With a simple fractional model of materials, we illustrate that the recovery behavior is correct with the initial creep history, but is wrong with initial values at the starting point of the recovery. We demonstrate the application of initial history by solving a forced fractional Lorenz system numerically.