In this paper, the necessary condition for the chaotic motion of a Duffing oscillator with the fractional-order derivative under harmonic excitation is investigated. The necessary condition for the chaos in the sense of Smale horseshoes is established based on the Melnikov method, and then the chaotic threshold curve is obtained. The largest Lyapunov exponents are provided, and some other typical numerical simulation results, including the time histories, frequency spectrograms, phase portraits, and Poincare maps, are presented and compared. From the analysis of the numerical simulation results, it could be found that, near the chaotic threshold curve, the system generates chaos via the period-doubling bifurcation, from single periodic motion to period-2 motion and period-4 motion to chaotic motion. The effects of fractional-order parameters, the stiffness coefficient, and the damping coefficient on the threshold value of the chaotic motion are analytically discussed. The results show that the coefficient of the fractional-order derivative has great effect on the threshold value of the chaotic motion, while the order of the fractional-order derivative has less. The analysis results reveal some new phenomena, and it could be useful for designing or controlling dynamic systems with the fractional-order derivative.