Structural inherent dynamic characteristics would change when damage occurs, and this fact is the theoretical basis of damage detection method based on vibration features. As one of structural inherent properties, frequency response function (FRF) contains rich information and has more potential in damage detection. On the basis of crack detection researches, a new method based on the change of FRF is proposed in this paper. First, the single-degree-of-freedom (SDOF) system is used to illustrated the change of dynamic features caused by cracks represented by changing stiffness, and it is found that the change of stiffness leads to the obvious nonlinear variation of frequency response function curvature (FRFC) and frequency response function curvature differentiation (FRFCD) near the natural frequency. To describe this phenomenon, the Pearson correlation coefficient is employed to analyze correlation relationship of FRF, FRFC and FRFCD between the undamaged and the damaged system. The results demonstrate that absolute values of these coefficients approach from 1 to 0 with the increase of stiffness change. The rail with cracks is taken as the research object, and relationships between the crack size and the correlation coefficients of FRF, FRFC and FRFCD are investigated through the combination of finite element simulation and frequency response experiment. The correlation coefficients of FRF, FRFC and FRFCD are close to 1 when the structure is intact. When cracks occur, the absolute value of the correlation coefficient will gradually trend from 1 to 0, and the sensitivity of FRFCD correlation coefficient to crack is much higher than that of FRF and FRFC. The selection of calculated frequency bands containing different modes and the damping of the structure would affect the absolute value of FRFCD correlation coefficient, but will not change the characteristics of high sensitivity of FRFCD and the relationship between the coefficient and crack size. Besides, noise signals would enhance the nonlinear relationship between responses of intact structure and the structure with cracks, and the FRFCD correlation coefficient value is influenced accordingly.