Flow-rutting is the main distress leading asphalt pavement to undergo premature maintenance, and is produced by the rapid accumulation of shear deformation in asphalt layers under high temperature and heavy loads. The excessive permanent deformation of the asphalt mixture at high temperature is related to the decrease of the material’s stability during the temperature increase and an unfavorable stress state, e.g., low confining pressure and high shear stress, which eventually leads to significant nonlinear viscoplastic behavior. In this research, dynamic modulus tests and repeated loading tests were carried out at 35 °C and 50 °C to analyze the deformation response of materials under a strain amplitude of <200 με and 400~500 μεs, respectively. Based on the in-lab repeated loading tests, the total deformation of the asphalt mixture in each loading and rest cycle was divided into three parts, being elastic, viscoelastic, and viscoplastic strain, and the measurement of the axial and lateral strain of cylindrical samples was realized with the aid of optical fiber Bragg grating strain sensors. It was found that the experimental index of the ratio between lateral strain and longitudinal strain (RLSLS), derived, but distinguished, from Poisson’s ratio defined limited in elastic strain, can characterize the deformation in viscoelastic and viscoplastic behaviors of the mixes. Furthermore, the indices of dynamic modulus, phase angle, complex Poisson’s ratio, stiffness, and creep rate of four types of mixes containing different volcanic ash fillers and asphalt binders at 35 °C and 50 °C were systematically analyzed by the jointed experiments of modified dynamic modulus tests and repeated loading tests, and their consistent trending to the RLSLS index was obtained.