The present work concerns the modeling of the Payne effect in nonlinear viscoelasticity. This effect is a characteristic property of filled elastomers. Indeed, under cyclic loading of increasing amplitude, a decrease is shown in the storage modulus and a peak in the loss modulus. In this study, the Payne effect is assumed to arise from a change of the material microstructure, i.e., the thixotropy. The so-called intrinsic time or shift time was inferred from solving a differential equation that represents the evolution of a material's microstructure. Then, the physical time is replaced by the shift time in the framework of a recent fractional visco-hyperelastic model, which was linearized in the neighborhood of a static pre-deformation. As a result, we have investigated the effects of static pre-deformation, frequency, and magnitude of dynamic strain on storage and loss moduli in the steady state. Thereafter, the same set of parameters identified from the complex Young's modulus was used to predict the stress in the pre-deformed configuration. Finally, it is demonstrated that the proposed model is reasonably accurate in predicting Payne effect.