Rutting and cracking occur simultaneously in asphalt mixtures as observed in the field and laboratory. Existing mechanical models have not properly addressed viscoelastic and viscoplastic deformation together with cracking due to model deficiencies, parameter calibration and numerical inefficiency. This study develops viscoelastic-plastic-fracture (VEPF) models characterizing viscoelasticity (VE) by Prony model and viscoplasticity (VP) by Perzyna's flow rule with a Generalized Drucker-Prager (GD-P) yield surface and a non-associated plastic potential. Viscofracture (VF) damage is modelled by a viscoelastic Griffith criterion and a pseudo J-integral Paris' law for crack initiation and propagation, respectively. The VEPF models are implemented in a finite element (FE) program using a weak form partial differential equation (PDE) modeling technique with no need to program user-defined material subroutines. Model parameters were derived from fundamental material properties using dynamic modulus, strength and repeated load tests. Simulations indicate that the VE-VP-VF characteristics are effectively modelled by the VEPF models for different asphalt mixtures at different confinements and temperatures. An asphalt mixture under monotonic compressive loads exhibits a sequenced process including a pure VE deformation stage, a coupled VE-VP deformation stage, a VE-VP deformation coupled with VF initiation and propagation stage, and then a VE-VF rupture stage with saturated VP deformation. The asphalt mixture under repeated loads yields an increasing VP strain at an increasing rate during the first half of the haversine load while a decreasing VP strain increment with load cycles. The PDE-based FE is capable of effectively modeling the coupled VE-VP-VF behaviors of the asphalt mixtures.