Dielectric elastomer (DE) transducers are known to exhibit a rate-dependent hysteresis in their force-displacement response, which is commonly attributed to the viscoelastic behavior of the elastomer material as well as the compliant electrodes. In case of DE materials characterized by low mechanical losses, such as silicone, the mechanical hysteresis often turns out to be practically rate-independent in the low frequency range (sub-Hz), while rate-dependent hysteretic effects only become relevant at higher deformation rates. Most of existing literature focuses on describing DE hysteretic losses through viscoelasticity theory. Even though this approach results in relatively simple dynamic models, those are not capable to describe rate-independent hysteretic behaviors. In this work, we propose a control-oriented modeling framework for both rate-dependent and rate-dependent hysteresis occurring in uniaxially-loaded DE actuators. The model combines classic thermodinamically-consistent modeling approaches for DEs with a new energy-based Maxwell-Lion formalization of the hysteretic losses. The resulting dynamic model consists of a set of nonlinear ordinary differential equations, and is capable of simultaneously describing geometry dependencies, large deformation nonlinearities, electro-mechanical coupling, as well as rate-independent and rate-dependent hysteretic effects. To deal with the large number of involved parameters, a new systematic identification algorithm based on quadratic programming is also proposed. After presenting the theory, the model is validated based on experiments conducted on a silicone-based rolled DE actuator, and its superiority compared to classic DE viscoelstic models is quantitatively assessed.