Thermodynamic models for viscoplastic solids are often formulated in the context of continuum thermodynamics and the dissipation principle. The purpose of the current work is to show that models for such material behavior can also be formulated in the form of a General Equation for Non-Equilibrium Reversible-Irreversible Coupling (GENERIC), see, e.g., Grmela and Öttinger (Phys Rev E, 56:6620-6632, 1997), Öttinger and Grmela (Phys Rev E, 56:6633-6655, 1997), Grmela (J Non-Newtonian Fluid Mech, 165:980-986, 2010). A GENERIC combines Hamiltonian-dynamics-based modeling of time-reversible processes with Onsager-Casimir-based modeling of time-irreversible processes. The result is a model for the approach of non-equilibrium systems to thermodynamic equilibrium. Originally developed to model complex fluids, it has recently been applied to anisotropic inelastic solids in Eulerian (Hütter and Tervoort, in J Non-Newtonian Fluid Mech, 152:45-52, 2008; Hütter and Tervoort, in J Non-Newtonian Fluid Mech, 152:53-65, 2008; Hütter and Tervoort, in Adv Appl Mech, 42:254-317, 2008) and Lagrangian (Hütter and Svendsen, in J Elast 104:357-368, 2011) settings, as well as to damage mechanics. For simplicity, attention is focused in the current work on the case of thermoelastic viscoplasticity. Central to this formulation is a GENERIC-based form for the viscoplastic flow rule. A detailed comparison with the formulation based on continuum thermodynamics and the dissipation principle is given.