The purpose of the current work is the formulation of models for conservative and non-conservative dynamics in solid systems with the help of the General Equation for the Non-Equilibrium Reversible-Irreversible Coupling (GENERIC: e.g., Grmela and Öttinger, Phys. Rev. E 56(6), 6620 (1997); Öttinger and Grmela, Phys. Rev. E 56(6), 6633 (1997)). In this context, the resulting models are inherently spatially strongly non-local (i.e., functional) and non-isothermal in character. They are applicable in particular to the modeling of phase transitions as well as mass and heat transport in multiphase, multicomponent solids. In the last part of the work, the strongly non-local model formulation is reduced to weakly non-local form with the help of generalized gradient approximation of the energy and entropy functionals. On this basis, the current model formulation is shown to be consistent with and reduce to a recent non-isothermal generalization (Gladkov et al., J. Non-Equilib. Thermodyn. 41(2), 131 (2016)) of the well-known phase-field models of Cahn and Hilliard (J. Chem. Phys. 28(2), 258 (1958)) for conservative dynamics and of Allen and Cahn (Acta Metall. 27(6), 1085 (1979)) for non-conservative dynamics. Finally, the current approach is applied to derive a non-isothermal generalization of a phase-field crystal model for binary alloys (see, e.g., Elder et al., Phys. Rev. B 75(6), 064107 (2007)).