PACS 05.20.-y -statistical mechanics PACS 05.10.Gg -stochastic models in statistical mechanics and non-linear dynamics PACS 05.70.Ln -irreversible thermodynamics Abstract -We study a class of non-equilibrium quasi-stationary states for a Markov system interacting with two different thermal baths. We show that the work done under a slow, external change of parameters admits a potential, i.e., the free energy. Three conditions are needed for the existence of free energy in this non-equilibrium system: time-scale separation between variables of the system, partial controllability (external fields couple only with the slow variable), and an effective detailed balance. These conditions are facilitated in the continuous limit for the slow variable. In contrast to its equilibrium counterpart, the non-equilibrium free energy can increase with temperature. One example of this is that entropy reduction by means of external fields (cooling) can be easier (in the sense of the work cost) if it starts from a higher temperature.