The status of heat and work in nonequilibrium thermodynamics is quite confusing and nonunique at present with conflicting interpretations even after a long history of the first law dE(t) = d e Q(t) − dW e (t) in terms of exchange heat and work, and is far from settled. Moreover, the exchange quantities lack certain symmetry (see text). By generalizing the traditional concept to also include their time-dependent irreversible components d i Q(t) and d i W (t) allows us to express the first law in a symmetric form dE(t) = dQ(t) − dW (t) in which dQ(t) and work dW (t) appear on equal footing and possess the symmetry. We prove that d i Q(t) ≡ d i W (t); as a consequence, irreversible work turns into irreversible heat. Statistical analysis in terms of microstate probabilities p i (t) uniquely identifies dW (t) as isentropic and dQ(t) as isometric (see text) change in dE(t), a result known in equilibrium. We show that such a clear separation does not occur for d e Q(t)and dW e (t). Hence, our new formulation of the first law provides tremendous advantages and results in an extremely useful formulation of non-equilibrium thermodynamics, as we have shown recently [Phys. Rev. E 81, 051130 (2010); ibid 85, 041128 and 041129 ( 2012)]. We prove that an adiabatic process does not alter p i . All these results remain valid no matter how far the system is out of equilibrium. When the system is in internal equilibrium, dQ(t) ≡ T (t)dS(t) in terms of the instantaneous temperature T (t) of the system, which is reminiscent of equilibrium, even though, neither d e Q(t) ≡ T (t)d e S(t) nor d i Q(t) ≡ T (t)d i S(t). Indeed, d i Q(t) and d i S(t) have very different physics. We express these quantities in terms of d e p i (t) and d i p i (t), and demonstrate that p i (t) has a form very different from that in equilibrium. The first and second laws are no longer independent so that we need only one law, which is again reminiscent of equilibrium. The traditional formulas like the Clausius inequality d e Q(t)/T 0 < 0, ∆ e W < −∆ [E(t − T 0 S(t))], etc. become equalities dQ(t)/T (t) ≡ 0, ∆W = −∆ [E(t − T (t)S(t)], etc, a quite remarkable but unexpected result in view of the fact that ∆ i S(t) > 0. We identify the uncompensated transformation N (t, τ ) during a cycle. We determine the irreversible components in two simple cases to show the usefulness of our approach; here, the traditional formulation is of no use. Our extension bring about a very strong parallel between equilibrium and non-equilibrium thermodynamics, except that one has irreversible entropy generation d i S(t) > 0 in the latter.