In the seminal work by G. Eilenberger, Z. Phys. 214, 195 (1968), a closed-form expression for the free energy of inhomogeneous spin-singlet superconductor in terms of quasiclassical propagators has been suggested. However, deriving this expression and generalizing it for superconductors or superfluids with general matrix structure, e.g., spin-triplet correlations, has remained problematic. Starting from the Luttinger-Ward formulation, we discuss here the general solution. Besides ordinary superconductors with various scattering mechanisms, the obtained free-energy functional can be used for systems, such as superfluid 3 He and superconducting systems with spatially inhomogeneous exchange field or spin-orbit coupling. Using this result, we derive the simplified expression for the free energy in the diffusive and hydrodynamic limits. As an example of using this formalism, we show that impurity scattering restores the first-order phase transition in superconductor-ferromagnetic insulator bilayers making this system similar to the bulk superconductor with the homogeneous built-in exchange field.