The foundations of the Boltzmann-Gibbs (BG) distributions for describing equilibrium statistical mechanics of systems are examined. Broadly, they fall into: (i) probabilistic approaches based on the principle of equal a priori probability (counting technique and method of steepest descents), law of large numbers, or the state density considerations and (ii) a variational scheme -maximum entropy principle (due to Gibbs and Jaynes) subject to certain constraints. A minimum set of requirements on each of these methods are briefly pointed out: in the first approach, the function space and the counting algorithm while in the second, "additivity" property of the entropy with respect to the composition of statistically independent systems. In the past few decades, a large number of systems, which are not necessarily in thermodynamic equilibrium (such as glasses, for example), have been found to display power-law distributions, which are not describable by the above-mentioned methods. In this paper, parallels to all the inquiries underlying the BG program described above are given in a brief form. In particular, in the probabilistic derivations, one employs a different function space and one gives up "additivity" in the variational scheme with a different form for the entropy. The requirement of stability makes the entropy choice to be that proposed by Tsallis. From this a generalized thermodynamics description of the system in a quasi-equilibrium state is derived. A brief account of a unified consistent formalism associated with systems obeying power-law distributions precursor to the exponential form associated with thermodynamic equilibrium of systems is presented here.