All ensembles of statistical mechanics are equivalent in the sense that they give the equivalent thermodynamic functions in the thermodynamic limit. However, when investigating microscopic structures in the first-order phase transition region, one must choose an appropriate statistical ensemble. The appropriate choice is particularly important when one investigates finite systems, for which even the equivalence of ensembles does not hold. We propose a class of statistical ensembles, which always give the correct equilibrium state even in the first-order phase transition region. We derive various formulas for this class of ensembles, including the one by which temperature is obtained directly from energy without knowing entropy. Moreover, these ensembles are convenient for practical calculations because of good analytic properties. We also derive formulas which relate statistical-mechanical quantities of different ensembles, including the conventional ones, for finite systems. The formulas are useful for obtaining results with smaller finite size effects, and for improving the computational efficiency. The advantages of the squeezed ensembles are confirmed by applying them to the Heisenberg model and the frustrated Ising model. arXiv:1903.04111v2 [cond-mat.stat-mech]