A system in equilibrium does not evolve -time independence is its telltale characteristic. However, in Newtonian physics the microstate of an individual system (a point in its phase space) evolves incessantly in accord with its equations of motion. Ensembles were introduced in XIX century to bridge that chasm between continuous motion of phase space points in Newtonian dynamics and stasis of thermodynamics: While states of individual classical systems inevitably evolve, a phase space distribution of such states -an ensemble -can be time-independent. I show that entanglement (e.g., with the environment) can yield a time-independent equilibrium in an individual quantum system. This allows one to eliminate ensembles -an awkward stratagem introduced to reconcile thermodynamics with Newtonian mechanics -and use an individual system interacting and therefore entangled with its heat bath to represent equilibrium and to elucidate the role of information and measurements in physics. Thus, in our quantum Universe one can practice statistical physics without ensembles -hence, in a sense, without statistics. The elimination of ensembles uses ideas that led to the recent derivation of Born's rule from the symmetries of entanglement, and I start with a review of that derivation. I then review and discuss difficulties related to the reliance on ensembles and illustrate the need for ensembles with the classical Szilard's engine. A similar quantum engine -a single system interacting with the thermal heat bath environment -is enough to establish thermodynamics. The role of Maxwell's demon (which in this quantum context resembles Wigner's friend) is also discussed.