Economics has seen a recent rise in interest in information theory as an alternative framework to the conventional notion of equilibrium as a fixed state, such as Walrasian marketclearing general equilibrium. The information theoretic approach is predicated on the notion of statistical equilibrium (SE) that takes a distribution over all possible states as an equilibrium, and therefore predicts the endogenous fluctuations of the system along with its central tendency simultaneously. For this reason, SE approaches can explain the observed data without relying on arbitrary assumptions about random noise and provide useful insights for many interesting economic problems that conventional methods have not been able to satisfactorily deal with. In this paper, we review the key elements of information theory focusing on the notions and applications of entropy and SE in economics, particularly paying attention to how entropy concepts open up a new frontline of economic research.