As batteries become more prevalent in grid energy storage applications, the controllers that decide when to charge and discharge become critical to maximizing their utilization. Controller design for these applications is based on models that mathematically represent the physical dynamics and constraints of batteries. Unrepresented dynamics in these models can lead to suboptimal control. Our goal is to examine the state-of-the-art with respect to the models used in optimal control of battery energy storage systems (BESSs). This review helps engineers navigate the range of available design choices and helps researchers by identifying gaps in the state-of-the-art. BESS models can be classified by physical domain: state-of-charge (SoC), temperature, and degradation. SoC models can be further classified by the units they use to define capacity: electrical energy, electrical charge, and chemical concentration. Most energy based SoC models are linear, with variations in ways of representing efficiency and the limits on power. The charge based SoC models include many variations of equivalent circuits for predicting battery string voltage. SoC models based on chemical concentrations use material properties and physical parameters in the cell design to predict battery voltage and charge capacity. Temperature is modeled through a combination of heat generation and heat transfer. Heat is generated through changes in entropy, overpotential losses, and resistive heating. Heat is transferred through conduction, radiation, and convection. Variations in thermal models are based on which generation and transfer mechanisms are represented and the number and physical significance of finite elements in the model. Modeling battery degradation can be done empirically or based on underlying physical mechanisms. Empirical stress factor models isolate the impacts of time, current, SoC, temperature, and depth-of-discharge (DoD) on battery state-of-health (SoH). Through a few simplifying assumptions, these stress factors can be represented using regularization norms. Physical degradation models can further be classified into models of side-reactions and those of material fatigue. This article demonstrates the importance of model selection to optimal control by providing several example controller designs. Simpler models may overestimate or underestimate the capabilities of the battery system. Adding details can improve accuracy at the expense of model complexity, and computation time. Our analysis identifies six gaps: deficiency of real-world data in control literature, lack of understanding in how to balance modeling detail with the number of representative cells, underdeveloped model uncertainty based risk-averse and robust control of BESS, underdevelopment of nonlinear energy based SoC models, lack of hysteresis in voltage models used for control, lack of entropy heating and cooling in thermal modeling, and deficiency of knowledge in what combination of empirical degradation stress factors is most accurate. These gaps are opportunities ...