Complex band structure generalizes conventional band structure by also considering wavevectors with complex components. In this way, complex band structure describes both the bulk-propagating states from conventional band structure and the evanescent states that grow or decay from one unit cell to the next. Even though these latter states are excluded by translational symmetry, they become important when translational symmetry is broken via, for example, a surface or impurity. Many studies over the last 80 years have directly or indirectly developed complex band structure for an impressive range of applications, but very few discuss its fundamentals or compare its various results. In this work we build upon these previous efforts to expose the physical foundation of complex band structure, which mathematically implies its existence. We find that a material's static and dynamic electronic structure are both completely described by complex band structure. Furthermore, we show that complex band structure reflects the minimal, intrinsic information contained in the material's Hamiltonian. These realizations then provide a context for comparing and unifying the different formulations and applications of complex band structure that have been reported over the years. Ultimately, this discussion introduces the idea of examining the amount of information contained in a material's Hamiltonian so that we can find and exploit the minimal information necessary for understanding a material's properties.