Despite dedicated effort for many decades, statistical description of highly technologically important wall turbulence remains a great challenge. Current models are unfortunately incomplete, or empirical, or qualitative. After a review of the existing theories of wall turbulence, we present a new framework, called the structure ensemble dynamics (SED), which aims at integrating the turbulence dynamics into a quantitative description of the mean flow. The SED theory naturally evolves from a statistical physics understanding of non-equilibrium open systems, such as fluid turbulence, for which mean quantities are intimately coupled with the fluctuation dynamics. Starting from the ensemble-averaged Navier-Stokes (EANS) equations, the theory postulates the existence of a finite number of statistical states yielding a multi-layer picture for wall turbulence. Then, it uses order functions (ratios of terms in the mean momentum as well as energy equations) to characterize the states and transitions between states. Application of the SED analysis to an incompressible channel flow and a compressible turbulent boundary layer shows that the order functions successfully reveal the multi-layer structure for wall-bounded turbulence, which arises as a quantitative extension of the traditional view in terms of sub-layer, buffer layer, log layer and wake. Furthermore, an idea of using a set of hyperbolic functions for modeling transitions between layers is proposed for a quantitative model of order functions across the entire flow domain. We conclude that the SED provides a theoretical framework for expressing the yet-unknown effects of fluctuation structures on the mean quantities, and offers new methods to analyze experimental and simulation data. Combined with asymptotic analysis, it also offers a way to evaluate convergence of simulations. The SED approach successfully describes the dynamics at both momentum and energy levels, in contrast with all prevalent approaches describing the mean velocity profile only. Moreover, the SED theoretical framework is general, independent of the flow system to study, while the actual functional form of the order functions may vary from flow to flow. We assert that as the knowledge of order functions is accumulated and as more flows are analyzed, new principles (such as hierarchy, symmetry, group invariance, etc.) governing the role of turbulent structures in the mean flow properties will be clarified and a viable theory of turbulence might emerge.