Agent-based modeling (ABM) has been successfully used, since its emergence in the 1990s, to model and simulate the dynamics at work in complex socio-environmental systems, in many domains and applications where interactions between people and their environments give rise to emergent phenomena that are difficult to study otherwise (urban planning, land-use change, adaptation to environmental changes, biodiversity protection in socio-ecosystems, environmental pollution control, etc.). The inclusion of multiple levels of analysis, abstraction, and representation in these models, however, is much more recent and is still the subject of many proposals and discussions within a relatively informal field, Multilevel Agent-Based Modeling (ML-ABM), which is most often presented as an approach that extends the classical ABM paradigm to include multilevel concepts. Over the past decade, ML-ABM has been increasingly adopted and explored by researchers as an effective paradigm for framing and defining the mechanisms underlying multilevel dynamics. However, due to the youth of the field, no single definition, methodology, or tool unifies studies in this rapidly expanding area. This review will begin with an introduction to socio-environmental systems (SES) and the challenges that modeling approaches face in representing them properly, especially regarding the complexity of human behaviors and organizations. ABM presents opportunities for modeling SESs with respect to these challenges, including the simulation of individual and social behavior and their ability to provide a descriptive and generative representation of the simulated system. However, ABM is limited in its ability to represent levels and scales, as these concepts are absent from the classical ABM metamodel. A complete review of the ML-ABM literature will be carried out, structured around a continuum that emerged during the review: that of the distribution of behaviors (and thus, from a software engineering perspective, of control) across the levels, from approaches that allow only one level to be active at a time, to approaches that rely on simultaneous activity and feedback loops between several levels. Different design choices will, thus, be presented to meet the different needs of multi-level representation, focusing on the interest on modelers and the strengths and limitations of each. In particular, we will highlight a limitation shared by all the reviewed approaches, namely their inability to represent several parallel hierarchies of levels and their interactions, a capability that appears more and more crucial to finely represent social behaviors in SES. A new perspective on the interest that the AGR approach could represent to allow this representation of hierarchies allows us to conclude on the research perspectives are still open.