Our understanding of the hierarchical structure of acute leukemia has yet to be fully translated into therapeutic approaches. Indeed, chemotherapy still has to take into account the possibility that leukemia-initiating cells may have a distinct chemosensitivity profile compared to the bulk of the tumor, and therefore are spared by the current treatment, causing the relapse of the disease. Therefore, the identification of the cell-of-origin of leukemia remains a longstanding question and an exciting challenge in cancer research of the last few decades. With a particular focus on acute lymphoblastic leukemia, we present in this review the previous and current concepts exploring the phenotypic, genetic and functional heterogeneity in patients. We also discuss the benefits of using engineered mouse models to explore the early steps of leukemia development and to identify the biological mechanisms driving the emergence of leukemia-initiating cells. Finally, we describe the major prospects for the discovery of new therapeutic strategies that specifically target their aberrant stem cell-like functions.