Hematopoietic stem cells (HSCs) and multipotent progenitors (MPPs) are crucial for maintaining lifelong hematopoiesis. Developing methods to distinguish stem cells from other progenitors and evaluate stem cell functions has been a central task in stem cell research. Deep learning has been demonstrated as a powerful tool in cell image analysis and classification. In this study, we explored the possibility of using deep learning to differentiate HSCs and MPPs based on their light microscopy (DIC) images. After extensive training and validation with large image data sets, we successfully develop a three-class classifier (we named it LSM model) that reliably differentiate long-term HSCs (LT-HSCs), short-term HSCs (ST-HSCs), and MPPs. Importantly, we demonstrated that our LSM model achieved its differentiating capability by learning the intrinsic morphological features from cell images. Furthermore, we showed that the performance of our LSM model was not affected by how these cells were identified and isolated, i.e., sorted by surface markers or intracellular GFP markers. Prospective identification of HSCs and MPPs in Evi1GFP transgenic mice by our LSM model suggested that the cells with the highest GFP expression were LT-HSCs, and this prediction was substantiated later by a long-term competitive reconstitution assay. Moreover, based on DIC image data sets, we also successfully built another two-class classifier that can effectively distinguish aged HSCs from young HSCs, which both express same surface markers but functionally different. This finding is of particular interest since it may provide a novel quick and efficient approach, obviating the need for a time-consuming transplantation experiment, to evaluate functional states of HSCs. Together, our study provides evidence for the first time that HSCs and MPPs can be differentiated by deep learning based on cell morphology. This novel and robust deep-learning-based platform will provide a basis for the future development of a new-generation stem cell identification and separation system. It may also provide new insight into molecular mechanism underlying self-renewal feature of stem cell.