Chimeric antigen receptor T-cell (CAR-T) therapy is the result of combining genetic engineering-based cancer immunotherapy with adoptive cell therapy (ACT). CAR-T therapy has been successful in treating various types of hematological cancers. CARs are receptors made of an extracellular domain, a membrane-spanning domain, and an intracellular domain. The extracellular domain of CARs harbors an antigen-targeting domain responsible for recognizing and binding cell surface-expressed target antigens. Conventionally, the single-chain fragment variable (scFv) of a monoclonal antibody (mAb) is used as the antigen-targeting domain of CARs. However, of late, researchers have exploited nanobodies for this aim based on numerous rationales including the small size of nanobodies, their stability, specificity, and high affinity, and their easy and feasible development process. Many findings have confirmed that nanobody-based CAR-Ts can be as functional as scFv-based CAR-Ts in preclinical and clinical settings. In this review, we discuss the advantages and disadvantages of scFvs and nanobodies in regards to their application as the targeting domain of CARs. Ultimately, we discuss various CAR target antigens which have been targeted using nanobody-based CAR-T cells for the treatment of different types of malignancies.