Non-B-cell acute leukemia is a term that encompasses T-cell acute lymphoblastic leukemia (T-ALL) and acute myeloid leukemia (AML). Currently, the therapeutic effectiveness of existing treatments for refractory or relapsed (R/R) non-B-cell acute leukemia is limited. In such situations, chimeric antigen receptor (CAR)-T cell therapy may be a promising approach to treat non-B-cell acute leukemia, given its promising results in B-cell acute lymphoblastic leukemia (B-ALL). Nevertheless, fratricide, malignant contamination, T cell aplasia for T-ALL, and specific antigen selection and complex microenvironment for AML remain significant challenges in the implementation of CAR-T therapy for T-ALL and AML patients in the clinic. Therefore, designs of CAR-T cells targeting CD5 and CD7 for T-ALL and CD123, CD33, and CLL1 for AML show promising efficacy and safety profiles in clinical trials. In this review, we summarize the characteristics of non-B-cell acute leukemia, the development of CARs, the CAR targets, and their efficacy for treating non-B-cell acute leukemia.