Chimeric antigen receptor (CAR) T cell therapy shows promise in treating malignant tumors. However, the use of human epidermal growth factor receptor-2 (HER2) CAR-T cells carries the risk of severe toxicity, including cytokine release syndrome, due to their “on-target off-tumor” recognition of HER2. Enhancing the quality and functionality of HER2 CARs could greatly improve the therapeutic potential of CAR-T cells. In this study, we developed a novel anti-HER2 monoclonal antibody, Ab8, which targets domain III of HER2, distinct from the domain IV recognition of trastuzumab. Although two anti-HER2 mAbs induced similar levels of antibody-dependent cellular cytotoxicity, trastuzumab-based CAR-T cells exhibited potent antitumor activity against HER2-positive cancer cells. In conclusion, our findings provide scientific evidence that antibody recognition of the membrane-proximal domain promotes the anti-tumor response of HER2-specific CAR-T cells.