Both Src homology-2 domain-containing phosphatase 2 (SHP2) and histone deacetylase (HDAC) are important oncoproteins and potential immunomodulators. In this study, we first observed a synergistic antiproliferation effect of an allosteric SHP2 inhibitor (SHP099) and HDAC inhibitor (SAHA) in MV4-11 cells. Inspired by this result, a series of SHP2/HDAC dual inhibitors were designed based on the pharmacophore fusion strategy. Among these inhibitors, the most potent compound 8t showed excellent inhibitory activities against SHP2 (IC50 = 20.4 nM) and HDAC1 (IC50 = 25.3 nM). In particular, compound 8t exhibited improved antitumor activities compared with those of SHP099 and SAHA in vitro and in vivo. Our study also indicated that treatment with 8t could trigger efficient antitumor immunity by activating T cells, enhancing the antigen presentation function and promoting cytokine secretion. To our knowledge, we report the first small molecular SHP2/HDAC dual inhibitor and demonstrate a new strategy for cancer immunotherapy.