Mesenchymal stem cell (MSC) is a potential therapeutic material that has self-renewal, multilineage differentiation, and immunomodulation properties. However, the biological function of MSCs may decline due to the influence of donor differences and the in vitro expansion environment, which hinders the advancement of MSC-based clinical therapy. Here, we investigated a method for improving the immunomodulatory function of MSCs with the help of small-molecule compounds, A-83-01, CHIR99021, and Y27632 (ACY). The results showed that small-molecule induced MSCs (SM-MSCs) could enhance their immunosuppressive effects on T cells and macrophages. In vivo studies showed that, in contrast to control MSCs (Ctrl-MSCs), SM-MSCs could inhibit the inflammatory response in mouse models of delayed hypersensitivity and acute peritonitis more effectively. In addition, SM-MSCs showed the stronger ability to inhibit the infiltration of pro-inflammatory T cells and macrophages. Thus, small-molecule compounds ACY could better promote the immunomodulatory effect of MSCs, indicating it could be a potential improving method in MSC culture.