Compared with traditional therapies, targeted therapy has merits in selectivity, efficacy, and tolerability. Small molecule inhibitors are one of the primary targeted therapies for cancer. Due to their advantages in a wide range of targets, convenient medication, and the ability to penetrate into the central nervous system, many efforts have been devoted to developing more small molecule inhibitors. To date, 88 small molecule inhibitors have been approved by the United States Food and Drug Administration to treat cancers. Despite remarkable progress, small molecule inhibitors in cancer treatment still face many obstacles, such as low response rate, short duration of response, toxicity, biomarkers, and resistance. To better promote the development of small molecule inhibitors targeting cancers, we comprehensively reviewed small molecule inhibitors involved in all the approved agents and pivotal drug candidates in clinical trials arranged by the signaling pathways and the classification of small molecule inhibitors. We discussed lessons learned from the development of these agents, the proper strategies to overcome resistance arising from different mechanisms, and combination therapies concerned with small molecule inhibitors. Through our review, we hoped to provide insights and perspectives for the research and development of small molecule inhibitors in cancer treatment.