Sirtuins are a class of enzymes with nicotinamide adenine dinucleotide (NAD)-dependent protein lysine deacylase function. By deacylating various substrate proteins, including histones, transcription factors, and metabolic enzymes, sirtuins regulate various biological processes, such as transcription, cell survival, DNA damage and repair, and longevity. Small molecules that can inhibit sirtuins have been developed and many of them have shown anti-cancer activity. Here we summarize the major biological findings that connect sirtuins to cancer and the different types of sirtuin inhibitors developed. Interestingly, biological data suggest that sirtuins have both tumor-suppressing and tumor-promoting roles. However, most pharmacological studies with small molecule inhibitors suggest that inhibiting sirtuin is a promising anti-cancer strategy. We discuss possible explanations for this discrepancy and suggest possible future directions to further establish sirtuin inhibitors as anticancer agents.