The histone deacetylases (HDACs) occur in 11 different isoforms, and these enzymes regulate the activity of a large number of proteins involved in cancer initiation and progression. The discovery of isoform selective HDAC inhibitors (HDACIs) is desirable, as it is likely that such compounds would avoid some of the undesirable side effects found with the first generation inhibitors. A series of HDACIs previously reported by us were found to display some selectivity for HDAC6 and to induce cell cycle arrest and apoptosis in pancreatic cancer cells. In the present work, we show that structural modification of these isoxazole-based inhibitors leads to high potency and selectivity for HDAC6 over HDAC1-3 and HDAC10, while unexpectedly abolishing their ability to block cell growth. Three inhibitors with lower HDAC6 selectivity inhibit the growth of cell lines BxPC3 and L3.6pl, and they only induce apoptosis in L3.6pl. We conclude that HDAC6 inhibition alone is insufficient for disruption of cell growth, and that some degree of class 1 HDAC inhibition is required. Moreover, the highly selective HDAC6Is reported herein that are weakly cytotoxic may find use in cancer immune system reactivation.