Rhodanine and its derivatives are an important class of heterocycles with diverse biological properties, including anticancer, antibacterial, and anti‐mycobacterial activities. In the present work, four series of new Rhodanine derivatives were synthesized and evaluated for their inhibitory activity against carbonic anhydrase I, II, IX, and XII isoforms. Interestingly, the tested compounds exhibited good inhibitory activity against the cytosolic isoform human carbonic anhydrase (hCA) II and tumor‐associated hCA IX. While the Rhodanine‐benzylidene derivatives (3a–l) and Rhodanine‐hydrazine derivatives (6a–e) are found to be selective against hCA II, the Rhodanine‐N‐carboxylate derivatives (8a–d) are found to be highly selective toward hCA IX. The Rhodanine‐linked isoxazole and 1,2,4‐oxadiazole derivatives (8ba, 8da, and 8db) exhibited inhibitory activity against hCA II and hCA IX. Among the tested compounds, 3b, 3j, 6d, and 8db were found to inhibit hCA II with Ki values of 9.8, 46.4, 7.7, and 4.7 µM, respectively. Furthermore, their mechanism of action is supported by molecular docking studies. Notably, the synthesized Rhodanine derivatives belong to a nonsulfonamide class of carbonic anhydrase inhibitors.