Recently, a high-throughput screen of 1900 clinically used drugs identified masitinib, an orally bioavailable tyrosine kinase inhibitor, as a potential treatment for COVID-19. Masitinib acts as a broad-spectrum inhibitor for human coronaviruses, including SARS-CoV-2 and several of its variants. In this work, we rely on atomistic molecular dynamics simulations with advanced sampling methods to develop a deeper understanding of masitinib’s mechanism of Mpro inhibition. To improve the inhibitory efficiency and to increase the ligand selectivity for the viral target, we determined the minimal portion of the molecule (fragment) that is responsible for most of the interactions that arise within the masitinib-Mpro complex. We found that masitinib forms highly stable and specific H-bond interactions with Mpro through its pyridine and aminothiazole rings. Importantly, the interaction with His163 is a key anchoring point of the inhibitor, and its perturbation leads to ligand unbinding within nanoseconds. Based on these observations, a small library of rationally designed masitinib derivatives (M1–M5) was proposed. Our results show increased inhibitory efficiency and highly reduced cytotoxicity for the M3 and M4 derivatives compared to masitinib.