BackgroundMutations in the IDH1 gene have been shown to be an important driver in the development of acute myeloid leukemia, gliomas and certain solid tumors, which is a promising target for cancer therapy.MethodsBidirectional recurrent neural network (BRNN) and scaffold hopping methods were used to generate new compounds, which were evaluated by principal components analysis, quantitative estimate of drug-likeness, synthetic accessibility analysis and molecular docking. ADME prediction, molecular docking and molecular dynamics simulations were used to screen candidate compounds and assess their binding affinity and binding stability with mutant IDH1 (mIDH1).ResultsBRNN and scaffold hopping methods generated 3890 and 3680 new compounds, respectively. The molecules generated by the BRNN performed better in terms of molecular diversity, druggability, synthetic accessibility and docking score. From the 3890 compounds generated by the BRNN model, 10 structurally diverse drug candidates with great docking score were preserved. Molecular dynamics simulations showed that the RMSD of the four systems, M1, M2, M3 and M6, remained stable, with local flexibility and compactness similar to the positive drug. The binding free energy results indicated that compound M1 exhibited the best binding properties in all energy aspects and was the best candidate molecule among the 10 compounds.ConclusionIn present study, compounds M1, M2, M3 and M6 generated by BRNN exhibited optimal binding properties. This study is the first attempt to use deep learning to design mIDH1 inhibitors, which provides theoretical guidance for the design of mIDH1 inhibitors.