Fosmidomycin (FOS) is a naturally occurring compound active against the 1-deoxy-D-xylulose 5-phosphate reductoisomerase (DXR) enzyme in the 2-C-methyl-D-erythritol 4-phosphate (MEP) pathway, and using it as a template for lead structure design is an effective strategy to develop new active compounds. In this work, by replacing the hydroxamate unit of FOS with pyrazole, isoxazole and the related heterocycles that also have metal ion binding affinity, while retaining the monophosphonic acid in FOS or replacing it with a bisphosphonic acid group, heterocycle-containing mono- and bisphosphonic acid compounds as FOS analogs were designed. The key steps involved in the facile synthesis of these FOS analogs included the Michael addition of diethyl vinylphosphonate or tetraethyl vinylidenebisphosphonate to β-dicarbonyl compounds and the subsequent cyclic condensation with hydrazine or hydroxylamine. Two additional isoxazolinone-bearing FOS analogs were synthesized via the Michaelis–Becker reaction with diethyl phosphite as a key step. The bioactivity evaluation on model plants demonstrated that several compounds have better herbicidal activities compared to FOS, with the most active compound showing a 3.7-fold inhibitory activity on Arabidopsis thaliana, while on the roots and stalks of Brassica napus L. and Echinochloa crus-galli in a pre-emergence inhibitory activity test, the activities of this compound were found to be 3.2- and 14.3-fold and 5.4- and 9.4-fold, respectively, and in a post-emergency activity test on Amaranthus retroflexus and Echinochloa crus-galli, 2.2- and 2.0-fold inhibition activities were displayed. Despite the significant herbicidal activity, this compound exhibited a DXR inhibitory activity lower than that of FOS but comparable to that of other non-hydroxamate DXR inhibitors, and the dimethylallyl pyrophosphate rescue assay gave no statistical significance, suggesting that a different target might be involved in the inhibiting process. This work demonstrates that using bioisosteric replacement can be considered as a valuable strategy to discover new FOS analogs that may have high herbicidal activities.