Accompanied by an X-ray burst, the fast radio burst FRB 200428 was recently confirmed to originate from the Galactic magnetar SGR J1935+2154. Just before FRB 200428 was detected, the Fivehundred-meter Aperture Spherical radio Telescope (FAST) had been monitoring SGR J1935+2154 for eight hours. From UTC 2020-04-27 23:55:00 to 2020-04-28 00:50:37, FAST detected no pulsed radio from SGR J1935+2154, while Fermi/GBM registered 34 SGR bursts in the X/soft γ-ray band, forming a unique sample of X-ray bursts in the absence of FRBs. After a comprehensive analysis on light curves, time-integrated and time-resolved spectral properties of these FRB-absent X-ray bursts, we compare this sample with the FRB-associated X-ray burst detected by Insight-HXMT. The FRB-associated burst distinguishes itself from other X-ray bursts by its non-thermal spectrum and a higher spectral peak energy, but is otherwise not atypical in many other aspects. We also compare the cumulative energy distribution of our X-ray burst sample with the cumulative radio energy distribution of first repeating FRB source, FRB 121102, with the calibration of FRB 200428-X-ray burst association. We find a similarity between the two, offering indirect support of the magnetar-origin of cosmological FRBs. The event rate density of magnetar bursts is about ∼ 150 times higher than the FRB event rate density at the energy of FRB 200428. This again suggests that only a small fraction of X-ray bursts are associated with FRBs if all FRBs originate from magnetars.