We suggest a model of the advection-dominated accretion flow (ADAF) with magnetically driven outflows to explain the hysteretic state transition observed in X-ray binaries (XRBs). The transition from a thin disk to an ADAF occurs when the mass accretion rate is below a critical value. The critical mass accretion rate for the ADAF can be estimated by equating the equilibration timescale to the accretion timescale of the ADAF, which is sensitive to its radial velocity. The radial velocity of thin disks is very small, which leads to the advection of the external field in thin disks becoming very inefficient. ADAFs are present in the low/hard states of XRBs, and their radial velocity is large compared with the thin disk. The external field can be dragged inward efficiently by the ADAF, so a strong large-scale magnetic field threading the ADAF can be formed, which may accelerate a fraction of gas in the ADAF into the outflows. Such outflows may carry away a large amount of angular momentum from the ADAF, which significantly increases the radial velocity of the ADAF. This leads to a high critical mass accretion rate, below which an ADAF with magnetic outflows can survive. Our calculations show that the critical luminosity of the ADAF with magnetic outflows can be one order of magnitude higher than that for a conventional ADAF, if the ratio of gas to magnetic pressure 4 b~in the disk. This can naturally explain the hysteretic state transition observed in XRBs.