Binary stochastic neurons (BSN's) form an integral part of many machine learning algorithms, motivating the development of hardware accelerators for this complex function. It has been recognized that hardware BSN's can be implemented using low barrier magnets (LBM's) by minimally modifying presentday magnetoresistive random access memory (MRAM) devices. A crucial parameter that determines the response of these LBM based BSN designs is the correlation time of magnetization, τc. In this letter, we show that for magnets with low energy barriers (∆ ≈ kBT and below), circular disk magnets with inplane magnetic anisotropy (IMA) lead to τc values that are two orders of magnitude smaller compared to τc for magnets having perpendicular magnetic anisotropy (PMA) and provide analytical descriptions. We show that this striking difference in τc is due to a precession-like fluctuation mechanism that is enabled by the large demagnetization field in IMA magnets. We provide a detailed energy-delay performance evaluation of previously proposed BSN designs based on Spin-Orbit-Torque (SOT) MRAM and Spin-Transfer-Torque (STT) MRAM employing low barrier circular IMA magnets by SPICE simulations. The designs exhibit sub-ns response times leading to energy requirements of ∼a few fJ to evaluate the BSN function, orders of magnitude lower than digital CMOS implementations with a much larger footprint. While modern MRAM technology is based on PMA magnets, results in this paper suggest that low barrier circular IMA magnets may be more suitable for this application.Index Terms-Binary stochastic neuron, hardware implementation, low barrier magnet, embedded MTJ, probabilistic computing