Accreting neutron stars (NSs) represent a unique laboratory for probing the physics of accretion in the presence of strong magnetic fields (B ≳ 108 G). Additionally, the matter inside the NS itself exists in an ultra-dense, cold state that cannot be reproduced in Earth-based laboratories. Hence, observational studies of these objects are a way to probe the most extreme physical regimes. Here we present an overview of the field and discuss the most important outstanding problems related to NS accretion. We show how these open questions regarding accreting NSs in both low-mass and high-mass X-ray binary systems can be addressed with the High-Energy X-ray Probe (HEX-P) via simulated data. In particular, with the broad X-ray passband and improved sensitivity afforded by a low X-ray background, HEX-P will be able to 1) distinguish between competing continuum emission models; 2) provide tighter upper limits on NS radii via reflection modeling techniques that are independent and complementary to other existing methods; 3) constrain magnetic field geometry, plasma parameters, and accretion column emission patterns by characterizing fundamental and harmonic cyclotron lines and exploring their behavior with pulse phase; 4) directly measure the surface magnetic field strength of highly magnetized NSs at the lowest accretion luminosities; as well as 5) detect cyclotron line features in extragalactic sources and probe their dependence on luminosity in the super-Eddington regime in order to distinguish between geometrical evolution and accretion-induced decay of the magnetic field. In these ways HEX-P will provide an essential new tool for exploring the physics of NSs, their magnetic fields, and the physics of extreme accretion.