The reflex motion and distortion of the Milky Way (MW) halo caused by the infall of a massive Large Magellanic Cloud (LMC) has been demonstrated to result in an excess of orbital poles of dark matter halo particles toward the LMC orbital pole. This was suggested to help explain the observed preference of MW satellite galaxies to coorbit along the Vast Polar Structure (VPOS). We test this idea by correcting the positions and velocities of the MW satellites for the Galactocentric-distance-dependent shifts inferred from a LMC-infall simulation. While this should substantially reduce the observed clustering of orbital poles if it were mainly caused by the LMC, we instead find that the strong clustering remains preserved. We confirm the initial study’s main result with our simulation of an MW-LMC-like interaction, and use it to identify two reasons why this scenario is unable to explain the VPOS: (1) the orbital pole density enhancement in our simulation is very mild (∼10% within 50–250 kpc) compared to the observed enhancement (∼220%–300%), and (2) it is very sensitive to the specific angular momenta (AM) of the simulation particles, with the higher-AM particles being affected the least. Particles in simulated dark matter halos tend to follow more radial orbits (lower AM), so their orbital poles are more easily affected by small offsets in position and velocity caused by a LMC infall than objects with more tangential velocity (higher AM), such as the observed dwarf galaxies surrounding the MW. The origin of the VPOS thus remains unexplained.