Microorganisms can efficiently navigate in anisotropic complex fluids, but the precise swimming mechanisms remain largely unexplored. Their dynamics is determined by the interplay between multiple effects, including the fluid's orientation order, swimmer's undulatory gait and the finite length. Here, we extend the numerical study of the two-dimensional undulatory motions of a flexible swimmer in lyotropic liquid-crystalline polymers (LCPs) by Lin et al. (J. Fluid Mech., vol. 921, 2021, p. A25) to the scenario of arbitrary swimming directions with respect to the nematic director. The swimmer is modelled as a nearly inextensible yet flexible fibre with imposed travelling-wave-like actuation. We investigate the orientation-dependent swimming behaviours in nematic LCPs for an infinitely long sheet (i.e. Taylor's swimming sheet model) and finite-length swimmers. We demonstrate that the swimmer must be sufficiently stiff to produce undulatory deformations to gain net motions. Moreover, a motile finite-length swimmer can reorient itself to swim parallel with the nematic director, due to a net body torque arising from the asymmetric distribution of the polymer force along the body.