Experiments on high-quality GaAs (311)A two-dimensional holes at low temperatures reveal a remarkable dependence of the magnetoresistance, measured with an in-plane magnetic field ( B), on the direction of B relative to both the crystal axes and the current direction. The magnetoresistance features, and in particular the value of B above which the resistivity exhibits an insulating behavior, depend on the orientation of B. To explain the data, the anisotropic band structure of the holes and a repopulation of the spin subbands in the presence of B, as well as the coupling of the orbital motion to B, need to be taken into account.