Reverse annealing is a variant of quantum annealing, in which the system is prepared in a classical state, reverse-annealed to an inversion point, and then forward-annealed. We report on reverse annealing experiments using the D-Wave 2000Q device, with a focus on the p = 2 p-spin problem, which undergoes a second order quantum phase transition with a gap that closes polynomially in the number of spins. We concentrate on the total and partial success probabilities, the latter being the probabilities of finding each of two degenerate ground states of all spins up or all spins down, the former being their sum. The empirical partial success probabilities exhibit a strong asymmetry between the two degenerate ground states, depending on the initial state of the reverse anneal. To explain these results, we perform open-system simulations using master equations in the limits of weak and strong coupling to the bath. The former, known as the adiabatic master equation (AME), with decoherence in the instantaneous energy eigenbasis, predicts perfect symmetry between the two degenerate ground states, thus failing to agree with the experiment. In contrast, the latter, known as the polaron transformed Redfield equation (PTRE), is in close agreement with experiment. The same is true for a purely classical model known as spin-vector Monte Carlo with transverse-fielddependent updates (SVMC-TF). Thus our results present a strong challenge to the sufficiency of the weak system-bath coupling limit in describing the dynamics of current experimental quantum annealers, at least for annealing on timescales of a µsec or longer.