Stegenga (forthcoming) formulates and defends a novel account of scientific progress, according to which science makes progress just in case there is a change in scientific justification. Here, we present several problems for Stegenga’s account, concerning, respectively, (i) obtaining misleading evidence, (ii) losses or destruction of evidence, (iii) oscillations in scientific justification, and (iv) the possibility of scientific regress. We conclude by sketching a substantially different justification-based account of scientific progress that avoids these problems.