An emerging body of research suggests that poor student performance on certain physics questions may stem, at least in part, from the nature of human reasoning itself. While students may demonstrate that they possess the requisite knowledge and skills to reason correctly on one question, they may abandon that same line of reasoning on an analogous question containing a salient distracting feature. As part of a larger effort to investigate and support student reasoning in physics by leveraging dual-process theories of reasoning, we developed and tested an intervention aimed at helping students draw upon the knowledge and skills they already possess to address such reasoning inconsistencies. In this study, we also explored specific factors to see if they were related to student reasoning and how students engage with the intervention. We found that the intervention was effective in helping students reason more productively and consistently, but its effectiveness appears to be related to students' cognitive reflection skills. In addition, out of the students who initially answered two analogous physics questions inconsistently, those who were able to successfully apply their correct reasoning from one question to the other question upon explicit prompting were more likely to revise their thinking and demonstrate consistent reasoning after the intervention.