When examining rapid instructed task learning behaviorally, one out of two paradigms is usually used, the Inducer-Diagnostic (I-D) and the NEXT paradigm. Even though both paradigms are supposed to examine the same phenomenon of Automatic Effect of Instructions (AEI), there are some meaningful differences between them, notably in the size of the AEI. In the current work, we examined, in two pre-registered studies, the potential reasons for these differences in AEI size. Study 1 examined the influence of the data-analytic approach by comparing two existing relatively large data-sets, one from each paradigm (Braem et al., in Mem Cogn 47:1582–1591, 2019; Meiran et al., in Neuropsychologia 90:180–189, 2016). Study 2 focused on the influence of instruction type (concrete, as in NEXT, and abstract, as in I-D) and choice complexity of the task in which AEI-interference is assessed. We did that while using variants of the NEXT paradigm, some with modifications that approximated it to the I-D paradigm. Results from Study 1 indicate that the data-analytic approach partially explains the differences between the paradigms in terms of AEI size. Still, the paradigms remained different with respect to individual differences and with respect to AEI size in the first step following the instructions. Results from Study 2 indicate that Instruction type and the choice complexity in the phase in which AEI is assessed do not influence AEI size, or at least not in the expected direction. Theoretical and study-design implications are discussed. Supplementary Information The online version contains supplementary material available at 10.1007/s00426-021-01596-1.