One of the most popular tasks used to test statistical learning (SL) involves asking participants to identify which of two stimuli, a triplet presented during the previous familiarization phase versus a new sequence made of the same stimuli never presented together, is more familiar based on the stream presented before, that is, to perform a two-alternative forced-choice (2-AFC) task. Despite the widespread use of this task, it has come under increasing criticism in current cognitive research due to psychometric flaws. A common practice to improve SL measurement involves increasing the number of 2-AFC trials by presenting the same items (triplets and foils) several times during the test phase. This work aimed to directly analyze the effect that this practice entails by examining how the proportion of correct discriminations of three-syllable nonsense words presented during the familiarization phase of an auditory triplet embedded task changed as the number of 2-AFC item repetitions increased. We also tested whether this effect was modulated by the predictability of the "words" embedded in the auditory streams (high and low) and the conditions under which they were presented to participants (implicit and explicit). Results showed that 2-AFC item repetitions had indeed detrimental effects on SL measurement, as indexed by a significant decrease in the proportion of correct discriminations as the number of items repetitions increased, both in the 2-AFC task performed under implicit and explicit conditions, although, in the first case, only for low-predictable "words." These findings recommend caution when using this strategy to improve SL measurement. Public Significance StatementDespite the widespread use of the 2-AFC task to test SL, it has come under increasing criticism. A common practice to improve SL measurement involves increasing the number of 2-AFC trials by repeating the same items during the test phase. Here, we demonstrated this practice to introduce other sources of "noise" into the SL measurement, hence recommending caution when using this strategy as an attempt to improve SL measurement.