Contending with talker variability has been found to lead to processing costs but also benefits by focusing learners on invariant properties of the signal. These discrepant findings may indicate that talker variability acts as a desirable difficulty. That is, talker variability may lead to initial costs followed by long term benefits for retention and generalization. Adult participants learned an artificial grammar affording learning of multiple components by 1-, 2-or 8-talkers, tested at 3 time points. The 8-talker condition did not impact learning. The 2-talker condition negatively impacted some aspects of learning, but only under more difficult learning conditions. Across both experiments, generalization of the grammatical dependency was difficult. Taken together, we discovered that high and limited talker variability differentially impact artificial grammar learning. However, talker variability does not act as a desirable difficulty in the current paradigm, as the few evidenced costs were not related to long-term benefits.