We examine a two-player game with two-armed exponential bandits à la (Keller et al. in Econometrica 73:39-68, 2005), where players operate different technologies for exploring the risky option. We characterise the set of Markov perfect equilibria and show that there always exists an equilibrium in which the player with the inferior technology uses a cutoff strategy. All Markov perfect equilibria imply the same amount of experimentation but differ with respect to the expected speed of the resolution of uncertainty. If and only if the degree of asymmetry between the players is high enough, there exists a Markov perfect equilibrium in which both players use cutoff strategies. Whenever this equilibrium exists, it welfare dominates all other equilibria. This contrasts with the case of symmetric players, where there never exists a Markov perfect equilibrium in cutoff strategies. Keywords Two-armed bandit • Heterogeneous agents • Free riding • Learning JEL Classification C73 • D83 • O31 The second author gratefully acknowledges support from the Social Sciences and Humanities Research Council of Canada. Part of the results presented in this paper was already contained in the third author's undergraduate thesis, entitled "Strategisches Experimentieren mit asymmetrischen Spielern", which she submitted at the University of Munich in 2009 under her maiden name Tönjes.