When searching for solutions to a problem, people often rely on the observation of their peers. How does this process of social learning impact the individual and the group's performance? On the one hand, research has shown that individuals benefit from social learning in numerous situations and across many domains. Through social learning, individuals can access good solutions found by others, improve them, and share them in turn. On the other hand, this individual benefit may come at a cost: An excessive tendency to copy others often decreases the overall exploration volume of the group, thus reducing the diversity of discovered solutions, and eventually impairing the collective performance.Here we investigate the conditions under which social learning can be beneficial or detrimental to individuals and to the group. For that, we model problem-solving as a search task and simulate various amounts of social learning. We avoid model specific considerations by relying on a simple framework whereby individuals gradually explore the search environment -a two-dimensional landscape of solutions -while being attracted to the best solution of the group. Our results highlight a collective search dilemma: When group members learn from one another, they tend to improve their own individual performance at the expense of the collective performance. How is this dilemma affected by the structure of the search environment? By varying two structural aspects of the search environment, our results reveal that the negative effect of the dilemma is mitigated in more difficult environments. Finally, we show that single individuals can profit from a high propensity of social learning, which in turn is damaging for the other group members. As a consequence, if individuals continually adapt their behavior to maximize their own payoff, groups converge to a sub-optimal level of social learning. Unraveling these intricate social dynamics helps to understand the complex picture of collective problem-solving.