This paper explores how the patterns of collaboration change throughout a scientist's career, especially their collaboration with so‐called “giants”—well‐known elite scholars with very high scientific impact. We divide scientist's careers into five stages based on the number of years since obtaining their PhD, and for each stage, we calculate a number of indicators, such as publication count, normalized citation count, count of collaborations, count of collaborations with “giants,” and percentage of uninterrupted and continuous presence. Our analysis shows that although collaborating with “giants” may benefit a young scholar, they are more likely to achieve greater scientific success if they first work independently and only collaborate with “giants” later in their career. Meanwhile, high‐impact authors tend to work with “giants” early in their career, but not in the later stages, which forces us to reconsider the established notions of when is best to collaborate with elite scholars. Moreover, we find that high‐impact authors tend to have fewer co‐authors per article, regardless of the stage of their careers, indicating that collaboration is important, but the size of collaborations also matters.