Previous research has shown that people hold an implicit gender inversion assumption regarding descriptive stereotypes of gay men and lesbian women. More specifically, people believe that the hobbies and preferences of gay men are similar to straight women, while lesbian women’s hobbies and preferences are similar to straight men. However, research to date has not provided a full understanding regarding the competencies of gay men and lesbian women when they engage in gender counter-normative activities. Across five studies (two pre-registered, N = 2454) we show a divergence between what gay men and lesbian women are expected to do and how well they are expected to do them. Gender inversion did not adequately explain in what domains people anticipated competence from gay men and lesbian women. Instead, gay men and lesbian women were seen as relatively incompetent compared to straight men and women across occupations (Studies 1A – C), skills (Study 2), and general domains (Study 3), particularly by straight people (Studies 1C and 3). Furthermore, this assumption was found implicitly (Studies 2 and 3), as even gay and lesbian participants were more likely to associate straight people with competence and gay/lesbian people with incompetence. We discuss the implications of these findings for gender inversion theory and the study of stereotypes more broadly.