The #MeToo movement spurred a handful of corresponding #HeToo cases, in which men voiced their experiences of sexual harassment by female aggressors. The general public seemed to respond differently to #HeToo cases compared to the #MeToo cases. As the #HeToo and #MeToo cases differed in morally relevant ways, a direct comparison of public reactions cannot comprehensively explore whether peoples’ evaluation of sexual harassment is influenced by the gender of the aggressor and victim. We performed two experiments (total N = 260), in which participants read a description of sexual harassment from a manager toward an employee. For half of the participants, the manager was referred to as a man, and the employee was described as a woman; for the other half, the manager was described as a woman, and the employee was referred to as a man. We expected participants to judge the case as significantly more harmful and repugnant when the aggressor was a man and the victim was a woman compared to the opposite scenario. However, our analyses revealed a pattern of gender differences. Female participants perceived the case as equally bad whether the aggressor was male or female. Male participants reacted equally strongly to the case of sexual harassment performed by a male aggressor. By contrast, male participants viewed the case as far less serious and warranting milder reactions when the aggressor was female and the victim was male. This suggests the trivialization of sexual harassment of male victims by female aggressors is largely driven by men.