Boosting the number of women and girls entering careers involving STEM (Science, Technology, Engineering and Maths) is crucial to achieving gender equality, one of the UN Sustainable Development Goals. Girls and women tend to gravitate away from STEM fields at multiple stages from childhood through mid-career. The leaky pipeline is a metaphor often used to describe the loss of women in STEM and arguably other fields before reaching senior roles. Do interests expressed on social media mirror the leaky pipeline phenomenon? In this article, we collected advertisement data (reach estimates) from Facebook and Instagram disaggregated by US metros, age, gender, and interests related to STEM. We computed the Gender Gap Index (GGI) for each US metro and age group. We found that on Instagram, the GGIs for interest in Science decrease as users’ age increases, suggesting that relatively there is evidence that that women, compared to men, are losing interest in STEM at older ages. In particular, we find that on Instagram, there are plausible relative trends but implausible absolute levels. Nevertheless, is this enough to conclude that online data available from Instagram mirror the leaky pipeline phenomenon? To scrutinize this, we compared the GGIs for an interest in Science with the GGIs for placebo interests unrelated to STEM. We found that the GGIs for placebo interests follow similar age patterns as the GGIs for the interest in Science across US metros. Second, we attempted to control for the time spent on the platform by computing a usage intensity gender ratio based on the difference between daily and monthly active users. This analysis showed that the usage intensity gender ratio is higher among teenagers (13–17 years) than other older age groups, suggesting that teenage girls are more engaged on the platform that teenage boys. We hypothesize that usage intensity differences, rather than inherent interest changes, might create the illusion of a leaky pipeline. Despite the previously demonstrated value and huge potential of social media advertisement data to study social phenomena, we conclude that there is little evidence that this novel data source can measure the decline in interest in STEM for young women in the USA.