Education plays an indispensable role in fostering societal well-being and is widely regarded as one of the most influential factors in shaping the future of generations to come. As artificial intelligence (AI) becomes more deeply integrated into our daily lives and the workforce, educational institutions at all levels are directing their focus on resources that cater to AI education. Yet, informal education, including online learning on social media platforms like YouTube, plays an increasingly significant role for both students and the general public. Offering greater accessibility compared to formal education, millions of individuals use YouTube for educational resources on AI today. Due to the substantial societal impact of AI, it is crucial for introductory AI courses to meaningfully address the ethical implications associated with AI. Our work investigates the current landscape of introductory AI courses on YouTube, and the potential for introducing ethics in this context. We qualitatively analyze the 20 most watched introductory AI courses on YouTube, coding a total of 92.2 hours of educational content viewed by close to 50 million people. We find that these introductory AI courses do not meaningfully engage with ethical or societal challenges of AI (RQ1). When defining and framing AI, introductory AI courses foreground excitement around AI's transformative role in society, over-exaggerate AI's current and future abilities, and anthropomorphize AI (RQ2). In teaching AI, we see a widespread reliance on corporate AI tools and frameworks as well as a prioritization on a hands-on approach to learning rather than on conceptual foundations (RQ3). In promoting key AI practices, introductory AI courses abstract away entirely the socio-technical nature of AI classification and prediction, for example by favoring data quantity over data quality (RQ4). Given the power of openly available introductory courses to shape enduring beliefs around AI and its field at the onset of a learning journey, we extend our analysis with recommendations that aim to integrate ethical reflections into introductory AI courses. We recommend that introductory AI courses